From report at bugs.python.org  Tue May  1 00:07:44 2018
From: report at bugs.python.org (Terry J. Reedy)
Date: Tue, 01 May 2018 04:07:44 +0000
Subject: [New-bugs-announce] [issue33396] IDLE: Improve and document help
 doc viewer
Message-ID: <1525147664.58.0.682650639539.issue33396@psf.upfronthosting.co.za>


New submission from Terry J. Reedy <tjreedy at udel.edu>:

The IDLE help doc viewer displays help.html in a tk Text subclass. This index issue follows #25198, which fixed three immediate issues, and left the following for later.  When tackled, the items below should be separate issues and PRs listed as dependencies.  

1. msg251181 Base font size on user config and control-mousewheel.
2. "       " Make within-document html links in work in the viewer.
3. "       " Make Find (^F) work within the viewer.
4. msg251571 Navigate with keys as in editor.
5. msg251556 Open TOC with Key (Esc?, F1?); enhance navigation.

New items:

6. Add Help Doc Viewer doc to Help section.
   I prefer not adding a menu and see an entry as an alternative.

7. 1, 3, and 4 should try to avoid duplicating editor code.  Perhaps we should enhance the plain text viewer and use it as a superclass for both base editor and html viewer.

----------
assignee: terry.reedy
components: IDLE
messages: 315982
nosy: csabella, terry.reedy
priority: normal
severity: normal
status: open
title: IDLE: Improve and document help doc viewer
type: enhancement
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33396>
_______________________________________

From report at bugs.python.org  Tue May  1 01:46:36 2018
From: report at bugs.python.org (Terry J. Reedy)
Date: Tue, 01 May 2018 05:46:36 +0000
Subject: [New-bugs-announce] [issue33397] IDLE help viewer: let users
 control font size
Message-ID: <1525153596.77.0.682650639539.issue33397@psf.upfronthosting.co.za>


New submission from Terry J. Reedy <tjreedy at udel.edu>:

Dependency of #33396. 

1. Base the initial font sizes on the on the configured editor font size.  Note that editor font size defaults to 10 while the current base size is 12.  I did not find any explicit discussion of this in the original issue, #16893, or the first followup, #25198.  Perhaps Mark Roseman just liked the result, or perhaps it made the relative sizing easier.

2. Change font size with Control-Mousewheel.  Manually tested code for doing this on Windows, Linux, and MacOS is in turtledemo.__main__.  The three systems are wildly different.  What we do here should allow easily doing same for other text windows.

----------
assignee: terry.reedy
components: IDLE
messages: 315985
nosy: terry.reedy
priority: normal
severity: normal
stage: patch review
status: open
title: IDLE help viewer: let users control font size
type: enhancement
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33397>
_______________________________________

From report at bugs.python.org  Tue May  1 07:51:56 2018
From: report at bugs.python.org (Jens Troeger)
Date: Tue, 01 May 2018 11:51:56 +0000
Subject: [New-bugs-announce] [issue33398] From, To,
 Cc lines break when calling send_message()
Message-ID: <1525175516.34.0.682650639539.issue33398@psf.upfronthosting.co.za>


New submission from Jens Troeger <jens.troeger at gmail.com>:

It looks like non-ascii characters in an Address()?s display_name parameter cause their lines in the header to get mangled when the message is being sent.  For example, a case to reproduce:

    >>> msg = EmailMessage()
    >>> msg["To"] = Address(display_name="Jens Tr?ger", addr_spec="jens.troeger at gmail.com")
    >>> msg["From"] = Address(display_name="Jens Troeger", addr_spec="jens.troeger at gmail.com")
    >>> msg.set_content("Some content.")
    >>> msg.as_string()
    'To: Jens =?utf-8?q?Tr=C3=B6ger?= <jens.troeger at gmail.com>\nContent-Type: text/plain; charset="utf-8"\nContent-Transfer-Encoding: 7bit\nMIME-Version: 1.0\nFrom: Jens Troeger <jens.troeger at gmail.com>\n\nSome content.\n'

Sending this email creates the following SMTP debug output:

    >>> smtpsrv = smtplib.SMTP("smtp.gmail.com:587")
    >>> ?
    >>> smtpsrv.send_message(msg)
    send: 'mail FROM:<jens.troeger at gmail.com> size=220\r\n'
    reply: b'250 2.1.0 OK z23sm16924622pfe.110 - gsmtp\r\n'
    reply: retcode (250); Msg: b'2.1.0 OK z23sm16924622pfe.110 - gsmtp'
    send: 'rcpt TO:<jens.troeger at gmail.com>\r\n'
    reply: b'250 2.1.5 OK z23sm16924622pfe.110 - gsmtp\r\n'
    reply: retcode (250); Msg: b'2.1.5 OK z23sm16924622pfe.110 - gsmtp'
    send: 'data\r\n'
    reply: b'354  Go ahead z23sm16924622pfe.110 - gsmtp\r\n'
    reply: retcode (354); Msg: b'Go ahead z23sm16924622pfe.110 - gsmtp'
    data: (354, b'Go ahead z23sm16924622pfe.110 - gsmtp')
    send: b'To: Jens =?utf-8?q?Tr=C3=B6ger?= <jens.troeger at gmail.com>\r\r\r\r\r\nContent-Type: text/plain; charset="utf-8"\r\nContent-Transfer- 
    Encoding: 7bit\r\nMIME-Version: 1.0\r\nFrom: Jens Troeger <jens.troeger at gmail.com>\r\n\r\nSome content.\r\n.\r\n'
    reply: b'250 2.0.0 OK 1525174591 z23sm16924622pfe.110 - gsmtp\r\n'
    reply: retcode (250); Msg: b'2.0.0 OK 1525174591 z23sm16924622pfe.110 - gsmtp'
    data: (250, b'2.0.0 OK 1525174591 z23sm16924622pfe.110 - gsmtp')
    {}

Notice the string of "\r\r\?" for the "To" field which consequently breaks off the remainder of the email?s header into a premature body:

    [?]
    Message-ID: <5ae8513e.17b9620a.eebf7.d56e at mx.google.com>                                                             
    Date: Tue, 01 May 2018 04:36:30 -0700 (PDT)                                                                          
    From: jens.troeger at gmail.com                                                                                               
    To: Jens Tr?ger <jens.troeger at gmail.com>                                                                             
                                                                                                                     
                                                                                                                     
                                                                                                                     
    Content-Type: text/plain; charset="utf-8"                                                                            
    Content-Transfer-Encoding: 7bit                                                                                      
    MIME-Version: 1.0                                                                                                    
    From: Jens Troeger <jens.troeger at gmail.com>                                                                          
                                                                                                                     
    Some content.                                                                                                        

Also notice the two From fields. The first one, I suspect, is supplied from the SMTP server?s login, the second one from them EmailMessage.  Without a From in the EmailMessage, I get the following error:

    >>> smtpsrv.send_message(msg)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "/?/lib/python3.6/smtplib.py", line 936, in send_message
        from_addr = email.utils.getaddresses([from_addr])[0][1]
      File "/?/lib/python3.6/email/utils.py", line 112, in getaddresses
        all = COMMASPACE.join(fieldvalues)
    TypeError: sequence item 0: expected str instance, NoneType found

Similar breakage of the header into premature body can be achieved with the Cc header field.

----------
components: email
messages: 315994
nosy: _savage, barry, r.david.murray
priority: normal
severity: normal
status: open
title: From, To, Cc lines break when calling send_message()
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33398>
_______________________________________

From report at bugs.python.org  Tue May  1 13:56:11 2018
From: report at bugs.python.org (Demian Brecht)
Date: Tue, 01 May 2018 17:56:11 +0000
Subject: [New-bugs-announce] [issue33399] site.abs_paths should handle None
 __cached__ type
Message-ID: <1525197371.8.0.682650639539.issue33399@psf.upfronthosting.co.za>


New submission from Demian Brecht <demianbrecht at gmail.com>:

Echoing an email sent to python-list:

I recently ran into an issue using the site package and I wanted to confirm that I'm not doing something wrong here before creating an issue:

I have a need to `git subtree` modules into a Django project and then add the dependencies such that they're discoverable in code. I also need to have consistent behavior between `python setup.py (develop|install)` and `pip install -r requirements.txt` (the latter is used by Heroku for project installation). An example project that demonstrates the issue is here: https://github.com/demianbrecht/python-sitehooks-example.

In the example, I've subtree'd requests into _vendor and forced site hooks to fire using `site.main()` in testme/settings.py: https://github.com/demianbrecht/python-sitehooks-example/commit/1b81e15c6f28bc80a4f984cffa78eb2ced80a320. Forcing the execution is not needed when the package is actually installed using setup.py, but is needed when using `pip install -r requirements.txt`. This works great when running within a virtualenv, but I get the following on system python:

$ python3.6 ./manage.py test
Traceback (most recent call last):
 File "./manage.py", line 22, in <module>
   execute_from_command_line(sys.argv)
 File "/usr/local/lib/python3.6/site-packages/django/core/management/__init__.py", line 364, in execute_from_command_line
   utility.execute()
 File "/usr/local/lib/python3.6/site-packages/django/core/management/__init__.py", line 308, in execute
   settings.INSTALLED_APPS
 File "/usr/local/lib/python3.6/site-packages/django/conf/__init__.py", line 56, in __getattr__
   self._setup(name)
 File "/usr/local/lib/python3.6/site-packages/django/conf/__init__.py", line 41, in _setup
   self._wrapped = Settings(settings_module)
 File "/usr/local/lib/python3.6/site-packages/django/conf/__init__.py", line 110, in __init__
   mod = importlib.import_module(self.SETTINGS_MODULE)
 File "/usr/local/lib/python3.6/importlib/__init__.py", line 126, in import_module
   return _bootstrap._gcd_import(name[level:], package, level)
 File "<frozen importlib._bootstrap>", line 978, in _gcd_import
 File "<frozen importlib._bootstrap>", line 961, in _find_and_load
 File "<frozen importlib._bootstrap>", line 950, in _find_and_load_unlocked
 File "<frozen importlib._bootstrap>", line 655, in _load_unlocked
 File "<frozen importlib._bootstrap_external>", line 678, in exec_module
 File "<frozen importlib._bootstrap>", line 205, in _call_with_frames_removed
 File "/<path>/<to>/testme/testme/settings.py", line 13, in <module>
   site.main()
 File "/usr/local/lib/python3.6/site.py", line 525, in main
   abs_paths()
 File "/usr/local/lib/python3.6/site.py", line 110, in abs_paths
   m.__cached__ = os.path.abspath(m.__cached__)
 File "/usr/local/lib/python3.6/posixpath.py", line 369, in abspath
   path = os.fspath(path)
TypeError: expected str, bytes or os.PathLike object, not NoneType

Using venv and system comparisons for debugging doesn't help much as site.py is different in the two environments. Any insight into what may be going on here or what I'm doing wrong would be much appreciated. I realize that I could just muck with PYTHONPATH, but I thought this approach would be a little nicer as it's transparent to the user.

Digging a little deeper, it seems that site.abs_paths fails when a module's __cached__ value is None. I haven't spent enough time digging into it to understand why this is valid behavior, hoping to get some insight from folks who have spent more time on it.

----------
components: Library (Lib)
messages: 316008
nosy: demian.brecht
priority: normal
severity: normal
status: open
title: site.abs_paths should handle None __cached__ type
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33399>
_______________________________________

From report at bugs.python.org  Tue May  1 15:37:24 2018
From: report at bugs.python.org (Paul Cyr)
Date: Tue, 01 May 2018 19:37:24 +0000
Subject: [New-bugs-announce] [issue33400] logging.Formatter does not default
 to ISO8601 date format
Message-ID: <1525203444.89.0.682650639539.issue33400@psf.upfronthosting.co.za>


New submission from Paul Cyr <python at paulcyr.ca>:

>From the docs:

https://docs.python.org/3.6/library/logging.html#logging.Formatter

"class logging.Formatter(fmt=None, datefmt=None, style='%') ...
If no datefmt is specified, the ISO8601 date format is used."

However, the output from the formatter when no datefmt is specified is not an ISO8601 date. It is also a datetime format, not a date format.

Example output: 2018-05-01 19:04:31,505

Not being an ISO member, I don't have access to the official source for the specification but, trusting other sources, it appears that the output is missing the required 'T' delimiter between the date and time, and it is also missing the timezone. A compliant output would be:

2018-05-01T19:04:31,505Z

The current output appears to actually be closer to RFC 3339, as that standards allows for a space instead of the 'T' delimiter, but RFC 3339 still requires a timezone.

Either the documentation should be updated to state that the format used is an RFC 3339 datetime but without a timezone, or the format should comply with the standard specified in the documentation.

----------
components: Library (Lib)
messages: 316012
nosy: paulc
priority: normal
severity: normal
status: open
title: logging.Formatter does not default to ISO8601 date format
type: behavior
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33400>
_______________________________________

From report at bugs.python.org  Tue May  1 16:32:55 2018
From: report at bugs.python.org (luav)
Date: Tue, 01 May 2018 20:32:55 +0000
Subject: [New-bugs-announce] [issue33401] `exec` attribute can't be set to
 objects in Python2 (SyntaxError)
Message-ID: <1525206775.24.0.682650639539.issue33401@psf.upfronthosting.co.za>


New submission from luav <luardev at gmail.com>:

The following code works fine on Python 3.5.2 but `exec` attribute fails to be set to the object on Python 2.7.12:
```python
>>> o = type('O', (object,), {})
>>> o.e = 1
>>> o.eval = 1
>>> o.exec = 1
  File "<stdin>", line 1
    o.exec = 1
         ^
SyntaxError: invalid syntax
```

OS Environments:  Ubuntu 16.04.4 LTS x64

----------
components: Interpreter Core
messages: 316015
nosy: luav
priority: normal
severity: normal
status: open
title: `exec` attribute can't be set to objects in Python2 (SyntaxError)
type: behavior
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33401>
_______________________________________

From report at bugs.python.org  Tue May  1 18:39:00 2018
From: report at bugs.python.org (Gabe Appleton)
Date: Tue, 01 May 2018 22:39:00 +0000
Subject: [New-bugs-announce] [issue33402] Change the fractions.Fraction
 class to convert to a unicode fraction string
Message-ID: <1525214340.23.0.682650639539.issue33402@psf.upfronthosting.co.za>


New submission from Gabe Appleton <gabe at gabeappleton.me>:

Currently it has a __repr__() which returns `Fraction(x, y)`, and a __str__() which returns `x/y`. I have a ready pull request to change this to a scheme where both return unicode fractions.

----------
components: Library (Lib)
messages: 316026
nosy: gappleto97
priority: normal
severity: normal
status: open
title: Change the fractions.Fraction class to convert to a unicode fraction string
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33402>
_______________________________________

From report at bugs.python.org  Tue May  1 20:33:54 2018
From: report at bugs.python.org (pyneda)
Date: Wed, 02 May 2018 00:33:54 +0000
Subject: [New-bugs-announce] [issue33403] asyncio.tasks.wait does not allow
 to set custom exception when return_when=FIRST_EXCEPTION
Message-ID: <1525221234.75.0.682650639539.issue33403@psf.upfronthosting.co.za>


Change by pyneda <rogerpineda13 at gmail.com>:


----------
components: asyncio
nosy: asvetlov, giampaolo.rodola, pyneda, yselivanov
priority: normal
severity: normal
status: open
title: asyncio.tasks.wait does not allow to set custom exception when return_when=FIRST_EXCEPTION
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33403>
_______________________________________

From report at bugs.python.org  Tue May  1 21:31:33 2018
From: report at bugs.python.org (Braiden Gole)
Date: Wed, 02 May 2018 01:31:33 +0000
Subject: [New-bugs-announce] [issue33404] Phone Number Generator
Message-ID: <1525224693.4.0.682650639539.issue33404@psf.upfronthosting.co.za>


New submission from Braiden Gole <bmxfiend1 at gmail.com>:

I was creating a small program to generate numbers for fun and I realized that my output is different from my input. The random phone number that I used is: (519-662-6963) and what I get is: (519-662-6953), where the second last digit changes, thought it would be worth looking at. Everything looks to be inline and in order of precedence but not sure what is going on.

----------
assignee: terry.reedy
components: IDLE
files: Phone_numbers.PNG
messages: 316031
nosy: Braiden Gole, terry.reedy
priority: normal
severity: normal
status: open
title: Phone Number Generator
type: behavior
versions: Python 3.6
Added file: https://bugs.python.org/file47559/Phone_numbers.PNG

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33404>
_______________________________________

From report at bugs.python.org  Wed May  2 02:18:14 2018
From: report at bugs.python.org (anthony shaw)
Date: Wed, 02 May 2018 06:18:14 +0000
Subject: [New-bugs-announce] [issue33405] PYTHONCOERCECLOCALE no longer
 being respected
Message-ID: <1525241894.06.0.682650639539.issue33405@psf.upfronthosting.co.za>


New submission from anthony shaw <anthony.p.shaw at gmail.com>:

observing a behaviour on Python 3.7 b2 that doesn't match what's documented in PEP 538

PEP 538 states that the locale coercion behaviour can be disabled through the PYTHONCOERCECLOCALE environment variable.
I would then expect the stdin encoding to be the same as Python 3.6 when the C locale is specified with no encoding value. 

bash-3.2$ LANG=C python3.6 -c "import sys; print(sys.stdin.encoding)"
US-ASCII
bash-3.2$ LANG=C python3.7 -c "import sys; print(sys.stdin.encoding)"
utf-8
bash-3.2$ PYTHONCOERCECLOCALE=0 LANG=C python3.7 -c "import sys; print(sys.stdin.encoding)"
utf-8

LC_ALL is not set

bash-3.2$ locale
LANG="C"
LC_COLLATE="C"
LC_CTYPE="C"
LC_MESSAGES="C"
LC_MONETARY="C"
LC_NUMERIC="C"
LC_TIME="C"
LC_ALL=

Trying to dig into the reason why the env flag isn't disabling the behaviour I found some subsequent changes after the PEP which look to have broken the original implementation behaviour.

https://github.com/python/cpython/commit/9454060e84a669dde63824d9e2fcaf295e34f687

----------
messages: 316045
nosy: anthony shaw
priority: normal
severity: normal
status: open
title: PYTHONCOERCECLOCALE no longer being respected
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33405>
_______________________________________

From report at bugs.python.org  Wed May  2 02:39:05 2018
From: report at bugs.python.org (Zuzu_Typ)
Date: Wed, 02 May 2018 06:39:05 +0000
Subject: [New-bugs-announce] [issue33406] [ctypes] increase refcount of a
 CFUNCTYPE instance when passing to a CDLL
Message-ID: <1525243145.23.0.682650639539.issue33406@psf.upfronthosting.co.za>


New submission from Zuzu_Typ <zuzu.typ at gmail.com>:

It should be considered to increase the refcount of functions that are passed on to an external library at some point.

If this is not done and e.g. the function was a local variable, when leaving the scope it gets garbage-collected. When the library now tries to execute said function, an access violation occurs.

So if the refcount were increased either upon creating the CFUNCTYPE instance or when it's passed on, this could be avoided.

----------
components: ctypes
messages: 316047
nosy: Zuzu_Typ
priority: normal
severity: normal
status: open
title: [ctypes] increase refcount of a CFUNCTYPE instance when passing to a CDLL
type: behavior

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33406>
_______________________________________

From report at bugs.python.org  Wed May  2 05:29:49 2018
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 02 May 2018 09:29:49 +0000
Subject: [New-bugs-announce] [issue33407] Implement Py_DEPRECATED() macro
 for Visual Studio
Message-ID: <1525253389.49.0.682650639539.issue33407@psf.upfronthosting.co.za>


New submission from STINNER Victor <vstinner at redhat.com>:

According to the following links, it would be possible to implement the Pc_DEPRECATED() macro for Visual Studio:

https://mail.python.org/pipermail/python-dev/2018-May/153299.html
https://stackoverflow.com/questions/295120/c-mark-as-deprecated/295229#295229

"""
#ifdef __GNUC__
#define DEPRECATED(func) func __attribute__ ((deprecated))
#elif defined(_MSC_VER)
#define DEPRECATED(func) __declspec(deprecated) func
#else
#pragma message("WARNING: You need to implement DEPRECATED for this compiler")
#define DEPRECATED(func) func
#endif
"""

Moreover, is Py_DEPRECATED() defined on clang which also supports the deprecated function attribute?
https://clang.llvm.org/docs/AttributeReference.html#deprecated-gnu-deprecated

Current Include/pyport.h code:

#if defined(__GNUC__) \
    && ((__GNUC__ >= 4) || (__GNUC__ == 3) && (__GNUC_MINOR__ >= 1))
#define Py_DEPRECATED(VERSION_UNUSED) __attribute__((__deprecated__))
#else
#define Py_DEPRECATED(VERSION_UNUSED)
#endif

----------
components: Windows
messages: 316059
nosy: paul.moore, pitrou, steve.dower, tim.golden, vstinner, zach.ware
priority: normal
severity: normal
status: open
title: Implement Py_DEPRECATED() macro for Visual Studio
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33407>
_______________________________________

From report at bugs.python.org  Wed May  2 06:35:32 2018
From: report at bugs.python.org (Filip *)
Date: Wed, 02 May 2018 10:35:32 +0000
Subject: [New-bugs-announce] [issue33408] AF_UNIX is now supported in Windows
Message-ID: <1525257332.06.0.682650639539.issue33408@psf.upfronthosting.co.za>


New submission from Filip * <filip.stamcar at hotmail.com>:

Unix socket (AF_UNIX) is now avalible in Windows 10 (April 2018 Update). Please add Python support for it.
More details about it on https://blogs.msdn.microsoft.com/commandline/2017/12/19/af_unix-comes-to-windows/

----------
components: Windows
messages: 316067
nosy: filips123, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: AF_UNIX is now supported in Windows
type: enhancement
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33408>
_______________________________________

From report at bugs.python.org  Wed May  2 07:33:42 2018
From: report at bugs.python.org (Nick Coghlan)
Date: Wed, 02 May 2018 11:33:42 +0000
Subject: [New-bugs-announce] [issue33409] Clarify the interaction between
 locale coercion & UTF-8 mode
Message-ID: <1525260822.94.0.682650639539.issue33409@psf.upfronthosting.co.za>


New submission from Nick Coghlan <ncoghlan at gmail.com>:

While Victor and I reconciled the respective designs & implementations of PEP 538/540 (with 538 relying on 540 to handle the "No suitable target locale" case, and 540 relying on 538 to handle extension modules like GNU readline), the interaction between the two mechanisms currently isn't clear in the documentation.

So, before 3.7rc1 goes out, we should:

1. Combine the presentation of the two PEPs in the What's New documentation to make it clearer that they're tackling different aspects of the same problem
2. Clarify in the reference docs for PYTHONCOERCECLOCALE and PYTHONUTF8MODE that they're independent settings (so you have to explicitly turn them *both* off if you truly want to force ASCII based text handling in the C or POSIX locale)



(Issue raised based on a Twitter discussion with Anthony Shaw at https://twitter.com/anthonypjshaw/status/991614899791933441)

----------
assignee: ncoghlan
components: Documentation
messages: 316069
nosy: ncoghlan, vstinner
priority: high
severity: normal
stage: needs patch
status: open
title: Clarify the interaction between locale coercion & UTF-8 mode
type: enhancement
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33409>
_______________________________________

From report at bugs.python.org  Wed May  2 10:22:10 2018
From: report at bugs.python.org (alex)
Date: Wed, 02 May 2018 14:22:10 +0000
Subject: [New-bugs-announce] [issue33410] Using type in a format with
 padding causes TypeError
Message-ID: <1525270930.67.0.682650639539.issue33410@psf.upfronthosting.co.za>


New submission from alex <alex.payne22 at gmail.com>:

When trying to print a type in a formatted string with padding TypeError is raised. See examples below.

These work:
>>> a = 'abc'
>>> print('{a}'.format(a=type(a)))
<class 'str'>
>>> print('{a}'.format(a=str(type(a))))
<class 'str'>

These don't:
>>> print('{a: >10}'.format(a=type(a)))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unsupported format string passed to type.__format__
>>> t = type(a)
>>> print('{a: >10}'.format(a=t))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unsupported format string passed to type.__format__

----------
messages: 316072
nosy: alexomics
priority: normal
severity: normal
status: open
title: Using type in a format with padding causes TypeError
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33410>
_______________________________________

From report at bugs.python.org  Wed May  2 10:24:09 2018
From: report at bugs.python.org (Quentin Millardet)
Date: Wed, 02 May 2018 14:24:09 +0000
Subject: [New-bugs-announce] [issue33411] All console message are in the
 error output in bash interpretor
Message-ID: <1525271049.22.0.682650639539.issue33411@psf.upfronthosting.co.za>


New submission from Quentin Millardet <qmillardet at gmail.com>:

The probleme is all display (normal and error message), in bash, are send to the screen by the error output. So it's impossible when someone make a bash script to get the error back in a log file for exemple, or just to display only the error on a screen.

Problem test on Ubuntu 18.04 and Elementary loki (an Ubuntu 16.04 variant)

----------
components: IO
files: Capture d'?cran du 2018-05-02 16.16.09.png
messages: 316073
nosy: Quentin Millardet
priority: normal
severity: normal
status: open
title: All console message are in the error output in bash interpretor
type: behavior
versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8
Added file: https://bugs.python.org/file47560/Capture d'?cran du 2018-05-02 16.16.09.png

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33411>
_______________________________________

From report at bugs.python.org  Wed May  2 13:51:49 2018
From: report at bugs.python.org (Ivan Pozdeev)
Date: Wed, 02 May 2018 17:51:49 +0000
Subject: [New-bugs-announce] [issue33412] Tkinter hangs if using multiple
 threads and event handlers
Message-ID: <1525283509.66.0.682650639539.issue33412@psf.upfronthosting.co.za>


New submission from Ivan Pozdeev <ivan_pozdeev at mail.ru>:

With threaded Tkinter, TkinterHanders3.py from https://bugs.python.org/issue33257 (attached) hangs.

Tracing with thread_debug and a modified trace.py (to show TIDs, attached) shows that worker threads are waiting for the Tcl lock while the main thread that holds it keeps waiting for some other lock with a strange timeout:

19000: PyThread_acquire_lock_timed(00000000001B0F80, 0) called
19000: PyThread_acquire_lock(00000000001B0F80, 0) -> 0
19000: PyThread_acquire_lock_timed(00000000001B0F80, -1000000) called

Tested on 3.6 head, win7 x64, debug build.

----------
components: Tkinter
files: TkinterHanders3.py
messages: 316082
nosy: Ivan.Pozdeev
priority: normal
severity: normal
status: open
title: Tkinter hangs if using multiple threads and event handlers
type: crash
versions: Python 3.6, Python 3.7, Python 3.8
Added file: https://bugs.python.org/file47562/TkinterHanders3.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33412>
_______________________________________

From report at bugs.python.org  Wed May  2 14:39:18 2018
From: report at bugs.python.org (Martin Teichmann)
Date: Wed, 02 May 2018 18:39:18 +0000
Subject: [New-bugs-announce] [issue33413] asyncio.gather should not use
 special Future
Message-ID: <1525286358.11.0.682650639539.issue33413@psf.upfronthosting.co.za>


New submission from Martin Teichmann <martin.teichmann at gmail.com>:

asyncio.gather() returns a _GatheringFuture, which inherits from asyncio.Future. This is weird in current asyncio, as futures are supposed to be created with loop.create_future(). So I tried to reimplement gather() without this weird special future. I succeeded, yet I stumbled over weird inconsistencies with cancellation. There are three cases:

- coroutines have no special notion of cancellation, they treat CancelledError as any other exception

- futures have a clear distinction between exceptions and cancellation: future.set_exception(CancelledError()) is different from future.cancel(), as only for the latter future.cancelled() is True. This is used in the _GatheringFuture: it is cancelled() only if it got cancelled via future.cancel(), if its children gets cancelled it may set_exception(CancelledError()), but it will not be cancelled itself.

- Tasks consider raising a CancelledError always as a cancellation, whether it actually got cancelled or the wrapped coroutine raised CancelledError for whatever other reason. There is one exception: if the coroutine manages to return immediately after being cancelled, it raises a CancelledError, but task.cancelled() is false. So if a coroutine ends in

    current_task().cancel()
    return

the current task raises a CancelledError, but task.cancelled() is false.

I consider the last exception actually a bug, but it allows me to make my inheritance-free gather() look to the outside exactly like it used to be.

----------
messages: 316085
nosy: Martin.Teichmann
priority: normal
severity: normal
status: open
title: asyncio.gather should not use special Future
type: behavior
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33413>
_______________________________________

From report at bugs.python.org  Thu May  3 00:09:33 2018
From: report at bugs.python.org (=?utf-8?q?Andr=C3=A9s_Delfino?=)
Date: Thu, 03 May 2018 04:09:33 +0000
Subject: [New-bugs-announce] [issue33414] Make shutil.copytree use
 os.scandir to take advantage of cached is_(dir|file|symlink)
Message-ID: <1525320573.53.0.682650639539.issue33414@psf.upfronthosting.co.za>


New submission from Andr?s Delfino <adelfino at gmail.com>:

Right now we are using os.path.is(dir|file|symlink) in shutil.copytree, but taking advantage of os.scandir's is_(dir|file|symlink) seems the way to go.

I'll make a PR with to start the discussion with a proof of concept.

----------
components: Library (Lib)
messages: 316108
nosy: adelfino
priority: normal
severity: normal
status: open
title: Make shutil.copytree use os.scandir to take advantage of cached is_(dir|file|symlink)
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33414>
_______________________________________

From report at bugs.python.org  Thu May  3 04:26:26 2018
From: report at bugs.python.org (Ariel Otilibili Anieli)
Date: Thu, 03 May 2018 08:26:26 +0000
Subject: [New-bugs-announce] [issue33415] When add_mutually_exclusive_group
 is built without argument,
 the help breaks with "IndexError: list index out of range"
Message-ID: <1525335986.39.0.682650639539.issue33415@psf.upfronthosting.co.za>


New submission from Ariel Otilibili Anieli <arielanieli at yahoo.fr>:

Hello,

When add_mutually_exclusive_group is built without argument, the help breaks with "IndexError: list index out of range". Indeed this snippet:

actions_ = parser.add_argument_group('Actions')
actions  = actions_.add_mutually_exclusive_group()

Breaks the help of argparse:

# --help
Traceback (most recent call last):
  File "./ovpauto", line 135, in <module>
    args = parser.parse_args()
  File "/usr/lib/python2.7/argparse.py", line 1701, in parse_args
    args, argv = self.parse_known_args(args, namespace)
  File "/usr/lib/python2.7/argparse.py", line 1733, in parse_known_args
    namespace, args = self._parse_known_args(args, namespace)
  File "/usr/lib/python2.7/argparse.py", line 1939, in _parse_known_args
    start_index = consume_optional(start_index)
  File "/usr/lib/python2.7/argparse.py", line 1879, in consume_optional
    take_action(action, args, option_string)
  File "/usr/lib/python2.7/argparse.py", line 1807, in take_action
    action(self, namespace, argument_values, option_string)
  File "/usr/lib/python2.7/argparse.py", line 996, in __call__
    parser.print_help()
  File "/usr/lib/python2.7/argparse.py", line 2340, in print_help
    self._print_message(self.format_help(), file)
  File "/usr/lib/python2.7/argparse.py", line 2314, in format_help
    return formatter.format_help()
  File "/usr/lib/python2.7/argparse.py", line 281, in format_help
    help = self._root_section.format_help()
  File "/usr/lib/python2.7/argparse.py", line 211, in format_help
    func(*args)
  File "/usr/lib/python2.7/argparse.py", line 319, in _format_usage
    action_usage = format(optionals + positionals, groups)
  File "/usr/lib/python2.7/argparse.py", line 390, in _format_actions_usage
    start = actions.index(group._group_actions[0])
IndexError: list index out of range

Moreover, the trace does not say whence the error came from.

# uname -a 
Linux quasar 4.8.0-2-amd64 #1 SMP Debian 4.8.15-2 (2017-01-04) x86_64 GNU/Linux

#  lsb_release -a
No LSB modules are available.
Distributor ID:	Debian
Description:	Debian GNU/Linux testing (buster)
Release:	testing
Codename:	buster

#  gcc --version
gcc (Debian 7.3.0-17) 7.3.0
Copyright (C) 2017 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

# python --version 
Python 2.7.15rc1

#  dpkg -l | grep -P '\s+python\s+.*2.7.15'
ii  python                                                           2.7.15~rc1-1                                   amd64        interactive high-level object-oriented language (default version)

Regards,
Ariel

----------
messages: 316113
nosy: ariel-anieli, bethard
priority: normal
severity: normal
status: open
title: When add_mutually_exclusive_group is built without argument, the help breaks with "IndexError: list index out of range"
type: behavior
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33415>
_______________________________________

From report at bugs.python.org  Thu May  3 06:15:25 2018
From: report at bugs.python.org (Ivan Levkivskyi)
Date: Thu, 03 May 2018 10:15:25 +0000
Subject: [New-bugs-announce] [issue33416] Add endline and endcolumn to every
 AST node
Message-ID: <1525342525.55.0.682650639539.issue33416@psf.upfronthosting.co.za>


New submission from Ivan Levkivskyi <levkivskyi at gmail.com>:

Some Python tools (in particular I am interested in type checkers) will benefit from knowing where a given expression ends to indicate/highlight location of an error in the source code. Other tools and IDEs may have also some other benefits. Currently such tools need to use some hacks and/or custom parser to find the end line and end column of an expression, while it would be more straightforward to just add this information to every AST node by CPythons own parser.

This will increase memory usage, but expectation is that this effect will be minor.

What do you think?

----------
components: Interpreter Core
messages: 316117
nosy: gvanrossum, levkivskyi, lukasz.langa, serhiy.storchaka, vstinner
priority: normal
severity: normal
status: open
title: Add endline and endcolumn to every AST node
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33416>
_______________________________________

From report at bugs.python.org  Thu May  3 06:30:05 2018
From: report at bugs.python.org (weapon)
Date: Thu, 03 May 2018 10:30:05 +0000
Subject: [New-bugs-announce] [issue33417] Isinstance() behavior is not
 consistent with the document
Message-ID: <1525343405.44.0.682650639539.issue33417@psf.upfronthosting.co.za>


New submission from weapon <hongweichen8888 at sina.com>:

In the PEP 3119(https://www.python.org/dev/peps/pep-3119/),it introduce about Customizing instance and subclass checks.

>The primary mechanism proposed here is to allow overloading the built-in functions isinstance() and issubclass(). The overloading works as follows: The call isinstance(x, C) first checks whether C.__instancecheck__ exists, and if so, calls C.__instancecheck__(x) instead of its normal implementation.


but my code doesn't works.
```
class Sizeable(object):
    def __instancecheck__(cls, instance):
        print("__instancecheck__ call")
        return hasattr(instance, "__len__")

class B(object):
    pass

b = B()
print(isinstance(b, Sizeable)) # output:False
```
The __instancecheck__ function  is not called. In PyObject_IsInstance,when isinstance(x, C) the following three conditions are reached, __instancecheck__ will take effect:
1. x can not be directly instantiated by C;

2. The C class specifies the metaclass;

3. The __instancecheck__ in the specified metaclass class is defined.

This code can work:
```
class MetaSizeable(type):
    def __instancecheck__(cls, instance):
        print("__instancecheck__ call")
        return hasattr(instance, "__len__")

class Sizeable(metaclass=MetaSizeable):
    pass

class B(object):
    pass

b = B()
print(isinstance(b, Sizeable))  # output: False
print(isinstance([], Sizeable)) # output: True
```

So,the problem is that the document does not conform to the reality.Can it be guaranteed that __instancecheck__  is always called?

If yes: PyObject_IsInstance should be tweaked.
If no:  document should be tweaked.

----------
assignee: docs at python
components: Documentation, Interpreter Core
messages: 316118
nosy: docs at python, hongweipeng
priority: normal
severity: normal
status: open
title: Isinstance() behavior is not consistent with the document
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33417>
_______________________________________

From report at bugs.python.org  Thu May  3 13:54:23 2018
From: report at bugs.python.org (Jeroen Demeyer)
Date: Thu, 03 May 2018 17:54:23 +0000
Subject: [New-bugs-announce] [issue33418] Memory leaks in functions
Message-ID: <1525370063.61.0.682650639539.issue33418@psf.upfronthosting.co.za>


New submission from Jeroen Demeyer <J.Demeyer at UGent.be>:

This is a memory leak:

>>> while True:
...    def f(): pass
...    f.__doc__ = f

This also:

>>> while True:
...     f = [].append                                                                                                                                                   
...     f.__module__ = f

This is because the classes "function" and "builtin_function_or_method" do not implement tp_clear.

----------
messages: 316125
nosy: jdemeyer
priority: normal
severity: normal
status: open
title: Memory leaks in functions

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33418>
_______________________________________

From report at bugs.python.org  Thu May  3 14:31:32 2018
From: report at bugs.python.org (Neil Girdhar)
Date: Thu, 03 May 2018 18:31:32 +0000
Subject: [New-bugs-announce] [issue33419] Add functools.partialclass
Message-ID: <1525372292.27.0.682650639539.issue33419@psf.upfronthosting.co.za>


New submission from Neil Girdhar <mistersheik at gmail.com>:

functools.partial is almost good enough for specifying some of the parameters of an object's initializer, but the partial object doesn't respond properly to issubclass.  Adding functools.partialclass is similar to the addition of partialmethod in 3.4.

----------
files: partialclass.diff
keywords: patch
messages: 316126
nosy: neil.g
priority: normal
severity: normal
status: open
title: Add functools.partialclass
versions: Python 3.7
Added file: https://bugs.python.org/file47567/partialclass.diff

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33419>
_______________________________________

From report at bugs.python.org  Thu May  3 14:31:41 2018
From: report at bugs.python.org (Adam Paszke)
Date: Thu, 03 May 2018 18:31:41 +0000
Subject: [New-bugs-announce] [issue33420] __origin__ invariant broken
Message-ID: <1525372301.54.0.682650639539.issue33420@psf.upfronthosting.co.za>


New submission from Adam Paszke <adam.paszke at gmail.com>:

Hi everyone,

I have a module that needs to inspect type annotations on a few functions. One of the types I need to special case on is typing.Tuple, and I used code like this to detect it:

if getattr(annotation, '__origin__', None) == typing.Tuple:
    ... 
else:
    ...

This was based on the comment from the typing module (Lib/typing.py:609) that specified this particular invariant on the __origin__ attribute:

> __origin__ keeps a reference to a type that was subscripted,
  e.g., Union[T, int].__origin__ == Union;

Everything worked just fine until I checked it on the alpha release of Python 3.7 in my CI. Turns out, that in that release we have

typing.Tuple[str, int].__origin__ == tuple

and not (which is the case in e.g. 3.6)

typing.Tuple[str, int].__origin__ == typing.Tuple


I know this is not a documented attribute, so it can change, but I wanted to highlight that it's either a regression, or the comment will need to be updated, so people won't try to depend on that.

----------
components: Library (Lib)
messages: 316127
nosy: apaszke
priority: normal
severity: normal
status: open
title: __origin__ invariant broken
type: behavior
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33420>
_______________________________________

From report at bugs.python.org  Thu May  3 19:33:14 2018
From: report at bugs.python.org (Travis DePrato)
Date: Thu, 03 May 2018 23:33:14 +0000
Subject: [New-bugs-announce] [issue33421] Missing documentation for
 typing.AsyncContextManager
Message-ID: <1525390394.87.0.682650639539.issue33421@psf.upfronthosting.co.za>


New submission from Travis DePrato <travigd at umich.edu>:

The documentation for the typing module makes no mention of AsyncContextManager, which is defined in Lib/typing.py as

    AsyncContextManager = _alias(contextlib.AbstractAsyncContextManager, T_co)

as of >= Python 3.8; before 3.8, no such AbstractAsyncContextManager class exists.

----------
assignee: docs at python
components: Documentation
messages: 316145
nosy: Travis DePrato, docs at python, eric.araujo, ezio.melotti, willingc
priority: normal
severity: normal
status: open
title: Missing documentation for typing.AsyncContextManager
type: enhancement
versions: Python 3.5, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33421>
_______________________________________

From report at bugs.python.org  Thu May  3 19:37:01 2018
From: report at bugs.python.org (=?utf-8?q?Andr=C3=A9s_Delfino?=)
Date: Thu, 03 May 2018 23:37:01 +0000
Subject: [New-bugs-announce] [issue33422] Fix and update string/byte
 literals in help()
Message-ID: <1525390621.75.0.682650639539.issue33422@psf.upfronthosting.co.za>


New submission from Andr?s Delfino <adelfino at gmail.com>:

Right now, for string/byte literals help() shows, for example:

No Python documentation found for 'r'.
Use help() to get the interactive help utility.
Use help(str) for help on the str class.

PR fixes the quotation mark removal and updates the list with f and u literals, while also adding uppercase versions of all literals. While the list is install incomplete (e.g., fR and the others could be listed) I believe that's too much.

----------
components: Library (Lib)
messages: 316148
nosy: adelfino
priority: normal
severity: normal
status: open
title: Fix and update string/byte literals in help()
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33422>
_______________________________________

From report at bugs.python.org  Thu May  3 22:38:56 2018
From: report at bugs.python.org (Daehee Kim)
Date: Fri, 04 May 2018 02:38:56 +0000
Subject: [New-bugs-announce] [issue33423] [logging] Improve consistency of
 logger mechanism.
Message-ID: <1525401536.14.0.682650639539.issue33423@psf.upfronthosting.co.za>


New submission from Daehee Kim <zsaladinz at gmail.com>:

There's a proposal for a fix inconsistency of logger mechanism.
See below.

Example 1.
---------------------------------------------------
import logging

app_logger = logging.getLogger('app')

app_logger.error('foo')
logging.error('goo')
app_logger.error('foo')
---------------------------------------------------

result:
foo
ERROR:root:goo
ERROR:app:foo
---------------------------------------------------
You can see the inconsistency of app_logger results.


Example 2.
---------------------------------------------------
import logging

stream_handler = logging.StreamHandler()
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
stream_handler.setFormatter(formatter)

app_logger: logging.Logger = logging.getLogger('app')
app_logger.setLevel(logging.ERROR)
app_logger.addHandler(stream_handler)


app_logger.critical('foo')
logging.critical('goo')
app_logger.critical('foo')
---------------------------------------------------

results:
2018-05-04 11:21:15,199 - app - CRITICAL - foo
CRITICAL:root:goo
2018-05-04 11:21:15,200 - app - CRITICAL - foo
CRITICAL:app:foo
---------------------------------------------------

You can see even inconsistency of app_logger`s outputs count.

So I have some proposals for fix it.

Proposal 1. 
Remove implicit calling `basicConfig`. Then you can get the consistent results.

Result of the example 1.
foo
goo
foo

Result of the example 2.
2018-05-04 11:31:36,069 - app - CRITICAL - foo
goo
2018-05-04 11:31:36,070 - app - CRITICAL - foo

link : https://github.com/zsaladin/cpython/commit/c4e789af32da36ba49196fd6be9c070f62346b7a#diff-043d23e54edc5360a7785ae212d1b806


Proposal 2.
Add calling `basicConfig` in `callHandlers` method if the root logger has no handler.

Result of the example 1.
foo
goo
foo

Result of the example 2.
2018-05-04 11:31:36,069 - app - CRITICAL - foo
goo
2018-05-04 11:31:36,070 - app - CRITICAL - foo

Result of the example 1.
ERROR:app:foo
ERROR:root:goo
ERROR:app:foo

Result of the example 2.
2018-05-04 11:36:45,318 - app - CRITICAL - foo
CRITICAL:app:foo
CRITICAL:root:goo
2018-05-04 11:36:45,319 - app - CRITICAL - foo
CRITICAL:app:foo

https://github.com/zsaladin/cpython/commit/fc295b46a4de584dc5c6724125b2c0f8b2694aa6#diff-043d23e54edc5360a7785ae212d1b806


Thank you.

----------
components: Library (Lib)
messages: 316153
nosy: Daehee Kim
priority: normal
severity: normal
status: open
title: [logging] Improve consistency of logger mechanism.
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33423>
_______________________________________

From report at bugs.python.org  Thu May  3 23:07:20 2018
From: report at bugs.python.org (CH)
Date: Fri, 04 May 2018 03:07:20 +0000
Subject: [New-bugs-announce] [issue33424] 4.4. break and continue Statements,
 and else Clauses on Loops
Message-ID: <1525403240.67.0.682650639539.issue33424@psf.upfronthosting.co.za>


New submission from CH <joechenghao at gmail.com>:

range function second parameter is excluded. Hence for x in range(2, n): will not execute when n = 2, and "2 is a prime number" will never appear.
Moreover, might need a break in the else block too.

----------
messages: 316158
nosy: joesatriani
priority: normal
severity: normal
status: open
title: 4.4. break and continue Statements, and else Clauses on Loops

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33424>
_______________________________________

From report at bugs.python.org  Fri May  4 06:14:47 2018
From: report at bugs.python.org (Robin Champavier)
Date: Fri, 04 May 2018 10:14:47 +0000
Subject: [New-bugs-announce] [issue33425] Library glob : Can't find a
 specific year with glob
Message-ID: <1525428887.45.0.682650639539.issue33425@psf.upfronthosting.co.za>


New submission from Robin Champavier <robin.champavier at gmail.com>:

Hello,

I work with file name like : PRECI_yyyymmddhh_00.txt and I put them in different directory depending of the year. For file between year 1999 and 2016 there is no problem but with file in year 2017 with months 12, glob.glob can't find them. When I do print K in my file Tri_fichier_texte.py there is no file with PRECI_201712xxxx.txt but it works to find the beginning of the year 2017 like PRECI_201701xxx.txt. All file get exactly the same syntax. 

Thanks for looking my issues.

----------
components: Library (Lib)
files: Tri_fichier_texte.py
messages: 316163
nosy: Robin Champavier
priority: normal
severity: normal
status: open
title: Library glob : Can't find a specific year with glob
versions: Python 2.7
Added file: https://bugs.python.org/file47571/Tri_fichier_texte.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33425>
_______________________________________

From report at bugs.python.org  Fri May  4 14:39:06 2018
From: report at bugs.python.org (Michael Klatt)
Date: Fri, 04 May 2018 18:39:06 +0000
Subject: [New-bugs-announce] [issue33426] Behavior of os.path.join does not
 match documentation
Message-ID: <1525459146.45.0.682650639539.issue33426@psf.upfronthosting.co.za>


New submission from Michael Klatt <michael.d.klatt at gmail.com>:

The behavior of os.path.join() regarding path separators does not match the documentation. This affects Python 3.6, and goes back to at least Python 2.7.

>From the documenation: 

"The return value is the concatenation of path and any members of *paths with exactly one directory separator (os.sep) following each non-empty part except the last, meaning that the result will only end in a separator if the last part is empty."

To me, this means that join will remove extraneous separators from the path, and that the only way to produce a trailing separator is to use join "" as the final path segment.

I expect `os.path.join("/abc//", "def/")` to produce the string "/abc/def" based on the documentation, but what it actually produces is "abc//def/".

----------
assignee: docs at python
components: Documentation, Library (Lib)
messages: 316184
nosy: Michael Klatt, docs at python
priority: normal
severity: normal
status: open
title: Behavior of os.path.join does not match documentation
type: behavior
versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33426>
_______________________________________

From report at bugs.python.org  Fri May  4 23:03:55 2018
From: report at bugs.python.org (MATSUURA, yosuke)
Date: Sat, 05 May 2018 03:03:55 +0000
Subject: [New-bugs-announce] [issue33427] Dead link in "The Python Standard
 Library" page
Message-ID: <1525489435.58.0.682650639539.issue33427@psf.upfronthosting.co.za>


New submission from MATSUURA, yosuke <howl at rampage.red>:

"The Python Standard Library" page (https://docs.python.org/3.6/library/index.html#the-python-standard-library) has a link to "Python Package Index" in the fourth paragraph
 but it linked to 404 URI(https://pypi.org/pypi/).
That link should be changed to a new URI(https://pypi.org/).

---
Fourth paragraph
"In addition to the standard library, there is a growing collection of several thousand components (from individual programs and modules to packages and entire application development frameworks), available from the ""Python Package Index""(This link is broken)."

----------
assignee: docs at python
components: Documentation
messages: 316193
nosy: docs at python, lighthawk
priority: normal
severity: normal
status: open
title: Dead link in "The Python Standard Library" page
type: enhancement
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33427>
_______________________________________

From report at bugs.python.org  Sat May  5 02:23:00 2018
From: report at bugs.python.org (Brian Sheldon)
Date: Sat, 05 May 2018 06:23:00 +0000
Subject: [New-bugs-announce] [issue33428] pathlib.Path.glob does not follow
 symlinks
Message-ID: <1525501380.69.0.682650639539.issue33428@psf.upfronthosting.co.za>


New submission from Brian Sheldon <brianmsheldon at gmail.com>:

Given a `pathlib.Path` that contains symlinked subfolders, `Path.glob` (and `.rglob`) do not follow symlinks.  This is not consistent with `glob.glob` which does.

For example given the following:
C:\Folder
C:\Folder\Subfolder -> D:\Subfolder
D:\Subfolder\File.txt

`pathlib.Path('C:/Folder').glob('**/*')` yields the following paths:
WindowsPath('C:/Folder/Subfolder')

`glob.glob('C:/Folder/**/*')` yields the following paths:
'C:/Folder\\Subfolder'
'C:/Folder\\Subfolder\\File.txt'

Notice how the contents of Subfolder are present in the `glob.glob` results but not for `Path.glob`.

I would expect `Path.glob` to be consistent with `glob.glob`.  This is not the only inconsistency (e.g. #22276, #31202) and perhaps `Path.glob` should be re-implemented using `glob.glob`.

----------
components: Library (Lib), Windows
messages: 316197
nosy: brianmsheldon, emilyemorehouse, paul.moore, pitrou, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: pathlib.Path.glob does not follow symlinks
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33428>
_______________________________________

From report at bugs.python.org  Sat May  5 03:16:18 2018
From: report at bugs.python.org (Raymond Hettinger)
Date: Sat, 05 May 2018 07:16:18 +0000
Subject: [New-bugs-announce] [issue33429] IDLE tooltips stopped working
 between 2.7.14 and 2.7.15
Message-ID: <1525504578.56.0.682650639539.issue33429@psf.upfronthosting.co.za>


New submission from Raymond Hettinger <raymond.hettinger at gmail.com>:

When upgrading from 2.7.14 to 2.7.15 on MacOS, the tooltips have stopped showing up:

    >>> range(    
             ^--- formerly a tooltip box would pop-up at this point

----------
assignee: terry.reedy
components: IDLE
messages: 316198
nosy: rhettinger, terry.reedy
priority: normal
severity: normal
status: open
title: IDLE tooltips stopped working between 2.7.14 and 2.7.15
type: behavior
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33429>
_______________________________________

From report at bugs.python.org  Sat May  5 04:54:12 2018
From: report at bugs.python.org (Daniel Chimeno)
Date: Sat, 05 May 2018 08:54:12 +0000
Subject: [New-bugs-announce] [issue33430] Import secrets module in secrets
 examples
Message-ID: <1525510452.55.0.682650639539.issue33430@psf.upfronthosting.co.za>


New submission from Daniel Chimeno <daniel at chimeno.me>:

In the secrets module documentation, the examples in `15.3.4. Recipes and best practices` need change things, I think this examples must run after a copy & paste into user terminal.

Options for first example:
1:
````
import string
from secrets import choice
alphabet = string.ascii_letters + string.digits
password = ''.join(choice(alphabet) for i in range(8))
````

2:
```
import string
import secrets
alphabet = string.ascii_letters + string.digits
password = ''.join(secrets.choice(alphabet) for i in range(8))
````

I've looked in the devguide, but I haven't found what's the standard approach here.

First issue here, be nice :)

----------
assignee: docs at python
components: Documentation
messages: 316200
nosy: dchimeno, docs at python
priority: normal
severity: normal
status: open
title: Import secrets module in secrets examples
type: enhancement
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33430>
_______________________________________

From report at bugs.python.org  Sat May  5 06:06:00 2018
From: report at bugs.python.org (lvhuiyang)
Date: Sat, 05 May 2018 10:06:00 +0000
Subject: [New-bugs-announce] [issue33431] Change description about doc in
 programming, faq.
Message-ID: <1525514760.53.0.682650639539.issue33431@psf.upfronthosting.co.za>


Change by lvhuiyang <ilvhuiyang at gmail.com>:


----------
assignee: docs at python
components: Documentation
nosy: docs at python, lvhuiyang
priority: normal
severity: normal
status: open
title: Change description about doc in programming, faq.
type: performance
versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33431>
_______________________________________

From report at bugs.python.org  Sat May  5 08:18:32 2018
From: report at bugs.python.org (Licht Takeuchi)
Date: Sat, 05 May 2018 12:18:32 +0000
Subject: [New-bugs-announce] [issue33432] No locale alias mapping key for
 en_IL
Message-ID: <1525522712.55.0.682650639539.issue33432@psf.upfronthosting.co.za>


New submission from Licht Takeuchi <licht-t at outlook.jp>:

The new locale en_IL is added to glibc.
But there is no key for this in the locale alias mapping.
https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=146ffc146fe3bf97cd3bc1a649f1ffa8acfa4a0d
https://github.com/python/cpython/blob/master/Lib/locale.py#L850

The locale in latest Ubuntu 18.04 contains en_IL as valid locale, but Python cannot resolve this.
This makes test failure in pandas.
https://github.com/pandas-dev/pandas/issues/20957

```
(pandas-dev) [pandas] locale -a
C
C.UTF-8
en_AG
en_AG.utf8
en_AU.utf8
en_BW.utf8
en_CA.utf8
en_DK.utf8
en_GB.utf8
en_HK.utf8
en_IE.utf8
en_IL
en_IL.utf8
en_IN
en_IN.utf8
en_NG
en_NG.utf8
en_NZ.utf8
en_PH.utf8
en_SG.utf8
en_US.utf8
en_ZA.utf8
en_ZM
en_ZM.utf8
en_ZW.utf8
ja_JP.utf8
POSIX
```

----------
components: Library (Lib)
messages: 316206
nosy: licht-t
priority: normal
severity: normal
status: open
title: No locale alias mapping key for en_IL
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33432>
_______________________________________

From report at bugs.python.org  Sat May  5 09:34:54 2018
From: report at bugs.python.org (Thomas Kriechbaumer)
Date: Sat, 05 May 2018 13:34:54 +0000
Subject: [New-bugs-announce] [issue33433] ipaddress is_private misleading
 for IPv4 mapped IPv6 addresses
Message-ID: <1525527294.14.0.682650639539.issue33433@psf.upfronthosting.co.za>


New submission from Thomas Kriechbaumer <kriechbaumer at gmail.com>:

ipaddress.IPv4Address and ipaddress.IPv6Address provide a is_private function that indicates if the address (or network) is part of a "private" IP range, as designated by the IANA special-use registry.

The current documentation is as follows:
> is_private
>    True if the address is allocated for private networks. See iana-ipv4-special-registry (for IPv4) or iana-ipv6-special-registry (for IPv6).


However, IPv4-mapped IPv6 addresses are currently being reported as "private" by this function (see https://github.com/python/cpython/commit/22c31764262b02338265a059c738b8d24fd9a0e4#diff-0fc57874e463b95dbdfe6f80ae918ea1R1869).

I consider this a bug or at least counter-intuitive, because e.g. ::ffff:8.8.8.8 is most definitely NOT a private IP address, and yet the ipaddress.is_private function reports True.
The IANA special-use registry does not contain any references to "being private" for the ::ffff region. It just designates it to be used for IPv4-mapped address.

----------
components: Library (Lib)
messages: 316213
nosy: Thomas Kriechbaumer
priority: normal
severity: normal
status: open
title: ipaddress is_private misleading for IPv4 mapped IPv6 addresses
versions: Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33433>
_______________________________________

From report at bugs.python.org  Sun May  6 03:54:27 2018
From: report at bugs.python.org (hexchain)
Date: Sun, 06 May 2018 07:54:27 +0000
Subject: [New-bugs-announce] [issue33434] ^L character in
 Lib/email/generator.py
Message-ID: <1525593267.83.0.682650639539.issue33434@psf.upfronthosting.co.za>


New submission from hexchain <i at hexchain.org>:

There is a "^L" character in line 25 of the Lib/email/generator.py file, and it seems it's there for a long time (since commit 8b3febef2f9).

Is it intended or some carelessness? It does not seem to have any functional impact, though.

----------
components: Library (Lib)
messages: 316229
nosy: hexchain
priority: normal
severity: normal
status: open
title: ^L character in Lib/email/generator.py
versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33434>
_______________________________________

From report at bugs.python.org  Sun May  6 05:53:32 2018
From: report at bugs.python.org (Andrey Bychkov)
Date: Sun, 06 May 2018 09:53:32 +0000
Subject: [New-bugs-announce] [issue33435] incorrect detection of information
 of some distributions
Message-ID: <1525600412.06.0.682650639539.issue33435@psf.upfronthosting.co.za>


New submission from Andrey Bychkov <mrandybu at gmail.com>:

In some linux distributions, the information about the distribution is incorrectly determined when the linux_distribution() method is called from the platform class. Since the information file os-release becomes a certain standard, I propose first of all to check its availability and if it is in the system, then parse the information from it. I apply the patch. It will work well with version 2.7.

----------
components: Library (Lib)
files: fixed_lib_platform_os-release.patch
keywords: patch
messages: 316232
nosy: Andrey Bychkov
priority: normal
severity: normal
status: open
title: incorrect detection of information of some distributions
type: behavior
versions: Python 3.6
Added file: https://bugs.python.org/file47572/fixed_lib_platform_os-release.patch

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33435>
_______________________________________

From report at bugs.python.org  Sun May  6 21:47:00 2018
From: report at bugs.python.org (Raymond Hettinger)
Date: Mon, 07 May 2018 01:47:00 +0000
Subject: [New-bugs-announce] [issue33436] Add an interactive shell for
 Sqlite3
Message-ID: <1525657620.33.0.682650639539.issue33436@psf.upfronthosting.co.za>


New submission from Raymond Hettinger <raymond.hettinger at gmail.com>:

To facilitate rapid experimentation with SQL queries, add a shell to run commands and display results.

Attached is minimal proof-of-concept modeled loosely on the Linux sqlite3 REPL.  Here's a sample session:

    SQLite version 2.6.0

     Enter "help" for usage hints

    sqlite> open Irises.db
    sqlite> select distinct species from irises
    ('Iris-setosa',)
    ('Iris-versicolor',)
    ('Iris-virginica',)
    sqlite> select species, avg(pet_len), avg(sep_len) from irises group by species
    ('Iris-setosa', 1.464, 5.005999999999999)
    ('Iris-versicolor', 4.26, 5.936)
    ('Iris-virginica', 5.552, 6.587999999999998)
    sqlite> select count(*) from irises
    (150,)
    sqlite> quit

----------
components: Library (Lib)
files: sqlite3_repl.py
messages: 316248
nosy: rhettinger
priority: normal
severity: normal
status: open
title: Add an interactive shell for Sqlite3
type: enhancement
versions: Python 3.8
Added file: https://bugs.python.org/file47573/sqlite3_repl.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33436>
_______________________________________

From report at bugs.python.org  Mon May  7 06:02:37 2018
From: report at bugs.python.org (Andres Ayala)
Date: Mon, 07 May 2018 10:02:37 +0000
Subject: [New-bugs-announce] [issue33437] Defining __init__ in enums
Message-ID: <1525687357.98.0.682650639539.issue33437@psf.upfronthosting.co.za>


New submission from Andres Ayala <killerrex at gmail.com>:

I have to read and write many files for an old fortran program where the elements are coded as integer (so 0 is the X, 1 is the Y...) and I have some tags associated to each one.

So I tried to create an Enum with some additional fields:
class Coord(Enum):
    def __init__(self, value, label, unit):
        super().__init__()
        self._value_ = value
        self.label = label
        self.unit = unit

    PX = (0, 'P.X', 'km')
    PY = (1, 'P.Y', 'km')
    VX = (2, 'V.X', 'km/s')
    VY = (3, 'V.Y', 'km/s')

This almost work:
>>> for element in CoordOk:
...    print("{0}: {0.label}[{0.unit}] = {0.value}".format(element))
CoordOk.PX: P.X[km] = 0
CoordOk.PY: P.Y[km] = 1
CoordOk.VX: V.X[km/s] = 2
CoordOk.VY: V.Y[km/s] = 3

But fails recovering the coordinate from the value:
>>> print(Coord(0))
Traceback (most recent call last):
  File "/home/killerrex/tmp/demo.py", line 33, in <module>
    print(Coord(0))
  File "/usr/lib64/python3.6/enum.py", line 291, in __call__
    return cls.__new__(cls, value)
  File "/usr/lib64/python3.6/enum.py", line 533, in __new__
    return cls._missing_(value)
  File "/usr/lib64/python3.6/enum.py", line 546, in _missing_
    raise ValueError("%r is not a valid %s" % (value, cls.__name__))
ValueError: 0 is not a valid Coord

Because the internal value of the enum is (0, 'P.X', 'km') and not 0

I found that it is possible to do what I am trying to do with:
class CoordOk(Enum):
    def __new__(cls, value, label, unit):
        obj = object.__new__(cls)
        obj._value_ = value
        obj.label = label
        obj.unit = unit
        return obj

    PX = (0, 'P.X', 'km')
    PY = (1, 'P.Y', 'km')
    VX = (2, 'V.X', 'km/s')
    VY = (3, 'V.Y', 'km/s')

Following the latest doc from Enum. Although it is not so intuitive (defining __new__ is not as easy as __init__)

To make the first proposal work fine is enough with moving the line 221 of enum.py after the line 224 (see the patch)
I haven't found problems with the change but obviously my little reduced tests are not representative at all.
And even if it works there may be good reasons to not allow changing the _value_ in the init.

So mi enhancement proposal is either:
Add an example like the CoordOk to the documentation to help others to find how to change the _value_ (and an explanation on why is better not to do it in the __init__ would be perfect)

or

Change the line enum.py so it is possible to update the _value_ in the __init__ method and if you want use the Coord example in the docs.

Thanks!

----------
components: Library (Lib)
files: enum_allow_change_value_in_init.patch
keywords: patch
messages: 316256
nosy: killerrex
priority: normal
severity: normal
status: open
title: Defining __init__ in enums
type: enhancement
versions: Python 3.6
Added file: https://bugs.python.org/file47575/enum_allow_change_value_in_init.patch

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33437>
_______________________________________

From report at bugs.python.org  Mon May  7 06:55:52 2018
From: report at bugs.python.org (Antoine Pitrou)
Date: Mon, 07 May 2018 10:55:52 +0000
Subject: [New-bugs-announce] [issue33438] pkg-config file misses flags for
 static linking
Message-ID: <1525690552.38.0.682650639539.issue33438@psf.upfronthosting.co.za>


New submission from Antoine Pitrou <pitrou at free.fr>:

Our pkg-config misses flags for static linking with libpythonXX.a, such as "-lpthread -lutil -lrt" on Windows.  For dynamic linking, this is not a problem since libpythonXX.so links explicitly against those system libraries.

----------
components: Build
messages: 316258
nosy: barry, doko, pitrou
priority: normal
severity: normal
status: open
title: pkg-config file misses flags for static linking
type: behavior
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33438>
_______________________________________

From report at bugs.python.org  Mon May  7 08:15:49 2018
From: report at bugs.python.org (Antoine Pitrou)
Date: Mon, 07 May 2018 12:15:49 +0000
Subject: [New-bugs-announce] [issue33439] python-config.py should be part of
 the stdlib
Message-ID: <1525695349.77.0.682650639539.issue33439@psf.upfronthosting.co.za>


New submission from Antoine Pitrou <pitrou at free.fr>:

Rather than (or in addition to) being a standalone script, python-config should be invokable as a stdlib module, e.g. "python -m sysconfig.config".  This would prevent the risk of invoking the wrong script on PATH.

----------
components: Build, Library (Lib)
messages: 316264
nosy: barry, doko, ncoghlan, pitrou
priority: normal
severity: normal
status: open
title: python-config.py should be part of the stdlib
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33439>
_______________________________________

From report at bugs.python.org  Mon May  7 08:28:40 2018
From: report at bugs.python.org (Nick Coghlan)
Date: Mon, 07 May 2018 12:28:40 +0000
Subject: [New-bugs-announce] [issue33440] Possible lazy import opportunities
 in `pathlib`
Message-ID: <1525696120.71.0.682650639539.issue33440@psf.upfronthosting.co.za>


New submission from Nick Coghlan <ncoghlan at gmail.com>:

Due to a python-ideas discussion about reducing boilerplate for __file__-relative path calculations, I was running "./python -X importtime -S -c 'import pathlib'" and noticed three potential candidates where it may be worthwhile deferring the imports until the modules are actually needed:

- re (used in _WildcardSelector)
- fnmatch (used in PurePath.match and _WildcardSelector)
- urllib.parse (used in PurePath.as_uri, by way of self._flavour.make_uri)

Using an optimised Python 3.7 on an SSD with warm disk caches, commenting out those 3 imports reduced my typical import times for pathlib from 12-13 ms to 7-8 ms)

----------
components: Library (Lib)
messages: 316266
nosy: ncoghlan, njs, pitrou
priority: normal
severity: normal
status: open
title: Possible lazy import opportunities in `pathlib`
type: performance
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33440>
_______________________________________

From report at bugs.python.org  Mon May  7 08:40:56 2018
From: report at bugs.python.org (Serhiy Storchaka)
Date: Mon, 07 May 2018 12:40:56 +0000
Subject: [New-bugs-announce] [issue33441] Expose the sigset_t converter via
 private API
Message-ID: <1525696856.75.0.682650639539.issue33441@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:

posix_spawn() (see issue20104) needs the converter to sigset_t defined in signalmodule.c. Since the code is not trivial, it is better to share it instead of duplicate. The proposed PR:

* Exposes the sigset_t converter via private API _Py_Sigset_Converter(). The implementation is moved to posixmodule.c.

* Uses Argument Clinic for parsing sigset_t in signalmodule.c. In particularly it causes that the first argument of signal.sigtimedwait() will be parsed before the second one.

* Make the converter always raising ValueError for signal numbers out of range 1..NSIG. OverflowError was raised before for integers out of the platform depending C long range.

----------
messages: 316267
nosy: pablogsal, pitrou, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Expose the sigset_t converter via private API
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33441>
_______________________________________

From report at bugs.python.org  Mon May  7 09:06:02 2018
From: report at bugs.python.org (Pete Moore)
Date: Mon, 07 May 2018 13:06:02 +0000
Subject: [New-bugs-announce] [issue33442] Python 3 doc sidebar dosnt follow
 page scrolling like 2.7 doc sidebar.
Message-ID: <1525698362.62.0.682650639539.issue33442@psf.upfronthosting.co.za>


New submission from Pete Moore <1petemoore at gmail.com>:

the 3.x docs sidebar at location https://docs.python.org/3.6/library/logging.html does not follow the user's scrolling movements like the 2.7 docs sidebar at location https://docs.python.org/2.7/library/logging.html

----------
assignee: docs at python
components: Documentation
messages: 316269
nosy: docs at python, pete312
priority: normal
severity: normal
status: open
title: Python 3 doc sidebar dosnt follow page scrolling like 2.7 doc sidebar.
type: behavior
versions: Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33442>
_______________________________________

From report at bugs.python.org  Mon May  7 11:12:33 2018
From: report at bugs.python.org (Yuki Wakisaka)
Date: Mon, 07 May 2018 15:12:33 +0000
Subject: [New-bugs-announce] [issue33443] Typo in Python/import.c
Message-ID: <1525705953.03.0.682650639539.issue33443@psf.upfronthosting.co.za>


New submission from Yuki Wakisaka <ukwksk+python at gmail.com>:

In the method, "_PyImportZip_Init(void)", of Python/import.c, the variable, "PyObject *path_hooks, *zimpimport;", seems to be typo, which can be "zipimport".

----------
components: Interpreter Core
messages: 316273
nosy: ukwksk
priority: normal
severity: normal
status: open
title: Typo in Python/import.c
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33443>
_______________________________________

From report at bugs.python.org  Tue May  8 07:26:46 2018
From: report at bugs.python.org (MultiSosnooley)
Date: Tue, 08 May 2018 11:26:46 +0000
Subject: [New-bugs-announce] [issue33444] Memory leak/high usage on copy in
 different thread
Message-ID: <1525778806.98.0.682650639539.issue33444@psf.upfronthosting.co.za>


New submission from MultiSosnooley <multisosnooley at gmail.com>:

On linux (ubuntu 16.04, 18.04 tested) with python 3.6.5, 3.5.5 and 3.7-dev (windows is not affected) there is ~850Mb of memory used by python process at sleep point. Replacing `submit` `result` with plain function call causes normal ~75Mb at sleep point.
Maybe it is linux side issue (or normal behavior).

----------
components: Interpreter Core
files: test-memory-leak.py
messages: 316285
nosy: MultiSosnooley
priority: normal
severity: normal
status: open
title: Memory leak/high usage on copy in different thread
versions: Python 3.5, Python 3.6, Python 3.7
Added file: https://bugs.python.org/file47576/test-memory-leak.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33444>
_______________________________________

From report at bugs.python.org  Tue May  8 10:29:58 2018
From: report at bugs.python.org (Jeroen Demeyer)
Date: Tue, 08 May 2018 14:29:58 +0000
Subject: [New-bugs-announce] [issue33445] test_cprofile should fail instead
 of displaying a message
Message-ID: <1525789798.41.0.682650639539.issue33445@psf.upfronthosting.co.za>


New submission from Jeroen Demeyer <J.Demeyer at UGent.be>:

When this test from Lib/test/test_profile.py fail, it just prints a message and doesn't fail the testsuite:

    def test_cprofile(self):
        results = self.do_profiling()
        expected = self.get_expected_output()
        self.assertEqual(results[0], 1000)
        for i, method in enumerate(self.methodnames):
            if results[i+1] != expected[method]:
                print("Stats.%s output for %s doesn't fit expectation!" %
                      (method, self.profilerclass.__name__))
                print('\n'.join(unified_diff(
                                  results[i+1].split('\n'),
                                  expected[method].split('\n'))))

----------
components: Tests
messages: 316287
nosy: jdemeyer
priority: normal
severity: normal
status: open
title: test_cprofile should fail instead of displaying a message

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33445>
_______________________________________

From report at bugs.python.org  Tue May  8 14:29:15 2018
From: report at bugs.python.org (Xavier de Gaye)
Date: Tue, 08 May 2018 18:29:15 +0000
Subject: [New-bugs-announce] [issue33446] destructors of local variables are
 not traced
Message-ID: <1525804155.74.0.682650639539.issue33446@psf.upfronthosting.co.za>


New submission from Xavier de Gaye <xdegaye at gmail.com>:

In the following code, the destructors of objects referenced by the 'a' and 'b' local variables are not traced by the 'step' command of pdb.

 1 class C:
 2     def __init__(self, name):
 3         self.name = name
 4
 5     def __del__(self):
 6         print('"%s" destructor' % self.name)
 7
 8 def main():
 9     a = C('a')
10     b = C('b')
11     import pdb; pdb.set_trace()
12     a = 1
13
14 main()

----------
components: Interpreter Core, Library (Lib)
messages: 316290
nosy: xdegaye
priority: normal
severity: normal
status: open
title: destructors of local variables are not traced
type: behavior
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33446>
_______________________________________

From report at bugs.python.org  Tue May  8 23:22:40 2018
From: report at bugs.python.org (Noah Simon)
Date: Wed, 09 May 2018 03:22:40 +0000
Subject: [New-bugs-announce] [issue33447] Asynchronous lambda syntax
Message-ID: <1525836160.23.0.682650639539.issue33447@psf.upfronthosting.co.za>


New submission from Noah Simon <noahs2003 at gmail.com>:

It would be very useful to add an asynchronous lambda syntax, as a shortcut for coroutines. I'm not experienced enough to write a PEP or edit the C source, but I have some ideas for syntax:

import asyncio
foo = async lambda a,b: 5 + await bar(b)

----------
components: asyncio
messages: 316304
nosy: Noah Simon, asvetlov, yselivanov
priority: normal
severity: normal
status: open
title: Asynchronous lambda syntax
type: enhancement
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33447>
_______________________________________

From report at bugs.python.org  Wed May  9 13:02:39 2018
From: report at bugs.python.org (vishva patel)
Date: Wed, 09 May 2018 17:02:39 +0000
Subject: [New-bugs-announce] [issue33448] Output_Typo_Error
Message-ID: <1525885359.71.0.682650639539.issue33448@psf.upfronthosting.co.za>


New submission from vishva patel <vnspvt.123 at gmail.com>:

There is an out[ut typo error in the documentation of python in the module (An Informal Introductio To Python) 3.1.2 section Strings the first example last output the output of:
>>>Isn\'t is given as Isn\t #but it shuld be
>>>Isn\'t
Isn't

so fix it

----------
messages: 316325
nosy: vishva_11
priority: normal
severity: normal
status: open
title: Output_Typo_Error
type: resource usage

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33448>
_______________________________________

From report at bugs.python.org  Wed May  9 13:38:58 2018
From: report at bugs.python.org (Francois Labelle)
Date: Wed, 09 May 2018 17:38:58 +0000
Subject: [New-bugs-announce] [issue33449] Documentation for email.charset
 confusing about the location of constants
Message-ID: <1525887538.2.0.682650639539.issue33449@psf.upfronthosting.co.za>


New submission from Francois Labelle <quantum.omega at gmail.com>:

When mentioning the constants contained in the module email.charset, the documentation prefixes them with "Charset", for example: Charset.QP, Charset.BASE64, Charset.SHORTEST. This suggests that the constants can be found on the class "Charset" (uppercase first letter) in the module, while they are in fact contained in the the "charset" (lowercase first letter) module itself.

This is probably a left-over from when the module was called email.Charset (in 2.2 I believe). Under Python 2.7, while the documentation uses the same nomenclature, at least the module is also importable under the name email.Charset so it makes more sense.

----------
assignee: docs at python
components: Documentation
messages: 316327
nosy: Francois Labelle, docs at python
priority: normal
severity: normal
status: open
title: Documentation for email.charset confusing about the location of constants
type: enhancement
versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33449>
_______________________________________

From report at bugs.python.org  Wed May  9 15:04:59 2018
From: report at bugs.python.org (Richard C)
Date: Wed, 09 May 2018 19:04:59 +0000
Subject: [New-bugs-announce] [issue33450] unexpected EPROTOTYPE returned by
 sendto on MAC OSX
Message-ID: <1525892699.63.0.682650639539.issue33450@psf.upfronthosting.co.za>


New submission from Richard C <rich at racitup.com>:

The following exception is raised unexpectedly on macOS versions 10.13, 10.12 & 10.11 at least. It appears to be macOS specific (works okay on Linux).

Further information can be found at the following links:
https://github.com/benoitc/gunicorn/issues/1487
http://erickt.github.io/blog/2014/11/19/adventures-in-debugging-a-potential-osx-kernel-bug/

[2017-03-20 00:46:39 +0100] [79068] [ERROR] Socket error processing request.
Traceback (most recent call last):
  File "/Users/ahmad/Projects/Side-Gigs/sa7beh-app/venv/lib/python3.6/site-packages/gunicorn-19.7.0-py3.6.egg/gunicorn/workers/async.py", line 66, in handle
    six.reraise(*sys.exc_info())
  File "/Users/ahmad/Projects/Side-Gigs/sa7beh-app/venv/lib/python3.6/site-packages/gunicorn-19.7.0-py3.6.egg/gunicorn/six.py", line 625, in reraise
    raise value
  File "/Users/ahmad/Projects/Side-Gigs/sa7beh-app/venv/lib/python3.6/site-packages/gunicorn-19.7.0-py3.6.egg/gunicorn/workers/async.py", line 56, in handle
    self.handle_request(listener_name, req, client, addr)
  File "/Users/ahmad/Projects/Side-Gigs/sa7beh-app/venv/lib/python3.6/site-packages/gunicorn-19.7.0-py3.6.egg/gunicorn/workers/ggevent.py", line 152, in handle_request
    super(GeventWorker, self).handle_request(*args)
  File "/Users/ahmad/Projects/Side-Gigs/sa7beh-app/venv/lib/python3.6/site-packages/gunicorn-19.7.0-py3.6.egg/gunicorn/workers/async.py", line 129, in handle_request
    six.reraise(*sys.exc_info())
  File "/Users/ahmad/Projects/Side-Gigs/sa7beh-app/venv/lib/python3.6/site-packages/gunicorn-19.7.0-py3.6.egg/gunicorn/six.py", line 625, in reraise
    raise value
  File "/Users/ahmad/Projects/Side-Gigs/sa7beh-app/venv/lib/python3.6/site-packages/gunicorn-19.7.0-py3.6.egg/gunicorn/workers/async.py", line 115, in handle_request
    resp.write(item)
  File "/Users/ahmad/Projects/Side-Gigs/sa7beh-app/venv/lib/python3.6/site-packages/gunicorn-19.7.0-py3.6.egg/gunicorn/http/wsgi.py", line 362, in write
    util.write(self.sock, arg, self.chunked)
  File "/Users/ahmad/Projects/Side-Gigs/sa7beh-app/venv/lib/python3.6/site-packages/gunicorn-19.7.0-py3.6.egg/gunicorn/util.py", line 321, in write
    sock.sendall(data)
  File "/Users/ahmad/Projects/Side-Gigs/sa7beh-app/venv/lib/python3.6/site-packages/gevent-1.2.1-py3.6-macosx-10.12-x86_64.egg/gevent/_socket3.py", line 418, in sendall
    data_sent += self.send(data_memory[data_sent:], flags)
  File "/Users/ahmad/Projects/Side-Gigs/sa7beh-app/venv/lib/python3.6/site-packages/gevent-1.2.1-py3.6-macosx-10.12-x86_64.egg/gevent/_socket3.py", line 391, in send
    return _socket.socket.send(self._sock, data, flags)
OSError: [Errno 41] Protocol wrong type for socket

----------
components: IO, macOS
messages: 316328
nosy: ned.deily, racitup, ronaldoussoren
priority: normal
severity: normal
status: open
title: unexpected EPROTOTYPE returned by sendto on MAC OSX
type: behavior
versions: Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33450>
_______________________________________

From report at bugs.python.org  Wed May  9 16:43:58 2018
From: report at bugs.python.org (Jean-Louis Tamburini)
Date: Wed, 09 May 2018 20:43:58 +0000
Subject: [New-bugs-announce] [issue33451] Start pyc file lock the file
Message-ID: <1525898638.9.0.682650639539.issue33451@psf.upfronthosting.co.za>


New submission from Jean-Louis Tamburini <jeanlouis at solutions66.com>:

Python v3.6.4:d48eceb
Windows 10.0.16299

I search on Internet and I don't find why Python Interpreter (v3.6.4) do that. With python 2.7, I don't have this "problem".

When I launch script like : 
c:\Python364\python.exe compile.py
While the execution, I can rename/modify the file "compile.py". 

But, when I compile the file to .pyc (with py_compile with an another script), and I launch the pyc like :
c:\Python364\python.exe dummy.pyc
While the execution, I CAN'T rename/modify the file because is locked by Python Interpreter. 

It's normal ?

----------
components: Interpreter Core
messages: 316330
nosy: Jean-Louis Tamburini
priority: normal
severity: normal
status: open
title: Start pyc file lock the file
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33451>
_______________________________________

From report at bugs.python.org  Wed May  9 16:50:00 2018
From: report at bugs.python.org (Rick Teachey)
Date: Wed, 09 May 2018 20:50:00 +0000
Subject: [New-bugs-announce] [issue33452] add user notification that parent
 init will not be called in dataclass init method
Message-ID: <1525899000.83.0.682650639539.issue33452@psf.upfronthosting.co.za>


New submission from Rick Teachey <ricky at teachey.org>:

The dataclasses module is incredibly easy to use. This is a good thing. BUT one downside is it will definitely be utilized by people who don't have a thorough understanding of how it does what it does.

Even for me, despite having a very good understanding of how it works, after heavily using it on a project for about 3 weeks now I made a mistake like the one below:

class ImportantMixin:
    def __init__(self):
        super().__init__()
        important_task()

@dataclass
class NaiveDClass(ImportantMixin):
  data1 = int
  data2 = int

I then went on along my merry way. Obviously, ImportantMixin.__init__ never gets called and I didn't realize this until it was a bigger problem than it should have been (should have written better tests! but I digress).

It would seem like a good idea for the dataclasses module to let the user know they did this, probably via the warning system. Seems like it would be relatively easy to do: if there is an init method being create, just inspect the MRO for any previously defined init methods that weren't created by dataclasses.

Thanks.

----------
components: Library (Lib)
messages: 316331
nosy: Ricyteach, eric.smith
priority: normal
severity: normal
status: open
title: add user notification that parent init will not be called in dataclass init method
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33452>
_______________________________________

From report at bugs.python.org  Wed May  9 17:38:42 2018
From: report at bugs.python.org (Rick Teachey)
Date: Wed, 09 May 2018 21:38:42 +0000
Subject: [New-bugs-announce] [issue33453] from __future__ import annotations
 breaks dataclasses ClassVar handling
Message-ID: <1525901922.71.0.682650639539.issue33453@psf.upfronthosting.co.za>


New submission from Rick Teachey <ricky at teachey.org>:

This is broken in 3.7 (both beta 3 and 4):

from __future__ import annotations
from dataclasses import dataclass
from typing import ClassVar, Any

@dataclass
class C():
    class_var: ClassVar[Any] = object()
    data: str

Traceback:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\ricky\AppData\Local\Programs\Python\Python37\lib\dataclasses.py", line 850, in dataclass
    return wrap(_cls)
  File "C:\Users\ricky\AppData\Local\Programs\Python\Python37\lib\dataclasses.py", line 842, in wrap
    return _process_class(cls, init, repr, eq, order, unsafe_hash, frozen)
  File "C:\Users\ricky\AppData\Local\Programs\Python\Python37\lib\dataclasses.py", line 763, in _process_class
    else 'self',
  File "C:\Users\ricky\AppData\Local\Programs\Python\Python37\lib\dataclasses.py", line 442, in _init_fn
    raise TypeError(f'non-default argument {f.name!r} '
TypeError: non-default argument 'data' follows default argument

----------
components: Library (Lib)
messages: 316333
nosy: Ricyteach, eric.smith
priority: normal
severity: normal
status: open
title: from __future__ import annotations breaks dataclasses ClassVar handling
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33453>
_______________________________________

From report at bugs.python.org  Thu May 10 06:17:56 2018
From: report at bugs.python.org (Serhiy Storchaka)
Date: Thu, 10 May 2018 10:17:56 +0000
Subject: [New-bugs-announce] [issue33454] Mismatched C function signature in
 _xxsubinterpreters.channel_close()
Message-ID: <1525947476.64.0.682650639539.issue33454@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:

The C function that implements _xxsubinterpreters.channel_close() is defined with the signature

    PyObject *channel_close(PyObject *self, PyObject *args, PyObject *kwds)

which corresponds the method convention METH_VARARGS | METH_KEYWORDS, but is used with the incompatible method convention METH_VARARGS. Either the signature or flags are not correct.

Actually it can use just METH_O. The proposed PR changes both signature and flags and simplifies the implementation.

The bug was found thanks to gcc 8 emitting a warning for invalid function cast (it also emits a lot of false alarms). See also issue33012.

----------
components: Extension Modules
messages: 316354
nosy: eric.snow, serhiy.storchaka, siddhesh
priority: normal
severity: normal
status: open
title: Mismatched C function signature in _xxsubinterpreters.channel_close()
type: behavior
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33454>
_______________________________________

From report at bugs.python.org  Thu May 10 09:50:36 2018
From: report at bugs.python.org (=?utf-8?q?Miro_Hron=C4=8Dok?=)
Date: Thu, 10 May 2018 13:50:36 +0000
Subject: [New-bugs-announce] [issue33455]
 test.test_posix.TestPosixSpawn::test_specify_environment fails with custom
 LD_LIBRARY_PATH
Message-ID: <1525960236.32.0.682650639539.issue33455@psf.upfronthosting.co.za>


New submission from Miro Hron?ok <miro at hroncok.cz>:

When we build Python in Fedora, we set LD_LIBRARY_PATH environment variable so the testsuite is run against the currently built Python.

However a test added in ef347535f289baad22c0601e12a36b2dcd155c3a (test_specify_environment) spawns a process without passing the environment variables. This means that the new process fails with 

    error while loading shared libraries: libpython3.7m.so.1.0: cannot open shared object file: No such file or directory

And the test fails with:

test_specify_environment (test.test_posix.TestPosixSpawn) ... /builddir/build/BUILD/Python-3.7.0b4/build/optimized/python: error while loading shared libraries: libpython3.7m.so.1.0: cannot open shared object file: No such file or directory
test test_posix failed
FAIL
======================================================================
FAIL: test_specify_environment (test.test_posix.TestPosixSpawn)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builddir/build/BUILD/Python-3.7.0b4/Lib/test/test_posix.py", line 1467, in test_specify_environment
    self.assertEqual(os.waitpid(pid, 0), (pid, 0))
AssertionError: Tuples differ: (11457, 32512) != (11457, 0)
First differing element 1:
32512
0
- (11457, 32512)
?         ^^^^^
+ (11457, 0)
?         ^
----------------------------------------------------------------------
Ran 101 tests in 0.608s
FAILED (failures=1, skipped=9)
1 test failed again:
    test_posix
Total duration: 23 min 22 sec
Tests result: FAILURE

I believe that a fix for this is to copy os.environ, update it with {'foo': 'bar'} and pass that copy. I'll check and send PR if it works.

----------
components: Tests
messages: 316363
nosy: hroncok, serhiy.storchaka
priority: normal
severity: normal
status: open
title: test.test_posix.TestPosixSpawn::test_specify_environment fails with custom LD_LIBRARY_PATH
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33455>
_______________________________________

From report at bugs.python.org  Thu May 10 10:57:34 2018
From: report at bugs.python.org (Lukas Waymann)
Date: Thu, 10 May 2018 14:57:34 +0000
Subject: [New-bugs-announce] [issue33456] site.py: by default,
 a virtual environment is *not* isolated from the system-level
 site-packages directories
Message-ID: <1525964254.05.0.682650639539.issue33456@psf.upfronthosting.co.za>


New submission from Lukas Waymann <meribold at gmail.com>:

PEP 405 says this:

> By default, a virtual environment is entirely isolated from the system-level site-packages directories.
>
> If the pyvenv.cfg file also contains a key include-system-site-packages with a value of true (not case sensitive), the site module will also add the system site directories to sys.path after the virtual environment site directories.

The documentation of the site module (https://docs.python.org/3/library/site.html) says (emphasis added):

> If pyvenv.cfg [?] contains the key include-system-site-packages set to anything other than false (case-insensitive), the system-level prefixes will still also be searched for site-packages; *otherwise they won?t*.

However, what actually happens in site.py is different: see <https://github.com/python/cpython/blob/3.6/Lib/site.py#L447>.  The system_site variable is initialized to "true" and doesn't change unless the key include-system-site-packages exists in pyvenv.cfg.

I think system_site should be initialized to "false" and the condition in line 468 should be `if system_site.lower() != "false"`.

----------
components: Library (Lib)
messages: 316367
nosy: meribold
priority: normal
severity: normal
status: open
title: site.py: by default, a virtual environment is *not* isolated from the system-level site-packages directories
type: behavior
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33456>
_______________________________________

From report at bugs.python.org  Thu May 10 11:42:35 2018
From: report at bugs.python.org (dimi)
Date: Thu, 10 May 2018 15:42:35 +0000
Subject: [New-bugs-announce] [issue33457] python-config ldflags,
 PEP 513 and explicit linking to libpython in python extensions
Message-ID: <1525966955.19.0.682650639539.issue33457@psf.upfronthosting.co.za>


New submission from dimi <dii.bogdanov at gmail.com>:

The python-config outputs ldflag for explicit linking to libpyhton, which is a good idea in many use cases. However, this is inconsistent with the PEP 513 recommendation for building manylinux1 python extensions (https://www.python.org/dev/peps/pep-0513/) that requires avoiding explicit linking to libpythonX.Y.so.1.

----------
components: Distutils
messages: 316368
nosy: dimi, dstufft, eric.araujo
priority: normal
severity: normal
status: open
title: python-config ldflags, PEP 513 and explicit linking to libpython in python extensions
type: behavior

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33457>
_______________________________________

From report at bugs.python.org  Thu May 10 16:27:37 2018
From: report at bugs.python.org (Xavier de Gaye)
Date: Thu, 10 May 2018 20:27:37 +0000
Subject: [New-bugs-announce] [issue33458] pdb.run() does not trace
 destructors of __main__
Message-ID: <1525984057.77.0.682650639539.issue33458@psf.upfronthosting.co.za>


New submission from Xavier de Gaye <xdegaye at gmail.com>:

This issue is a companion to issue 13044.

Running with Python 3.6.5 the following code fails with NameError:

1 class C:
2     def __del__(self):
3         print('deleted')
4
5 c = C()
6 x = 1

$  python -m pdb bar.py
> ./bar.py(1)<module>()
-> class C:
(Pdb) break 6
Breakpoint 1 at ./bar.py:6
(Pdb) continue
> ./bar.py(6)<module>()
-> x = 1
(Pdb) step
--Return--
> ./bar.py(6)<module>()->None
-> x = 1
(Pdb) step
--Return--
> <string>(1)<module>()->None
(Pdb) step
> /usr/lib/python3.6/bdb.py(438)run()
-> self.quitting = True
(Pdb) step
The program finished and will be restarted
Exception ignored in: <bound method C.__del__ of <__main__.C object at 0x7f3fb3485b70>>
Traceback (most recent call last):
  File "./bar.py", line 3, in __del__
    print('deleted')
NameError: name 'print' is not defined
> ./bar.py(1)<module>()
-> class C:
(Pdb)

----------
components: Library (Lib)
messages: 316375
nosy: xdegaye
priority: normal
severity: normal
status: open
title: pdb.run() does not trace destructors of __main__
type: behavior
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33458>
_______________________________________

From report at bugs.python.org  Thu May 10 19:16:52 2018
From: report at bugs.python.org (=?utf-8?q?Andr=C3=A9s_Delfino?=)
Date: Thu, 10 May 2018 23:16:52 +0000
Subject: [New-bugs-announce] [issue33459] Fix "tuple display" mention in
 Expressions
Message-ID: <1525994212.08.0.682650639539.issue33459@psf.upfronthosting.co.za>


New submission from Andr?s Delfino <adelfino at gmail.com>:

Expressions mentions "tuple displays" in 6.16 (Operator precedence).

AFAIK, there ano "tuple displays". Expressions mentions list, dict, and set displays, and then talks about generator expressions.

I guess "parenthesized expressions" should be the term that fits here?

----------
assignee: docs at python
components: Documentation
messages: 316380
nosy: adelfino, docs at python
priority: normal
severity: normal
status: open
title: Fix "tuple display" mention in Expressions
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33459>
_______________________________________

From report at bugs.python.org  Thu May 10 19:33:17 2018
From: report at bugs.python.org (Lew Kurtz)
Date: Thu, 10 May 2018 23:33:17 +0000
Subject: [New-bugs-announce] [issue33460] ... used to indicate many
 different things in chapter 3, some are confusing
Message-ID: <1525995197.73.0.682650639539.issue33460@psf.upfronthosting.co.za>


New submission from Lew Kurtz <lew.kurtz at icloud.com>:

https://docs.python.org/3/tutorial/introduction.html
uses ... to mean 5 or 6 different things. Since ... is continuation prompt, some of these other uses on that page are confusing - at least they were to this newbie. The most confusing are where it is used to hide output in the error messages.

If someone has the patience to send me pointers on how to fix this, I would love to fix this myself. -Lew

ps. the section headers in chapter 3 also need some work, 3.1.1 is the only section that is really about python being a calculator 3.1.2 and 3.1.3 don't seem very 'calculator-y' - Id like to fix this too.

----------
assignee: docs at python
components: Documentation
messages: 316381
nosy: docs at python, lew18
priority: normal
severity: normal
status: open
title: ... used to indicate many different things in chapter 3, some are confusing
type: enhancement
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33460>
_______________________________________

From report at bugs.python.org  Thu May 10 22:31:13 2018
From: report at bugs.python.org (Matthias Bussonnier)
Date: Fri, 11 May 2018 02:31:13 +0000
Subject: [New-bugs-announce] [issue33461] json.loads(encoding=<not None>)
 does not emit deprecation warning.
Message-ID: <1526005873.13.0.682650639539.issue33461@psf.upfronthosting.co.za>


New submission from Matthias Bussonnier <bussonniermatthias at gmail.com>:

The `encoding` keyword of json.loads is ignored and deprecated. AFAICT this is since Python 3.1. 

Passing a value for encoding does not emit a deprecation warning.

----------
messages: 316384
nosy: mbussonn
priority: normal
severity: normal
status: open
title: json.loads(encoding=<not None>) does not emit deprecation warning.

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33461>
_______________________________________

From report at bugs.python.org  Thu May 10 23:30:18 2018
From: report at bugs.python.org (Michael Selik)
Date: Fri, 11 May 2018 03:30:18 +0000
Subject: [New-bugs-announce] [issue33462] reversible dict
Message-ID: <1526009418.33.0.682650639539.issue33462@psf.upfronthosting.co.za>


New submission from Michael Selik <mike at selik.org>:

Now that dicts are tracking insertion order, they can be made reversible via the built-in reversed, just like OrderedDict.

----------
messages: 316386
nosy: selik
priority: normal
severity: normal
status: open
title: reversible dict
type: enhancement

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33462>
_______________________________________

From report at bugs.python.org  Thu May 10 23:31:03 2018
From: report at bugs.python.org (Michael Selik)
Date: Fri, 11 May 2018 03:31:03 +0000
Subject: [New-bugs-announce] [issue33463] Can namedtuple._asdict return a
 regular dict instead of OrderedDict?
Message-ID: <1526009463.33.0.682650639539.issue33463@psf.upfronthosting.co.za>


New submission from Michael Selik <mike at selik.org>:

Since the basic dict is now keeping insertion order, can we switch namedtuple._asdict to return a basic dict? Other than OrderedDict.move_to_end and the repr, I believe there is no compatibility issue.

----------
messages: 316387
nosy: selik
priority: normal
severity: normal
status: open
title: Can namedtuple._asdict return a regular dict instead of OrderedDict?
type: enhancement
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33463>
_______________________________________

From report at bugs.python.org  Fri May 11 07:50:27 2018
From: report at bugs.python.org (Michael Felt)
Date: Fri, 11 May 2018 11:50:27 +0000
Subject: [New-bugs-announce] [issue33464] AIX and "specialized downloads"
 links
Message-ID: <1526039427.77.0.682650639539.issue33464@psf.upfronthosting.co.za>


New submission from Michael Felt <michael at felt.demon.nl>:

I struggled with how python was packaged by others. Key points being: many dependencies, no assurance you could uninstall the dependencies and/or python.

A little over two years ago I started looking for 'root causes' of problems that potential users of python (on AIX) were having.

To me it is clear that the packaging problems from two years back are just as alive as they were in 2015 and 2016. Since then I have learned a few things about python (thanks to all who helped). I would like to apply this knowledge and provide a 'cleaner' packaging of Python for AIX - both Python3 and Python2.

I would appreciate my downloadable packages being listed on the "other" downloads list. If you wanted to make it complete - the other two I know of are Perzl's site (http://www.perzl.org/aix/index.php?n=main.python), and, of course, the IBM AIX Toolbox (https://www-03.ibm.com/systems/power/software/aix/linux/toolbox/alpha.html#P).

I have involved (currently managed and run it) an AIX portal since 2001 - http://www.rootvg.net. The last 5 or 6 years I have been hosting a second portal I call "aixtools".

In short, I hope you will consider adding - at least - my aixtools wiki pages: http://www.aixtools.net/index.php/python3 and http://www.aixtools.net/index.php/python (for python2).

Many thanks for your consideration,

Michael

----------
messages: 316395
nosy: Michael.Felt, gvanrossum
priority: normal
severity: normal
status: open
title: AIX and "specialized downloads" links
versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33464>
_______________________________________

From report at bugs.python.org  Fri May 11 12:07:04 2018
From: report at bugs.python.org (Matthias Klose)
Date: Fri, 11 May 2018 16:07:04 +0000
Subject: [New-bugs-announce] [issue33465]
 test_from_import_missing_attr_has_name_and_so_path fails when select is a
 builtin instead of an extension
Message-ID: <1526054824.37.0.682650639539.issue33465@psf.upfronthosting.co.za>


New submission from Matthias Klose <doko at debian.org>:

======================================================================
ERROR: test_from_import_missing_attr_has_name_and_so_path (test.test_import.ImportTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3.7/test/test_import/__init__.py", line 98, in test_from_import_missing_attr_has_name_and_so_path
    self.assertEqual(cm.exception.path, select.__file__)
AttributeError: module 'select' has no attribute '__file__'

----------------------------------------------------------------------
Ran 75 tests in 0.618s

FAILED (errors=1, skipped=1)
test test_import failed

----------
components: Tests
messages: 316403
nosy: brett.cannon, doko, eric.snow, ncoghlan
priority: normal
severity: normal
status: open
title: test_from_import_missing_attr_has_name_and_so_path fails when select is a builtin instead of an extension
type: behavior
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33465>
_______________________________________

From report at bugs.python.org  Fri May 11 12:59:44 2018
From: report at bugs.python.org (=?utf-8?q?Alexander_B=C3=B6hn?=)
Date: Fri, 11 May 2018 16:59:44 +0000
Subject: [New-bugs-announce] =?utf-8?q?=5Bissue33466=5D_Distutils_does_no?=
 =?utf-8?q?t_support_the_compilation_of_Objective-C++_=28=E2=80=9C=2Emm?=
 =?utf-8?b?4oCdKSBmaWxlcw==?=
Message-ID: <1526057984.85.0.682650639539.issue33466@psf.upfronthosting.co.za>


New submission from Alexander B?hn <fish2000 at gmail.com>:

Attempting to compile an Objective-C++ source file suffixed ?.mm? results in an error complaining about the file extension being unrecognized.

----------
components: Distutils
messages: 316405
nosy: dstufft, eric.araujo, fish2000
priority: normal
severity: normal
status: open
title: Distutils does not support the compilation of Objective-C++ (?.mm?) files
type: enhancement
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33466>
_______________________________________

From report at bugs.python.org  Fri May 11 14:22:19 2018
From: report at bugs.python.org (Rahul Ravindran)
Date: Fri, 11 May 2018 18:22:19 +0000
Subject: [New-bugs-announce] [issue33467] Python 3.7: profile-opt build
 errors because a test seems to hang
Message-ID: <1526062939.54.0.682650639539.issue33467@psf.upfronthosting.co.za>


New submission from Rahul Ravindran <rahulrv at gmail.com>:

make run_profile_task

runs the tests and does not seem to have any mechanism to exclude tests that I could find based on looking at the Makefile.

Previously, on Python 3.6, this test test_poplib was failing(https://bugs.python.org/issue32753) and the profile_task would ignore failing tests. 

Now, with the Python 3.7 build, the test seems to hang and hence profile opt builds cannot be built.
Attached is the trace of the build based on Python-3.7.0b4

----------
components: Build
files: stack_trace1.txt
messages: 316408
nosy: Rahul Ravindran
priority: normal
severity: normal
status: open
title: Python 3.7: profile-opt build errors because a test seems to hang
versions: Python 3.7
Added file: https://bugs.python.org/file47582/stack_trace1.txt

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33467>
_______________________________________

From report at bugs.python.org  Fri May 11 18:31:27 2018
From: report at bugs.python.org (Nick Coghlan)
Date: Fri, 11 May 2018 22:31:27 +0000
Subject: [New-bugs-announce] [issue33468] Add try-finally
 contextlib.contextmanager example
Message-ID: <1526077887.55.0.682650639539.issue33468@psf.upfronthosting.co.za>


New submission from Nick Coghlan <ncoghlan at gmail.com>:

The current example for contextlib.contextmanager doesn't use try/finally, which sets folks up for writing resource management context managers that don't clean up after exceptions properly.

There's an example with try/finally down in the contextlib.closing docs, but I'm thinking it would be worthwhile to change the presentation of contextmanager itself.

----------
assignee: ncoghlan
components: Documentation
messages: 316416
nosy: ncoghlan
priority: normal
severity: normal
stage: needs patch
status: open
title: Add try-finally contextlib.contextmanager example
type: enhancement
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33468>
_______________________________________

From report at bugs.python.org  Sat May 12 03:53:08 2018
From: report at bugs.python.org (=?utf-8?b?SHJ2b2plIE5pa8WhacSH?=)
Date: Sat, 12 May 2018 07:53:08 +0000
Subject: [New-bugs-announce] [issue33469] RuntimeError after closing loop
 that used run_in_executor
Message-ID: <1526111588.74.0.682650639539.issue33469@psf.upfronthosting.co.za>


New submission from Hrvoje Nik?i? <hniksic at gmail.com>:

Looking at a StackOverflow question[1], I was unable to find a way to correctly close an event loop that uses run_in_executor() for long-running tasks.

The question author tried to implement the following scenario:

1. create some tasks that use run_in_executor
2. run asyncio.wait(tasks, return_when=FIRST_EXCEPTION)
3. cancel pending tasks, if any
4. close the loop and continue with non-async work

However, when there are pending tasks from wait(), a RuntimeError is raised some time after step #4. In the test programs, it happens while waiting for the program to finish. I have attached a minimal example to reproduce the issue.

The immediate cause is that a callback installed by wrap_future() notices that the underlying concurrent.futures.Future is done and calls loop.call_soon_threadsafe() to copy the result to the asyncio.Future. call_soon_threadsafe() fails when the loop is closed.

This would be reasonable behavior if not for the fact that the code explicitly cancelled the asyncio future, and awaited it to ensure that the cancellation took effect. While it is clear that asyncio cannot interrupt a function already running in an executor, it should probably detach the connection between the concurrent future and the asyncio future, to prevent this kind of error (and possibly other problems).

For example, the _call_check_cancel closure in _chain_future could remove (or disable) the done_callback installed on source after the call to source.cancel(). Since at that point we know that destination (asyncio.Future) is already canceled, there is no longer value in invoking the done callback for source (concurrent.futures.Future).


[1]
https://stackoverflow.com/q/50279522/1600898

----------
files: executor-cancel
messages: 316420
nosy: hniksic
priority: normal
severity: normal
status: open
title: RuntimeError after closing loop that used run_in_executor
versions: Python 3.6, Python 3.7
Added file: https://bugs.python.org/file47583/executor-cancel

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33469>
_______________________________________

From report at bugs.python.org  Sat May 12 04:40:06 2018
From: report at bugs.python.org (=?utf-8?q?Miro_Hron=C4=8Dok?=)
Date: Sat, 12 May 2018 08:40:06 +0000
Subject: [New-bugs-announce] [issue33470] Changes from GH-1638 (GH-3575,
 bpo-28411) are not documented in Porting to Python 3.7
Message-ID: <1526114406.62.0.682650639539.issue33470@psf.upfronthosting.co.za>


New submission from Miro Hron?ok <miro at hroncok.cz>:

Background: gdb fails to build with Python 3.7 as described in https://bugzilla.redhat.com/show_bug.cgi?id=1577396

This is due to _PyImport_FixupBuiltin changing it's API. I feel that _underscored functions are probably not guaranteed to not change, however I miss anything about the following changes:

https://github.com/python/cpython/pull/1638
https://github.com/python/cpython/pull/3565
https://github.com/python/cpython/pull/3575

In here:

https://docs.python.org/3.7/whatsnew/3.7.html#porting-to-python-37
(or that page generally)

That page doesn't mention PyInterpreterState at all, so even if you decide that _PyImport_FixupBuiltin doesn't deserve a mention, maybe the general change does.

Thanks

----------
assignee: docs at python
components: Documentation
messages: 316421
nosy: docs at python, eric.snow, hroncok, vstinner
priority: normal
severity: normal
status: open
title: Changes from GH-1638 (GH-3575, bpo-28411) are not documented in Porting to Python 3.7
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33470>
_______________________________________

From report at bugs.python.org  Sat May 12 07:30:06 2018
From: report at bugs.python.org (David Vasseur)
Date: Sat, 12 May 2018 11:30:06 +0000
Subject: [New-bugs-announce] [issue33471] string format with 'n' failling
 with french locales
Message-ID: <1526124606.41.0.682650639539.issue33471@psf.upfronthosting.co.za>


New submission from David Vasseur <dvasseur at gmail.com>:

Python 3.6.5 (default, Apr 14 2018, 13:17:30) 
[GCC 7.3.1 20180406] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> print('{:n}'.format(int(12)))
12 <-- ok --

>>> import locale
>>> locale.setlocale(locale.LC_ALL, '')
'fr_FR.UTF-8'
>>> print('{:n}'.format(int(12)))
(empty) <-- error --
'{:n}'.format(int(12))
'??' <-- error --

please note that with >999 numbers, it's working fine:
>>> print('{:n}'.format(int(1000)))
1?000 <-- ok --

----------
components: Extension Modules
messages: 316426
nosy: David Vasseur
priority: normal
severity: normal
status: open
title: string format with 'n' failling with french locales
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33471>
_______________________________________

From report at bugs.python.org  Sat May 12 12:49:15 2018
From: report at bugs.python.org (Eitan Adler)
Date: Sat, 12 May 2018 16:49:15 +0000
Subject: [New-bugs-announce] [issue33472] build system incorrectly handles
 CC, CFLAGS, LDFLAGS, and related.
Message-ID: <1526143755.39.0.682650639539.issue33472@psf.upfronthosting.co.za>


New submission from Eitan Adler <eitan at eitanadler.com>:

There are a number of incorrect items in python's build system. 

Expected behavior:
- CC is correctly report as the compiler, and without any additional options
- CFLAGS is correctly reported for the required flags to the compiler
- CXXFLAGS is the same as CFLAGS, but for C++
- LDFLAGS is correctly report for linker flags

Actual behavior:
- CC is reported as a non-existing compiler plus an option
- C*FLAGS and LDFLAGS are not correctly reported

----------
components: Build
messages: 316429
nosy: Eitan Adler
priority: normal
pull_requests: 6458
severity: normal
status: open
title: build system incorrectly handles CC, CFLAGS, LDFLAGS, and related.
type: compile error

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33472>
_______________________________________

From report at bugs.python.org  Sat May 12 13:00:57 2018
From: report at bugs.python.org (Eitan Adler)
Date: Sat, 12 May 2018 17:00:57 +0000
Subject: [New-bugs-announce] [issue33473] build system incorrectly handles
 CC, CFLAGS, LDFLAGS, and related.
Message-ID: <1526144457.86.0.682650639539.issue33473@psf.upfronthosting.co.za>


New submission from Eitan Adler <lists at eitanadler.com>:

There are a number of incorrect items in python's build system. 

Expected behavior:
- CC is correctly report as the compiler, and without any additional options
- CFLAGS is correctly reported for the required flags to the compiler
- CXXFLAGS is the same as CFLAGS, but for C++
- LDFLAGS is correctly report for linker flags

Actual behavior:
- CC is reported as a non-existing compiler plus an option
- C*FLAGS and LDFLAGS are not correctly reported

----------
components: Build
messages: 316431
nosy: eitan.adler
priority: normal
pull_requests: 6459
severity: normal
status: open
title: build system incorrectly handles CC, CFLAGS, LDFLAGS, and related.
type: compile error

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33473>
_______________________________________

From report at bugs.python.org  Sat May 12 17:28:57 2018
From: report at bugs.python.org (Daniel Lindeman)
Date: Sat, 12 May 2018 21:28:57 +0000
Subject: [New-bugs-announce] [issue33474] Support immutability per-field in
 dataclasses
Message-ID: <1526160537.62.0.682650639539.issue33474@psf.upfronthosting.co.za>


New submission from Daniel Lindeman <lindemda at gmail.com>:

Currently, all fields in dataclasses can be frozen by passing frozen=True to the decorator. There are cases where it would be useful to have some fields be mutable, and others immutable. By using a strategy similar to how the decorator frozen works, using __setattr__ and __delattr__, it would be possible to support immutability per-field. This could perhaps be done by adding a frozen argument to the field constructor.

----------
messages: 316438
nosy: Daniel Lindeman, eric.smith
priority: normal
severity: normal
status: open
title: Support immutability per-field in dataclasses
type: enhancement
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33474>
_______________________________________

From report at bugs.python.org  Sat May 12 18:22:34 2018
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sat, 12 May 2018 22:22:34 +0000
Subject: [New-bugs-announce] [issue33475] Fix converting AST expression to
 string and optimize parenthesis
Message-ID: <1526163754.81.0.682650639539.issue33475@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:

The proposed patch fixes bugs in ast_unparse.c, makes it generating cleaner string representation, and makes the code cleaner.

Known fixed bugs:

1. Extended slices crash Python. "s[a, b:c]".

2. 1-tuples produce illegal syntax: "(a,)" => "(, a)".

3. Lambdas in f-strings produce illegal syntax. "f'{(lambda x: x)}'" => "f'{lambda x: x}'".

4. Some expressions that need parenthesis don't have them. E.g. "lambda x: (x, x)" => "lambda x: x, x".

5. Generators and yield expression always must be surrounded with parenthesis. "(x for x in y)" => "x for x in y", "(yield)" => "yield".

6. Top-level tuples must be surrounded with parenthesis. "(a, b)" => "a, b".

Produced string representation is now more clean and almost not contains redundant parenthesis.

"(a + b) * (c + d)" => "(a + b) * (c + d)"
"(a * b) + (c * d)" => "a * b + c * d"
"(a * b) * (c * d)" => "a * b * (c * d)"
"(a ** b) ** (c ** d)" => "(a ** b) ** c ** d"
"[(a + b)]" => "[a + b]"
"[(i ** 2) for i in range(1, (a + 1))]" => "[i ** 2 for i in range(1, a + 1)]"

Maybe other bugs were fixed in process.

And finally I use macros for simple repeated fragments of code like

        if (-1 == append_charp(writer, str)) {
            return -1;
        }

This made the meaningful code much shorter and saved around 250 lines of code.

----------
components: Interpreter Core
messages: 316439
nosy: lukasz.langa, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Fix converting AST expression to string and optimize parenthesis
type: behavior
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33475>
_______________________________________

From report at bugs.python.org  Sat May 12 20:50:02 2018
From: report at bugs.python.org (Enrique Perez-Terron)
Date: Sun, 13 May 2018 00:50:02 +0000
Subject: [New-bugs-announce] [issue33476] String index out of range in
 get_group(), email/_header_value_parser.py
Message-ID: <1526172602.81.0.682650639539.issue33476@psf.upfronthosting.co.za>


New submission from Enrique Perez-Terron <enrique at perezterron.net>:

When address group is missing final ';', 'value' will be an empty string. I suggest the following patch

$ diff -u _save_header_value_parser.py _header_value_parser.py
--- _save_header_value_parser.py        2018-03-14 01:07:54.000000000 +0100
+++ _header_value_parser.py     2018-05-13 02:17:13.830053600 +0200
@@ -1876,7 +1876,7 @@
     if not value:
         group.defects.append(errors.InvalidHeaderDefect(
             "end of header in group"))
-    if value[0] != ';':
+    elif value[0] != ';':
         raise errors.HeaderParseError(
             "expected ';' at end of group but found {}".format(value))
     group.append(ValueTerminal(';', 'group-terminator'))

----------
components: email
messages: 316440
nosy: Cacadril, barry, r.david.murray
priority: normal
severity: normal
status: open
title: String index out of range in get_group(), email/_header_value_parser.py
type: crash
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33476>
_______________________________________

From report at bugs.python.org  Sat May 12 20:57:30 2018
From: report at bugs.python.org (Matthias Bussonnier)
Date: Sun, 13 May 2018 00:57:30 +0000
Subject: [New-bugs-announce] [issue33477] Document that compile(code,
 'exec') has different behavior in 3.7+
Message-ID: <1526173050.51.0.682650639539.issue33477@psf.upfronthosting.co.za>


New submission from Matthias Bussonnier <bussonniermatthias at gmail.com>:

In recent Python the following 

>>> from ast import PyCF_ONLY_AST
>>> compile("'a'", 'whatever', 'exec', PyCF_ONLY_AST).body

In 3.6 it return

   [<_ast.Expr at 0x10b7441d0>] # that contail Str('a')

While on master:

   []

This is inconveninent for alternative repl like IPython, where basically if the user is entering a single string, the result is nothing in Python 3.7+, while it _does_ return something on earlier Python [1]. 

The documentation of `compile` says:

>  ... it can be 'exec' if source consists of a sequence of statements,

Which is not technically true any more as the first statement, if a string, will be removed. 

What's happening here is that since Python 3.7 if the _first_ statement is actually an expression containing a lonely string it is assign to the module docstring. So that's basically assuming you are parsing a module, and that the docstring make sens in this context, while in a REPL you are parsing a sucesssion of statements, in which case there is no need for a docstring that make no sens in this context.


This is _usually_ not an issue, unless this lonely statement is also the last, and what the user wants to execute in a REPL, in which case it has no side effect.

I don't have any objection to the new behavior, though I was wondering if this kind of side effect was anticipated. 

If that affect IPython, it will likely effect other alternative REPLs.

Thus, I believe it would be good to have this at least documented a tiny bit better and added in what's new, and potentially clarified in the `exec` docs.

I could argue that now the  `exec` name may be a tiny bit unsuitable for the new behavior, and would love if this could be optional.

1: https://github.com/ipython/ipython/issues/11133#issuecomment-388591332

----------
assignee: docs at python
components: Documentation, Interpreter Core
messages: 316441
nosy: docs at python, mbussonn
priority: normal
severity: normal
status: open
title: Document that compile(code, 'exec') has different behavior in 3.7+
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33477>
_______________________________________

From report at bugs.python.org  Sat May 12 23:54:59 2018
From: report at bugs.python.org (Amit Saha)
Date: Sun, 13 May 2018 03:54:59 +0000
Subject: [New-bugs-announce] [issue33478] PEP 8 CapWords reference wrong?
Message-ID: <1526183699.39.0.682650639539.issue33478@psf.upfronthosting.co.za>


New submission from Amit Saha <amitsaha.in at gmail.com>:

PEP 8 suggests class names and type variable names to be in CapWords case. However:

>>> import string
>>> string.capwords('CapWord')
'Capword'

Wondering if this this an oversight or am I misunderstanding something?

----------
assignee: docs at python
components: Documentation
messages: 316446
nosy: Amit.Saha, docs at python
priority: normal
severity: normal
status: open
title: PEP 8 CapWords reference wrong?

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33478>
_______________________________________

From report at bugs.python.org  Sun May 13 00:02:32 2018
From: report at bugs.python.org (Terry J. Reedy)
Date: Sun, 13 May 2018 04:02:32 +0000
Subject: [New-bugs-announce] [issue33479] Document tkinter and threads
Message-ID: <1526184152.45.0.682650639539.issue33479@psf.upfronthosting.co.za>


New submission from Terry J. Reedy <tjreedy at udel.edu>:

(Proposed patch below)

Library Reference Chapter 25, Graphical User Interfaces with Tk, covers tinter, some of its subpackages, and IDLE.  The introduction, https://docs.python.org/3/library/tk.html states "the internal module _tkinter provides a threadsafe mechanism which allows Python and Tcl to interact."  Or as Martin Loewis claimed, "Tkinter is thread-safe." (#11077, msg127979).

Unfortunately, the unqualified 'threadsafe' claim is not true.  If Tcl is compiled with thread support (the default for 8.6) worker thread calls work (for the examples I have tested).  If not (the default for 8.5 and before), they are unreliable and can fail in multiple ways.  Known factors include the number of worker threads making call, the type of calls, and the number of calls.

The deceptive claim and the lack of knowledge about the crucial role of how tcl is compiled has negative effects.  First, people who have non-thread tcl have suffered trying to deal with random failures.  (See issues listed in #11077, msg183774, and #33257.)   Second, non-coredev tkinter experts have spread the equally wrong claim that 'tkinter is not threadsafe".  (See the same message.)  So people who have do have thread tcl are discouraged from exploiting the fact.

I suggest

1. Remove 'threadsafe' from the intro sentence.  After it, add "See <Tkinter and threads> for more details." <..> indicates a link to the proposed new section below.

2. The tkinter section itself, https://docs.python.org/3/library/tkinter.html, currently says nothing about threads.  I propose to add a new subsection, location to be decided.
"""
Tkinter and threads.

If you want to use both tkinter and threads, the safest method is to make all tkinter calls in the main thread.  If worker threads generate data needed for tkinter calls, use a queue.Queue to send the data to the main thread.  For a clean shutdown, add a method to wait for threads to stop and have it called when the window close button [X] is pressed.

If you are using Tcl/Tk compiled with thread support, you can skip the queue and call tkinter methods in the worker threads.  To avoid deadlocks when shutting down, you may have to join working threads in a separate 'join' thread that does not make tkinter calls.

If you are using Tcl/Tk compiled without thread support, tkinter calls in worker threads may initially work, but may eventually fail somewhat randomly.  Known factors include the number of threads, the type of call, and the number of calls.

To determine whether your tcl/tk has thread support, look in <install-dir>/Lib/DLLs for files such as, tcl86(t).dll and tk86(t).dll (The tcl/tk version would then be '8.6'.)  The t for 'thread' suffix is present or absent as tcl/tk was compiled with or without thread support.
<Add info for Linux and MacOS.>

<Consider adding code examples.>
"""

----------
messages: 316447
nosy: serhiy.storchaka, terry.reedy
priority: normal
severity: normal
stage: patch review
status: open
title: Document tkinter and threads
type: behavior
versions: Python 2.7, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33479>
_______________________________________

From report at bugs.python.org  Sun May 13 04:11:58 2018
From: report at bugs.python.org (brent s.)
Date: Sun, 13 May 2018 08:11:58 +0000
Subject: [New-bugs-announce] [issue33480] Improvement suggestions for
 urllib.parse.urlparser
Message-ID: <1526199118.32.0.682650639539.issue33480@psf.upfronthosting.co.za>


New submission from brent s. <brent.saner at gmail.com>:

Currently, a parsed urlparse() object looks (roughly) like this:

urlparse('http://example.com/foo;key1=value1?key2=value2#key3=value3#key4=value4')

returns:

ParseResult(scheme='http', netloc='example.com', path='/foo', params='key1=value1', query='key2=value2', fragment='key3=value3#key4=value4')

However, I recommend a couple things:

0.) that ParseResult objects support dict emulation. e.g. one can run:

        dict(parseresult_obj)

    and get (using the example string above (corrected classification for RFC2986 compliance and common usage):

        {'fragment': [('key4', 'value4')],
         'netloc': 'foo.tld',
         'params': [('key2', 'value2')],
         'path': '/foo',
         'query': [('key3', 'value3')],
         'scheme': 'http'}

    Obviously, fragment, params, and query could instead be serialized into a nested dict. I'm not sure which is more preferred in the pythonic sense.

1.) Better RFC3986 compliance.
    Per RFC3986 ? 3 (https://tools.ietf.org/html/rfc3986#section-3), the URL can be further split into separate components. For instance, while considered deprecated, should "userinfo" (e.g. "http://user:password at ...") be parsed? At the very least, the port should be parsed out to a separate component from the netloc (or userinfo parsed out separate from netloc) - this will assist in parsing host:port combinations in netlocs that contain both userinfo and a specified port (and allow the port to be given as an int type, thus more easily used in e.g. the socket lib).

2.) If a component is not present, I suggest it be a None object instead of an empty string.
    e.g.:

        urlparse('http://example.com/foo')

    Would return:

        ParseResult(scheme='http', netloc='example.com', path='/foo', params=None, query=None, fragment=None)

    instead of

        ParseResult(scheme='http', netloc='example.com', path='/foo', params='', query='', fragment='')

----------
components: Library (Lib)
messages: 316454
nosy: bsaner
priority: normal
severity: normal
status: open
title: Improvement suggestions for urllib.parse.urlparser
type: enhancement

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33480>
_______________________________________

From report at bugs.python.org  Sun May 13 06:26:00 2018
From: report at bugs.python.org (David James Peters)
Date: Sun, 13 May 2018 10:26:00 +0000
Subject: [New-bugs-announce] [issue33481] configparser.write() does not save
 comments.
Message-ID: <1526207160.12.0.682650639539.issue33481@psf.upfronthosting.co.za>


New submission from David James Peters <pebaudhi at gmail.com>:

The ConfigParser().write() does not save the comments; this makes using comments harder because it requires a separate demo ini file the user must be able to locate and read from without learning anything from the INI file they are using. Not Good. Pls make configparser save the comments or guide me to what I can do to fully support comments in my app. I am still reading the configparser documentation but it says its heavily customizable. Saving and loading comments is a must for my ini files.

----------
components: Extension Modules
messages: 316457
nosy: pebaudhi
priority: normal
severity: normal
status: open
title: configparser.write() does not save comments.
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33481>
_______________________________________

From report at bugs.python.org  Sun May 13 10:33:47 2018
From: report at bugs.python.org (Jelle Zijlstra)
Date: Sun, 13 May 2018 14:33:47 +0000
Subject: [New-bugs-announce] [issue33482] codecs.StreamRecoder.writelines is
 broken
Message-ID: <1526222027.51.0.682650639539.issue33482@psf.upfronthosting.co.za>


New submission from Jelle Zijlstra <jelle.zijlstra at gmail.com>:

codecs.StreamRecoder.writelines is implemented as:

    def writelines(self, list):

        data = ''.join(list)
        data, bytesdecoded = self.decode(data, self.errors)
        return self.writer.write(data)

It can't take a list of bytes because then the ''.join throws an error, and it can't take a list of str because self.decode takes bytes.

It looks like bytes are intended (self.write takes bytes), so I'll submit a simple PR to fix this.

----------
messages: 316465
nosy: Jelle Zijlstra
priority: normal
severity: normal
status: open
title: codecs.StreamRecoder.writelines is broken
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33482>
_______________________________________

From report at bugs.python.org  Sun May 13 14:28:54 2018
From: report at bugs.python.org (Eitan Adler)
Date: Sun, 13 May 2018 18:28:54 +0000
Subject: [New-bugs-announce] [issue33483] build system requires explicit
 compiler, but should discover it
Message-ID: <1526236134.66.0.682650639539.issue33483@psf.upfronthosting.co.za>


New submission from Eitan Adler <lists at eitanadler.com>:

configure.ac requires explicit configuration for finding the preferred compiler. This results in a non-native way of configuring the system, and does not actually work on most platforms.

Expected behavior:

CC is used to discover the C compiler; CPP, LD, CXX, etc. are used to discover their appropriate tools

Actual: 
--without-gcc or --with-icc must be explicitly passed

----------
components: Build
messages: 316468
nosy: eitan.adler
priority: normal
severity: normal
status: open
title: build system requires explicit compiler, but should discover it
type: compile error

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33483>
_______________________________________

From report at bugs.python.org  Sun May 13 14:38:36 2018
From: report at bugs.python.org (Eitan Adler)
Date: Sun, 13 May 2018 18:38:36 +0000
Subject: [New-bugs-announce] [issue33484] build system runs when it merely
 link
Message-ID: <1526236716.08.0.682650639539.issue33484@psf.upfronthosting.co.za>


New submission from Eitan Adler <lists at eitanadler.com>:

The build system attempts to run certain test code when it can actually just link the code. This is a minor performance optimization but is really a semantic correctness issue.

----------
components: Build
messages: 316471
nosy: eitan.adler
priority: normal
severity: normal
status: open
title: build system runs when it merely link

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33484>
_______________________________________

From report at bugs.python.org  Sun May 13 14:51:58 2018
From: report at bugs.python.org (Eitan Adler)
Date: Sun, 13 May 2018 18:51:58 +0000
Subject: [New-bugs-announce] [issue33485] autoconf target does not
Message-ID: <1526237518.92.0.682650639539.issue33485@psf.upfronthosting.co.za>


New submission from Eitan Adler <lists at eitanadler.com>:

The current "autoconf" target is suboptimal in a few ways:
- it does not use a quoted srcdir
- it uses autoheader and autoconf instead of autoreconf
- it does act sanely if 'cd' fails.

----------
components: Build
messages: 316472
nosy: eitan.adler
priority: normal
severity: normal
status: open
title: autoconf target does not
type: behavior

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33485>
_______________________________________

From report at bugs.python.org  Sun May 13 15:08:03 2018
From: report at bugs.python.org (Eitan Adler)
Date: Sun, 13 May 2018 19:08:03 +0000
Subject: [New-bugs-announce] [issue33486] regen autotools related files
Message-ID: <1526238483.3.0.682650639539.issue33486@psf.upfronthosting.co.za>


New submission from Eitan Adler <lists at eitanadler.com>:

There are few if any functional changes, but this is helpful when making other changes to the build system.

----------
components: Build
messages: 316477
nosy: eitan.adler
priority: normal
severity: normal
status: open
title: regen autotools related files
type: enhancement

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33486>
_______________________________________

From report at bugs.python.org  Sun May 13 17:42:24 2018
From: report at bugs.python.org (Matthias Bussonnier)
Date: Sun, 13 May 2018 21:42:24 +0000
Subject: [New-bugs-announce] [issue33487] BZ2File(buffering=None) does not
 emit deprecation warning, deprecation version not documented.
Message-ID: <1526247744.31.0.682650639539.issue33487@psf.upfronthosting.co.za>


New submission from Matthias Bussonnier <bussonniermatthias at gmail.com>:

BZ2File(buffering=None) should emit a deprecation warning as the argument is deprecated.

The deprecation message and documentation should say since when it is deprecated. (AFAICT since 3.3)

----------
messages: 316482
nosy: mbussonn
priority: normal
severity: normal
status: open
title: BZ2File(buffering=None) does not emit deprecation warning, deprecation version not documented.

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33487>
_______________________________________

From report at bugs.python.org  Sun May 13 20:40:40 2018
From: report at bugs.python.org (Eitan Adler)
Date: Mon, 14 May 2018 00:40:40 +0000
Subject: [New-bugs-announce] [issue33488] github pull request template does
 not satisfy markdownlint
Message-ID: <1526258440.92.0.682650639539.issue33488@psf.upfronthosting.co.za>


New submission from Eitan Adler <lists at eitanadler.com>:

markdownlint is, as the name implies, a tool for linting markdown files.

The current template has the following lint warnings:

```
./.github/PULL_REQUEST_TEMPLATE.md:8: MD031 Fenced code blocks should be surrounded by blank lines
./.github/PULL_REQUEST_TEMPLATE.md:10: MD031 Fenced code blocks should be surrounded by blank lines
./.github/PULL_REQUEST_TEMPLATE.md:19: MD031 Fenced code blocks should be surrounded by blank lines
./.github/PULL_REQUEST_TEMPLATE.md:21: MD031 Fenced code blocks should be surrounded by blank lines
```

These should be fixed

----------
assignee: docs at python
components: Documentation
messages: 316484
nosy: docs at python, eitan.adler
priority: normal
severity: normal
status: open
title: github pull request template does not satisfy markdownlint
type: enhancement

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33488>
_______________________________________

From report at bugs.python.org  Sun May 13 21:25:29 2018
From: report at bugs.python.org (Terry J. Reedy)
Date: Mon, 14 May 2018 01:25:29 +0000
Subject: [New-bugs-announce] [issue33489] Newer externals for windows do not
 always trigger rebuild
Message-ID: <1526261129.26.0.682650639539.issue33489@psf.upfronthosting.co.za>


New submission from Terry J. Reedy <tjreedy at udel.edu>:

I added a new 2.7 worktree, with 2.7 checked out to track origin (my github fork).  I built Python with tcl/tk 8.5.15.  Whoops, my fork had 2.7 at 2.7.13+.  Fetch upstream, merge upstream/2.7, and rebuild.  I noticed that tcl/tk 8.5.19 was downloaded.  But IDLE subsequently reported that tk was still at 8.5.15.  I renamed the 8.5.15 and the compiled tcltk directories, built again, and tcltk was recompiled to 8.5.19.  It seems that when new tcl/tk externals are downloaded, the compiled tcltk should be deleted.

If I understand, this part of the output says that there was no recompile of tcl/tk.

  _hashlib.vcxproj -> f:\dev\27\PCBuild\_hashlib_d.pyd
  _tkinter.c
     Creating library f:\dev\27\PCBuild\_tkinter_d.lib and object f:\dev\27\PCBuild\_tkinter_d.e
  xp
  _tkinter.vcxproj -> f:\dev\27\PCBuild\_tkinter_d.pyd
  _bsddb.c

My installed 2.7.15 also has tcl/tk at 8.5.15.  If tcl/tk was upgraded before this recent release, the upgrade did not take effect.  

When tcl/tk has been upgraded during 3.x alpha/beta periods, tcl/tk seems to have been recompiled when it should be.

The upstream update also caused the following updates:
Fetching bsddb-4.7.25.0...
Fetching openssl-1.0.2o...
Fetching sqlite-3.14.2.0...

There were new compiles to build
f:\dev\27\PCBuild\_sqlite3_d.pyd
f:\dev\27\PCBuild\ssleay_d.lib
f:\dev\27\PCBuild\_bsddb_d.pyd
so I presume these are ok and I can just delete the old directories.

----------
components: Build, Windows
messages: 316487
nosy: paul.moore, steve.dower, terry.reedy, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Newer externals for windows do not always trigger rebuild
type: behavior
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33489>
_______________________________________

From report at bugs.python.org  Sun May 13 23:37:49 2018
From: report at bugs.python.org (Eitan Adler)
Date: Mon, 14 May 2018 03:37:49 +0000
Subject: [New-bugs-announce] [issue33490] pthread auto-detection can use
 AX_PTHREAD
Message-ID: <1526269069.92.0.682650639539.issue33490@psf.upfronthosting.co.za>


New submission from Eitan Adler <lists at eitanadler.com>:

There is a ton of logic in configure.ac to detect pthreads support. There is a standard macro to use instead: https://www.gnu.org/software/autoconf-archive/ax_pthread.html#ax_pthread 

In order to simplify our pthread logic we should make use of the standard macro instead of writing our own code.

----------
components: Build
messages: 316491
nosy: eitan.adler
priority: normal
severity: normal
status: open
title: pthread auto-detection can use AX_PTHREAD
type: enhancement

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33490>
_______________________________________

From report at bugs.python.org  Mon May 14 03:38:51 2018
From: report at bugs.python.org (Ivan Gushchin)
Date: Mon, 14 May 2018 07:38:51 +0000
Subject: [New-bugs-announce] [issue33491] mistype of method's name
Message-ID: <1526283531.17.0.682650639539.issue33491@psf.upfronthosting.co.za>


New submission from Ivan Gushchin <gui74327 at gmail.com>:

This link 
https://docs.python.org/2/library/unittest.html#unittest.SkipTest

names method from Capital letter. At the same time method name (according to help(unittest) ) is skipTest (starting lowercase). For those who reads documentation online this mistype leads to error.

----------
assignee: docs at python
components: Documentation
messages: 316493
nosy: Ivan Gushchin, docs at python
priority: normal
severity: normal
status: open
title: mistype of method's name
type: enhancement
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33491>
_______________________________________

From report at bugs.python.org  Mon May 14 04:14:24 2018
From: report at bugs.python.org (Martijn Pieters)
Date: Mon, 14 May 2018 08:14:24 +0000
Subject: [New-bugs-announce] [issue33492] Updating the Evaluation order
 section to cover *expression in calls
Message-ID: <1526285664.99.0.682650639539.issue33492@psf.upfronthosting.co.za>


New submission from Martijn Pieters <mj at python.org>:

Can the *Evaluation order* (_evalorder) section in reference/expressions.rst please be updated to cover this exception in a *call* primary (quoting from the _calls section):


    A consequence of this is that although the ``*expression`` syntax may appear
    *after* explicit keyword arguments, it is processed *before* the
    keyword arguments (and any ``**expression`` arguments -- see below).  So::

This exception to the normal expression evaluation order is rather buried in the _calls section only.

----------
assignee: docs at python
components: Documentation
messages: 316494
nosy: docs at python, mjpieters
priority: normal
severity: normal
status: open
title: Updating the Evaluation order section to cover *expression in calls
versions: Python 3.5, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33492>
_______________________________________

From report at bugs.python.org  Mon May 14 06:32:30 2018
From: report at bugs.python.org (Eric V. Smith)
Date: Mon, 14 May 2018 10:32:30 +0000
Subject: [New-bugs-announce] [issue33493] dataclasses: allow keyword-only
 arguments
Message-ID: <1526293950.07.0.682650639539.issue33493@psf.upfronthosting.co.za>


New submission from Eric V. Smith <eric at trueblade.com>:

I've had several requests for keyword-only arguments. This is a placeholder to remind me to work on it. I have not decided if it's a good idea or not.

I propose adding a keyword_only argument to field(), defaulting to False.

I'm thinking that the basic idea would be to put all keyword-only fields at the end of the arguments to __init__, but for all other uses, leave them where they appear in the class definition. That way comparison operations, in particular, would use the fields as they appear in the class definition (which is the current behavior). Since they'd be at the end of __init__, and since order doesn't matter (they're keyword-only, after all), then this would work as expected for base classes.

That is, given:

@dataclass
class B:
    a: field(type=int, keyword_only=True)
    b: int

@dataclass
class C(B):
    c: int
    d: field(type=int, keyword_only=True)

Then B's __init__ would take (b, c, *, a, d) as its arguments, but its comparison functions would compare the tuples as (a, b, c, d).

It would be an error for a ClassVar field to be keyword-only. I think it would be okay if an InitVar field were keyword-only, but I haven't given it a lot of thought.

----------
assignee: eric.smith
components: Library (Lib)
messages: 316498
nosy: eric.smith
priority: normal
severity: normal
status: open
title: dataclasses: allow keyword-only arguments
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33493>
_______________________________________

From report at bugs.python.org  Mon May 14 07:36:38 2018
From: report at bugs.python.org (Steven D'Aprano)
Date: Mon, 14 May 2018 11:36:38 +0000
Subject: [New-bugs-announce] [issue33494] random.choices ought to check that
 cumulative weights are in ascending order
Message-ID: <1526297798.91.0.682650639539.issue33494@psf.upfronthosting.co.za>


New submission from Steven D'Aprano <steve+python at pearwood.info>:

As mentioned on the Python-List:

https://mail.python.org/pipermail/python-list/2018-May/733061.html

random.choices() silently returns the wrong values when cumulative weights are not given, i.e. if the user misreads the documentation and provides non-cumulative weights, expecting that cumulative weights will be constructed for them.

I think that the documentation should be more clear, and preferably the choices() function ought to fail early when given invalid weights.

----------
components: Library (Lib)
messages: 316499
nosy: mark.dickinson, rhettinger, steven.daprano
priority: normal
severity: normal
status: open
title: random.choices ought to check that cumulative weights are in ascending order
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33494>
_______________________________________

From report at bugs.python.org  Mon May 14 08:36:16 2018
From: report at bugs.python.org (Eric V. Smith)
Date: Mon, 14 May 2018 12:36:16 +0000
Subject: [New-bugs-announce] [issue33495] dataclasses: repr of Field objects
 should use repr of each member
Message-ID: <1526301376.3.0.682650639539.issue33495@psf.upfronthosting.co.za>


New submission from Eric V. Smith <eric at trueblade.com>:

This is especially true for the "type" member, since it might be a string that looks like a type name (depending on how #33453 is resolved). But repr should be used for all Field members.

----------
assignee: eric.smith
components: Library (Lib)
messages: 316502
nosy: eric.smith, ned.deily
priority: release blocker
severity: normal
status: open
title: dataclasses: repr of Field objects should use repr of each member
type: behavior
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33495>
_______________________________________

From report at bugs.python.org  Mon May 14 10:24:22 2018
From: report at bugs.python.org (devala)
Date: Mon, 14 May 2018 14:24:22 +0000
Subject: [New-bugs-announce] [issue33496] Accept Pathlib paths for sqlite
 file
Message-ID: <1526307862.09.0.682650639539.issue33496@psf.upfronthosting.co.za>


New submission from devala <augustz at augustz.com>:

I'd love to be able to pass pathlib paths to sqlite's connect

conn = sqlite3.connect(DB_FILE)
TypeError: argument 1 must be str

The workaround is not hard (str(DB_FILE), but reducing friction in using pathlib would be great.

----------
components: Library (Lib)
messages: 316508
nosy: devala
priority: normal
severity: normal
status: open
title: Accept Pathlib paths for sqlite file
type: enhancement
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33496>
_______________________________________

From report at bugs.python.org  Mon May 14 10:30:10 2018
From: report at bugs.python.org (Amber Brown)
Date: Mon, 14 May 2018 14:30:10 +0000
Subject: [New-bugs-announce] [issue33497] cgi.parse_multipart does not have
 an associated "errors" parameter with its "encoding" parameter
Message-ID: <1526308210.26.0.682650639539.issue33497@psf.upfronthosting.co.za>


New submission from Amber Brown <hawkowl at atleastfornow.net>:

Lack of this parameter means that you may get an exception (if the incoming data is an invalid encoding) you can not get around.

This causes Twisted to be unable to provide a compatible API on Python 3.7.

----------
messages: 316510
nosy: hawkowl
priority: normal
severity: normal
status: open
title: cgi.parse_multipart does not have an associated "errors" parameter with its "encoding" parameter
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33497>
_______________________________________

From report at bugs.python.org  Mon May 14 10:30:35 2018
From: report at bugs.python.org (Aaron Hall)
Date: Mon, 14 May 2018 14:30:35 +0000
Subject: [New-bugs-announce] [issue33498] pathlib.Path wants an rmtree method
Message-ID: <1526308235.34.0.682650639539.issue33498@psf.upfronthosting.co.za>


New submission from Aaron Hall <aaronchall at yahoo.com>:

pathlib.Path wants the rmtree method from shutil

I think we need this method for a couple of reasons.

1. in shell, rm has the -r flag - In Python, we use shutil.rmtree as a best practice for this.

2. I prefer to teach my students about pathlib.Path as opposed to other ways of dealing with files. It's a great abstraction. But it's somewhat leaky, especially when it comes to recursively deleting a directory with its contents, as I now need to import rmtree from shutil.

Perhaps we need this as a method in the abstract base class that recursively uses the methods provided by the concrete implementations. I can look at the rmtree method for a reference implementation. 

Perhaps we should just give Path.rmdir a default recursive argument? Default would be False, of course, to retain current behavior.

----------
components: Library (Lib)
messages: 316511
nosy: Aaron Hall
priority: normal
severity: normal
status: open
title: pathlib.Path wants an rmtree method
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33498>
_______________________________________

From report at bugs.python.org  Mon May 14 11:12:49 2018
From: report at bugs.python.org (Carl Meyer)
Date: Mon, 14 May 2018 15:12:49 +0000
Subject: [New-bugs-announce] [issue33499] Environment variable to set
 alternate location for pycache tree
Message-ID: <1526310769.71.0.682650639539.issue33499@psf.upfronthosting.co.za>


New submission from Carl Meyer <carl at oddbird.net>:

We would like to set an environment variable that would cause Python to read and write `__pycache__` directories from a separate location on the filesystem (outside the source code tree). We have two reasons for this:

1. In our development setup (with a webserver running in a container on the dev-tree code), the `__pycache__` directories end up root-owned, and managing permissions on them so that they don't disrupt VCS operations on the code repo is untenable. (Currently we use PYTHONDONTWRITEBYTECODE as a workaround, but we have enough code that this costs us multiple seconds of developer time on every restart; we'd like to take advantage of cached bytecode without requiring that it pollute the code tree.)

2. In addition to just _having_ cached bytecode, we'd like to keep it on a ramdisk to minimize filesystem overhead.

Proposal: a `PYTHON_BYTECODE_PATH` environment variable. If set, `source_from_cache` and `cache_from_source` in `importlib._bootstrap_external` will respect it, creating a directory tree under that prefix that mirrors the source tree.

----------
messages: 316518
nosy: brett.cannon, carljm, eric.snow, lukasz.langa, ncoghlan
priority: normal
severity: normal
status: open
title: Environment variable to set alternate location for pycache tree
type: enhancement

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33499>
_______________________________________

From report at bugs.python.org  Mon May 14 11:38:44 2018
From: report at bugs.python.org (Michael Romero)
Date: Mon, 14 May 2018 15:38:44 +0000
Subject: [New-bugs-announce] [issue33500] Python TKinter for Mac on latest
 2.7.15 still extremely slow vs Windows
Message-ID: <1526312324.63.0.682650639539.issue33500@psf.upfronthosting.co.za>


New submission from Michael Romero <romerom at gmail.com>:

Hello, 

I have recently been working with a PAC-MAN project developed by Berkeley for their A.I. course.  My primary workstation is a Touchbar MBP (3.1ghz core i7 w/16gb RAM and an SSD).  I had noticed how PAC-MAN's performance would sort of... stutter, when running on my Mac, while performing incredibly quick on a nearby gaming computer.  I had previously believed it to be due to the differences in specs between systems.  This is not the case, however, as the performance of PAC-MAN is extremely quick even within a VM ran via Parallels on the same Macbook whose native python performance was extremely sluggish.

I have seen recent issues submitted here and supposedly resolved that are intended to address a performance issue with Tkinter and Mac OS/X.  While the underlying issue mentioned in those particular tickets may have been addressed, the overall performance difference between Mac and Windows (and perhaps others) with Tkinter on Mac has not been. 

The following URL includes the PAC-MAN implementation I'm referring to:
http://ai.berkeley.edu/project_log.html

The link above contains another link to download the "multiagent.zip".
https://s3-us-west-2.amazonaws.com/cs188websitecontent/projects/release/multiagent/v1/002/multiagent.zip

After extraction, running "python pacman.py" with a system running the most recent 2.7.15 64bit release (on a fully patched OS/X) will allow you to see the sluggish performance of PAC-MAN I'm describing when compared to the same exact codebase running on a Windows system (and perhaps linux, though I have not yet tried to run it there), on a Windows VM living on the same system.

----------
components: Tkinter
messages: 316520
nosy: Michael Romero
priority: normal
severity: normal
status: open
title: Python TKinter for Mac on latest 2.7.15 still extremely slow vs Windows
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33500>
_______________________________________

From report at bugs.python.org  Mon May 14 12:18:37 2018
From: report at bugs.python.org (Carl Meyer)
Date: Mon, 14 May 2018 16:18:37 +0000
Subject: [New-bugs-announce] [issue33501] split existing optimization levels
 into granular options
Message-ID: <1526314717.24.0.682650639539.issue33501@psf.upfronthosting.co.za>


New submission from Carl Meyer <carl at oddbird.net>:

It doesn't make sense for e.g. docstring-stripping to necessarily imply assert-stripping. These are totally separate options, useful for separate reasons, but currently tied together in the `-O` option.

This is not just a theoretical problem; at work we must strip docstrings in production for memory reasons, but we would prefer not to strip asserts. In fact we currently lint against use of `assert` because it is stripped in production, and we replace it with our own assertion function, which is less efficient and also integrates poorly with mypy's type binder.

A better option would be to enable each of these separate optimizations with a separate command-line flag (probably a string tag passed to a single flag, e.g. `-o strip_docstrings`).

PYC filename generation will also need to include all individually-enabled optimization string tags as part of the filename.

For backwards-compatibility, the existing `-O` flags should still be supported with the same meaning they currently have; `-O` and the new granular `-o` should be additive.

(A version of this was previously proposed as a minor part of PEP 511.)

Please let me know if this proposal is of sufficient complexity that a PEP is needed instead of just an issue.

----------
messages: 316531
nosy: brett.cannon, carljm, eric.snow, lukasz.langa, ncoghlan, vstinner
priority: normal
severity: normal
status: open
title: split existing optimization levels into granular options
type: enhancement

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33501>
_______________________________________

From report at bugs.python.org  Mon May 14 13:42:19 2018
From: report at bugs.python.org (Eric V. Smith)
Date: Mon, 14 May 2018 17:42:19 +0000
Subject: [New-bugs-announce] [issue33502] dataclasses: repr of
 _DataclassParams objects should use repr of each member
Message-ID: <1526319739.29.0.682650639539.issue33502@psf.upfronthosting.co.za>


New submission from Eric V. Smith <eric at trueblade.com>:

This is mostly a debugging aid, but if you need it, it's very handy.

----------
assignee: eric.smith
components: Library (Lib)
messages: 316542
nosy: eric.smith, ned.deily
priority: release blocker
severity: normal
status: open
title: dataclasses: repr of _DataclassParams objects should use repr of each member
type: behavior
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33502>
_______________________________________

From report at bugs.python.org  Mon May 14 14:07:27 2018
From: report at bugs.python.org (=?utf-8?q?St=C3=A9phane_Wirtel?=)
Date: Mon, 14 May 2018 18:07:27 +0000
Subject: [New-bugs-announce] [issue33503] use pypi.org instead of
 pypi.python.org
Message-ID: <1526321247.82.0.682650639539.issue33503@psf.upfronthosting.co.za>


Change by St?phane Wirtel <stephane at wirtel.be>:


----------
nosy: matrixise
priority: normal
severity: normal
status: open
title: use pypi.org instead of pypi.python.org
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33503>
_______________________________________

From report at bugs.python.org  Mon May 14 14:18:18 2018
From: report at bugs.python.org (John Reese)
Date: Mon, 14 May 2018 18:18:18 +0000
Subject: [New-bugs-announce] [issue33504] configparser should use dict
 instead of OrderedDict in 3.7+
Message-ID: <1526321898.65.0.682650639539.issue33504@psf.upfronthosting.co.za>


New submission from John Reese <jreese at leetcode.net>:

The configparser module uses `collections.OrderedDict` as its default, but this is no longer necessary in 3.7+ due to the semantics of core dictionaries being ordered by design.  configparser should just use `dict` by default now, instead.

----------
components: Library (Lib)
messages: 316547
nosy: jreese
priority: normal
severity: normal
status: open
title: configparser should use dict instead of OrderedDict in 3.7+
type: enhancement
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33504>
_______________________________________

From report at bugs.python.org  Mon May 14 16:45:32 2018
From: report at bugs.python.org (Jimmy Lai)
Date: Mon, 14 May 2018 20:45:32 +0000
Subject: [New-bugs-announce] [issue33505] Optimize asyncio.ensure_future by
 reordering if conditions
Message-ID: <1526330732.96.0.682650639539.issue33505@psf.upfronthosting.co.za>


New submission from Jimmy Lai <yurinai at gmail.com>:

`ensure_future` converts the input as future if it's not already a future.
The condition is the following:

if futures.isfuture(coro_or_future):
    ...
elif coroutines.iscoroutine(coro_or_future):
    ...
elif inspect.isawaitable(coro_or_future):
    ...

In real world, `ensure_future` is mostly called by `run_until_complete` and gather with async function call (coroutine) as input.
We should check `iscoroutine` first to make it most efficient.

----------
components: asyncio
messages: 316572
nosy: asvetlov, jimmylai, yselivanov
priority: normal
severity: normal
status: open
title: Optimize asyncio.ensure_future by reordering if conditions
type: performance
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33505>
_______________________________________

From report at bugs.python.org  Mon May 14 16:53:27 2018
From: report at bugs.python.org (Steve R. Hastings)
Date: Mon, 14 May 2018 20:53:27 +0000
Subject: [New-bugs-announce] [issue33506] [logging] template for filename
 timestamps
Message-ID: <1526331207.94.0.682650639539.issue33506@psf.upfronthosting.co.za>


New submission from Steve R. Hastings <steveha at users.sourceforge.net>:

When a logged file has a timed rotation, it is renamed with an archival filename that includes a timestamp.  The logging library has a small set of predefined filename templates and no way is provided to override them.

Also, the current rule is that while the file is in active use for logging, it will not have a timestamp in the filename; the filename will be something like: foo.log

Then, at file rotation time, foo.log is closed and then renamed to an archival name that includes a timestamp (something like: foo-2018-05-14.log), and a new active log file is opened with the name foo.log again.

Proposed enhancement: it should be possible to provide template codes that specify the format of the timestamp, and there should be an option that when the file is in active use its filename will include the timestamp.  (Then at rotation time, the file is simply closed, with no need to rename it; and a new file with a new timestamp is opened.)

For example, specifying a log filename of "foo-%Y%m%d-%H%M%S" would specify a filename like: foo-20180514-160000.log

Use case: the company that employs me had a logging system requiring both of the above features.  The timestamp in the filename had to be in a format different than the one format built-in to the logging module, and the timestamp needed to be in the file at all times.  The logging system included a daemon that did the equivalent of tail -f on the logfile and collected log events as they were written.

Note: I have written code that implements the features described above.

Additional note: some template formats could cause a problem with deleting the oldest backup files.

When the option backupCount is set to a number >= 1, then at rotation time the handler will check whether the number of backup files exceeds the specified count, and will delete the oldest files to reduce the number of backup files to exactly the backupCount number.

The oldest files are found by sorting the filenames; the presumption is that the archival filenames will sort with the oldest files first.  All the built-in templates produce filenames where this presumption is correct.  A foolish user could specify a timestamp format where month appears before date, or hour appears before month, or any other template that does not follow this ordering: year/month/date/hour/minutes/seconds

We could also add a feature where, when the template is specified and backupCount is specified, the handler will check that the template follows the ordering so that the oldest files do sort first, and raise an exception if the template doesn't follow the ordering.  Alternatively, we can add a note in the documentation warning of this issue.

----------
components: Library (Lib)
messages: 316573
nosy: steveha, vinay.sajip
priority: normal
severity: normal
status: open
title: [logging] template for filename timestamps
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33506>
_______________________________________

From report at bugs.python.org  Mon May 14 16:55:45 2018
From: report at bugs.python.org (=?utf-8?q?St=C3=A9phane_Blondon?=)
Date: Mon, 14 May 2018 20:55:45 +0000
Subject: [New-bugs-announce] [issue33507] Improving the html rendered by
 cgitb.html
Message-ID: <1526331345.66.0.682650639539.issue33507@psf.upfronthosting.co.za>


New submission from St?phane Blondon <stephane.blondon at gmail.com>:

cgitb.html() returns HTML code viewable in a browser.

It would be nice to improve the rendering to have something more readable (increase space between each call, increase contrast for the source code) and more modern.

You can see the difference in the attached screenshot.

----------
components: Library (Lib)
files: next.png
messages: 316575
nosy: sblondon
priority: normal
severity: normal
status: open
title: Improving the html rendered by cgitb.html
type: enhancement
versions: Python 3.8
Added file: https://bugs.python.org/file47588/next.png

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33507>
_______________________________________

From report at bugs.python.org  Mon May 14 16:58:10 2018
From: report at bugs.python.org (Steve R. Hastings)
Date: Mon, 14 May 2018 20:58:10 +0000
Subject: [New-bugs-announce] [issue33508] [logging] allow %p code to put PID
 into log filename
Message-ID: <1526331490.03.0.682650639539.issue33508@psf.upfronthosting.co.za>


New submission from Steve R. Hastings <steveha at users.sourceforge.net>:

Python logging is not multi-process safe.  When a Python program has multiple processes, one way to log safely would be to put the process ID number into the filename of the log file, giving each process its own log file.

It would be convenient if a %p format code would be expanded to the process ID number when the logging system opens the log file.

----------
components: Library (Lib)
messages: 316576
nosy: steveha, vinay.sajip
priority: normal
severity: normal
status: open
title: [logging] allow %p code to put PID into log filename
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33508>
_______________________________________

From report at bugs.python.org  Mon May 14 17:09:06 2018
From: report at bugs.python.org (Amber Brown)
Date: Mon, 14 May 2018 21:09:06 +0000
Subject: [New-bugs-announce] [issue33509] warnings.warn_explicit with
 module_globals=True raises a SystemError
Message-ID: <1526332146.43.0.682650639539.issue33509@psf.upfronthosting.co.za>


New submission from Amber Brown <hawkowl at atleastfornow.net>:

Reproducible with:

import warnings
warnings.warn_explicit("msg", DeprecationWarning, "name", 1, module_globals=None)

----------
components: Interpreter Core
messages: 316578
nosy: hawkowl
priority: normal
severity: normal
status: open
title: warnings.warn_explicit with module_globals=True raises a SystemError
type: crash
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33509>
_______________________________________

From report at bugs.python.org  Mon May 14 18:20:20 2018
From: report at bugs.python.org (Steve R. Hastings)
Date: Mon, 14 May 2018 22:20:20 +0000
Subject: [New-bugs-announce] [issue33510] [logging] add JSON log formatter
Message-ID: <1526336420.56.0.682650639539.issue33510@psf.upfronthosting.co.za>


New submission from Steve R. Hastings <steveha at users.sourceforge.net>:

Many logging aggregation and indexing systems require that log data must be written in an easy-to-parse format such as JSON.  A "batteries included" JSON log formatter would be a useful enhancement.

I have written code with the following features:

Log events are translated into JSON.  The JSON is formatted without indentation, so that one log event will produce one line in the output file.  Each line is a single JSON object.  Because log files contain many lines and JSON logging is usually not read by humans, the JSON separator punctuation are set to simply ':' and ',' (no leading or trailing spaces).

The standard fields of a log event turn into key/value pairs in the JSON object, using the standard field name as the output JSON key name.  For example, a log statement like this: log.info("foo initialized") would result in JSON that included: "message": "foo initialized", "levelname": "INFO"

By default, the JSON output has the timestamp as the first key/value pair; therefore if the file is sorted, or multiple log files are concatenated and then sorted, the logged events will be sorted into elapsed order.  By default, the timestamp format is RFC3339: https://tools.ietf.org/html/rfc3339  Also, by default, the seconds field will have a fractional part with three digits (i.e. millisecond resolution on log event timestamps).

The user can provide a dictionary that will translate key names.  If the user's log collection system requires the logging level to have a key name of "priority", then by specifying that "levelname" should be mapped to "priority" the log event will be output with "priority": "INFO"  This dictionary also provides a way to control which logging event fields are written to the logfile and in what order.

Taking advantage of a feature already present in the Python logging code: The user can provide a dictionary as the only argument after the logging message, and it will be handled specially.  For the JSON formatter, the special handling is that the key/value pairs from the dictionary will be included in the JSON output.  Alternatively, the user can provide a message string that includes % templating codes, and specify arguments to be templated in, as is common practice in logging.  (This is to be backwards-compatible with code written to the current formatters.)

The Python logging code feature mentioned above can be found in LogRecord.__init__()  The first thing that init code does is check whether there is exactly one argument and it is a non-empty Mapping.

In the case where there are multiple arguments provided and templated into the message string, one extra key/value pair is written in the JSON object: the key is "args" and the value is a JSON array with the values of the arguments.

Because logging is fundamental, if anything goes wrong it's important to log as much as possible.  If any exception is raised during the call to json.dumps() that creates an output line, there is fallback code that converts the arguments dictionary to a string and logs that string under an identifiable key name such as "args_text"  So if someone attempts to log arguments that json.dumps() can't handle, something will be written to the logfile even if it's just something like "<__main__.Foo object at 0x7f8fe7621da0>"


Here is a concrete example.  The JSON logging formatter is initialized with the following field name translation table:

TRANSLATE_FIELD_NAMES = {
    "asctime": "@timestamp",
    "levelname": "priority",
    "message": None,
    "process": "pid",
}

The above dictionary means that asctime, levelname, message, and process will be written to the output file, in that order.  The output JSON object will have key names of, respectively: "@timestamp", priority, message (no change), and pid.

Then the following logging call occurs:

 log.info("connecting to server", {"url": server_url, "port": sever_port})



The output written to the log file would be similar to this:

{"@timestamp":"2018-05-14T17:28:04.112-04:00","priority":"INFO","message":"connecting to server","pid":32073,"url":"http://127.0.0.1","port":8043}


As noted above, I have already written this code, and it has been in production for months at the company where I work.  However, the code can be simplified and cleaned up a bit, since it contains Python 2.7 backward compatibility code.

----------
components: Library (Lib)
messages: 316597
nosy: steveha, vinay.sajip
priority: normal
severity: normal
status: open
title: [logging] add JSON log formatter
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33510>
_______________________________________

From report at bugs.python.org  Mon May 14 22:34:03 2018
From: report at bugs.python.org (Eitan Adler)
Date: Tue, 15 May 2018 02:34:03 +0000
Subject: [New-bugs-announce] [issue33511] update.config.sub
Message-ID: <1526351643.4.0.682650639539.issue33511@psf.upfronthosting.co.za>


New submission from Eitan Adler <lists at eitanadler.com>:

confg.sub has been updated upstream. lets pull it in.

----------
components: Build
messages: 316612
nosy: eitan.adler
priority: normal
severity: normal
status: open
title: update.config.sub
type: enhancement

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33511>
_______________________________________

From report at bugs.python.org  Mon May 14 23:14:55 2018
From: report at bugs.python.org (Eitan Adler)
Date: Tue, 15 May 2018 03:14:55 +0000
Subject: [New-bugs-announce] [issue33512] use more standard approach for
 detecting long double in configure
Message-ID: <1526354095.19.0.682650639539.issue33512@psf.upfronthosting.co.za>


New submission from Eitan Adler <lists at eitanadler.com>:

autotools has a standard macro for defining HAVE_LONG_DOUBLE. Lets use it rather than using our own.

----------
components: Build
messages: 316615
nosy: eitan.adler
priority: normal
severity: normal
status: open
title: use more standard approach for detecting long double in configure

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33512>
_______________________________________

From report at bugs.python.org  Tue May 15 02:59:11 2018
From: report at bugs.python.org (Andrey Bychkov)
Date: Tue, 15 May 2018 06:59:11 +0000
Subject: [New-bugs-announce] [issue33513] incorrect detection of information
 of some distributions python2
Message-ID: <1526367551.54.0.682650639539.issue33513@psf.upfronthosting.co.za>


New submission from Andrey Bychkov <mrandybu at gmail.com>:

In some linux distributions, the information about the distribution is incorrectly determined when the linux_distribution() method is called from the platform class. Since the information file os-release becomes a certain standard, I propose first of all to check its availability and if it is in the system - parse the information from it.

----------
components: Library (Lib)
messages: 316628
nosy: mrandybu
priority: normal
severity: normal
status: open
title: incorrect detection of information of some distributions python2
type: behavior
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33513>
_______________________________________

From report at bugs.python.org  Tue May 15 05:27:42 2018
From: report at bugs.python.org (=?utf-8?q?Miro_Hron=C4=8Dok?=)
Date: Tue, 15 May 2018 09:27:42 +0000
Subject: [New-bugs-announce] =?utf-8?q?=5Bissue33514=5D_async_and_await_a?=
 =?utf-8?q?s_keywords_not_mentioned_in_What=E2=80=99s_New_In_Python_3=2E7?=
Message-ID: <1526376462.69.0.682650639539.issue33514@psf.upfronthosting.co.za>


New submission from Miro Hron?ok <miro at hroncok.cz>:

According to PEP 492, async and await should be full keywords in Python 3.7. That happened in https://bugs.python.org/issue30406

There is no mention of it at all at https://docs.python.org/3.7/whatsnew/3.7.html#porting-to-python-3-7 or anywhare on that page.

I consider this a blocker because it can break things:

    $ python3.6 -c 'async = 42'
    $ echo $?
    0
    $ python3.7 -c 'async = 42'
      File "<string>", line 1
        async = 42
              ^
    SyntaxError: invalid syntax

----------
assignee: docs at python
components: Documentation
messages: 316632
nosy: Jelle Zijlstra, docs at python, hroncok
priority: normal
severity: normal
status: open
title: async and await as keywords not mentioned in What?s New In Python 3.7
type: behavior
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33514>
_______________________________________

From report at bugs.python.org  Tue May 15 07:01:31 2018
From: report at bugs.python.org (Abigail)
Date: Tue, 15 May 2018 11:01:31 +0000
Subject: [New-bugs-announce] [issue33515] subprocess.Popen on a Windows
 batch file always acts as if shell=True
Message-ID: <1526382091.26.0.682650639539.issue33515@psf.upfronthosting.co.za>


New submission from Abigail <abigail.buccaneer at gmail.com>:

It's possible to invoke an application without interpreting any of its arguments as shell magic:

>>> print(subprocess.check_output(["C:/testapplication.exe", "foo", "&&", "echo", "%PROGRAMFILES%"]))
Hello world from application! 5 arguments:
Argument 0: 'C:/testapplication.exe'
Argument 1: 'foo'
Argument 2: '&&'
Argument 3: 'echo'
Argument 4: '%PROGRAMFILES%'

But not so for batch scripts:

>>> print(subprocess.check_output(["C:/testscript.bat", "foo", "&&", "echo", "%PROGRAMFILES%"]))
Hello world from script! 2 arguments:
Argument 0: 'C:/testscript.bat'
Argument 1: 'foo'
C:\Program Files

I don't know if this is a fundamental limitation of Windows' batch script processing, or of the Win32 CreateProcess API, but this looks exploitable, as it allows shell injection: the subprocess docs warn about shell injection in a big red box, and promise you'll be safe if you a list of arguments and the default shell=False.

Tested on Python 2.7.15 and Python 3.6.5.

----------
components: Library (Lib), Windows
messages: 316638
nosy: abigail, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: subprocess.Popen on a Windows batch file always acts as if shell=True
type: security
versions: Python 2.7, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33515>
_______________________________________

From report at bugs.python.org  Tue May 15 07:48:12 2018
From: report at bugs.python.org (Martijn Pieters)
Date: Tue, 15 May 2018 11:48:12 +0000
Subject: [New-bugs-announce] [issue33516] unittest.mock: Add __round__ to
 supported magicmock methods
Message-ID: <1526384892.72.0.682650639539.issue33516@psf.upfronthosting.co.za>


New submission from Martijn Pieters <mj at python.org>:

I notice that __trunc__, __floor__ and __ceil__ are supported methods for MagicMock, but __round__ (in the same grouping of numeric types emulation methods, see https://docs.python.org/3/reference/datamodel.html#object.__round__), is not.

Please add this to the mapping too.

----------
components: Library (Lib)
messages: 316641
nosy: mjpieters
priority: normal
severity: normal
status: open
title: unittest.mock:  Add __round__ to supported magicmock methods
type: enhancement

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33516>
_______________________________________

From report at bugs.python.org  Tue May 15 08:01:16 2018
From: report at bugs.python.org (Eric V. Smith)
Date: Tue, 15 May 2018 12:01:16 +0000
Subject: [New-bugs-announce] [issue33517] dataclasses: Add the field type to
 Field repr
Message-ID: <1526385676.97.0.682650639539.issue33517@psf.upfronthosting.co.za>


New submission from Eric V. Smith <eric at trueblade.com>:

With the recent (and ongoing) change to string annotations, it's no longer obvious what the type of a field is (normal, ClassVar, or InitVar).

I propose exposing the heretofore undocumented _field_type to the repr. Instead of writing it directly, I'm going to make it say one of:

_field_type=_FIELD
_field_type=_FIELD_CLASSVAR
_field_type=_FIELD_INITVAR

This is because it's an object reference, and would otherwise look like:

_field_type=<object object at 0x10cc5f1f0>

Which is all but useless.

----------
assignee: eric.smith
components: Library (Lib)
messages: 316644
nosy: eric.smith, ned.deily
priority: release blocker
severity: normal
status: open
title: dataclasses: Add the field type to Field repr
type: enhancement
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33517>
_______________________________________

From report at bugs.python.org  Tue May 15 09:02:24 2018
From: report at bugs.python.org (=?utf-8?q?Andr=C3=A9s_Delfino?=)
Date: Tue, 15 May 2018 13:02:24 +0000
Subject: [New-bugs-announce] [issue33518] Add PEP to glossary
Message-ID: <1526389344.41.0.682650639539.issue33518@psf.upfronthosting.co.za>


New submission from Andr?s Delfino <adelfino at gmail.com>:

I think PEP should be mentioned in the glossary, as it is an important piece of how Python is developed and learned.

The PR's wording is taken from PEP1, mostly verbatim.

----------
assignee: docs at python
components: Documentation
messages: 316653
nosy: adelfino, docs at python
priority: normal
severity: normal
status: open
title: Add PEP to glossary
type: enhancement
versions: Python 2.7, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33518>
_______________________________________

From report at bugs.python.org  Tue May 15 11:09:27 2018
From: report at bugs.python.org (Jelle Zijlstra)
Date: Tue, 15 May 2018 15:09:27 +0000
Subject: [New-bugs-announce] [issue33519] Should MutableSequence provide
 .copy()?
Message-ID: <1526396967.72.0.682650639539.issue33519@psf.upfronthosting.co.za>


New submission from Jelle Zijlstra <jelle.zijlstra at gmail.com>:

https://docs.python.org/3.7/library/stdtypes.html#mutable-sequence-types lists .copy() among the methods provided by mutable sequences. However, MutableSequence does not actually define .copy(): https://github.com/python/cpython/blob/master/Lib/_collections_abc.py#L945.

Should we add .copy() to the ABC or remove the promise that all mutable sequences implement .copy()?

----------
components: Library (Lib)
messages: 316664
nosy: Jelle Zijlstra, rhettinger, stutzbach
priority: normal
severity: normal
status: open
title: Should MutableSequence provide .copy()?
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33519>
_______________________________________

From report at bugs.python.org  Tue May 15 11:16:22 2018
From: report at bugs.python.org (Isaiah Peng)
Date: Tue, 15 May 2018 15:16:22 +0000
Subject: [New-bugs-announce] [issue33520] ast.Tuple has wrong col_offset
Message-ID: <1526397382.35.0.682650639539.issue33520@psf.upfronthosting.co.za>


New submission from Isaiah Peng <issaria at gmail.com>:

The `col_offset` of the ast.Tuple node is set to the column offset of the first element, shown in code:

>>> a = "{1,2,3}"
>>> b = ast.parse(a).body[0]                                                                                                                                                                  
>>> b.value.col_offset
0
>>> a = "[1,2,3]"                                                                                                                                                                             
>>> b = ast.parse(a).body[0]                                                                                                                                                                  
>>> b.value.col_offset                                                                                                                                                                        
0                                                                                                                                                                                             
>>> a = "(1,2,3)"                                                                                                                                                                             
>>> ast.parse(a).body[0].value.col_offset                                                                                                                                                     
1 
>>> a = "()"                                                                                                                                                                                  
>>> ast.parse(a).body[0].value.col_offset                                                                                                                                                     
0  

It's correct for dict, set, list, even empty tuple, Though this is not a serious bug, for other python implementations that uses the tests as language spec, this is annoying.

----------
components: Library (Lib)
messages: 316665
nosy: isaiah
priority: normal
severity: normal
status: open
title: ast.Tuple has wrong col_offset
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33520>
_______________________________________

From report at bugs.python.org  Tue May 15 11:31:32 2018
From: report at bugs.python.org (Jimmy Lai)
Date: Tue, 15 May 2018 15:31:32 +0000
Subject: [New-bugs-announce] [issue33521] Optimize asyncio.isfuture by
 providing C implementation
Message-ID: <1526398292.34.0.682650639539.issue33521@psf.upfronthosting.co.za>


New submission from Jimmy Lai <yurinai at gmail.com>:

asyncio.isfuture called whenever ensure_future is called. Providing C implementation to make it fast.

----------
components: asyncio
messages: 316670
nosy: asvetlov, jimmylai, yselivanov
priority: normal
severity: normal
status: open
title: Optimize asyncio.isfuture by providing C implementation
type: performance
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33521>
_______________________________________

From report at bugs.python.org  Tue May 15 12:15:54 2018
From: report at bugs.python.org (Steve Dower)
Date: Tue, 15 May 2018 16:15:54 +0000
Subject: [New-bugs-announce] [issue33522] Enable CI builds on Visual Studio
 Team Services
Message-ID: <1526400954.34.0.682650639539.issue33522@psf.upfronthosting.co.za>


New submission from Steve Dower <steve.dower at python.org>:

We have been provided with https://python.visualstudio.com/cpython and a measure of build machines to help us run continuous integration builds. The benefits are more reliable builds (especially Mac and Windows) and higher parallelism than our current services.

My PR adds the build definition files and fixes some test bugs to ensure we can successfully test on this platform. There are a few more manual steps to make it work against our repo, which we will do immediately after merging the changes.

----------
assignee: steve.dower
components: Build, Tests
messages: 316677
nosy: brett.cannon, steve.dower
priority: normal
pull_requests: 6539
severity: normal
status: open
title: Enable CI builds on Visual Studio Team Services
versions: Python 2.7, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33522>
_______________________________________

From report at bugs.python.org  Tue May 15 12:21:06 2018
From: report at bugs.python.org (Jason Fried)
Date: Tue, 15 May 2018 16:21:06 +0000
Subject: [New-bugs-announce] [issue33523] loop.run_until_complete
 re-entrancy to support more complicated codebases in transition to asyncio
Message-ID: <1526401266.56.0.682650639539.issue33523@psf.upfronthosting.co.za>


New submission from Jason Fried <me at jasonfried.info>:

At Facebook and Instagram we have large interconnected codebases without clear boundaries of ownership. As we move more and more services to utilize asyncio we are finding that once blocking (but fast) code paths, are now cropping up with asyncio code using run_until_complete().  Now this is fine for all the blocking callers, but some times we have async callers to that blocking code path and now it doesn't work.  

So we have two options revert the change to not use asyncio deep in the dep tree or Convert all functions in the stack to be asyncio.  Both are not possible and engineers have solved them in two crazy ways.

1. Nested Event Loops, when you hit a run_until_complete, create a new eventloop and do the async and return the result.
2. Like the first, but each library creates its own eventloop, and swaps it with the running loop for the duration of the run_until_complete, restoring the original loop when its done. 

Both of these ultimately have the same problem, everything on the primary event loop stops running until the new loop is complete. What if we could instead start servicing the existing eventloop from the run_until_complete. This would insure that tasks don't timeout.

This would allow us to convert things to asyncio faster without having to have absolute knowledge of a codebase and its call graph, and not have to have all engineers completely synchronized.

----------
components: asyncio
messages: 316678
nosy: asvetlov, fried, yselivanov
priority: normal
severity: normal
status: open
title: loop.run_until_complete re-entrancy to support more complicated codebases in transition to asyncio
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33523>
_______________________________________

From report at bugs.python.org  Tue May 15 12:35:30 2018
From: report at bugs.python.org (radical164)
Date: Tue, 15 May 2018 16:35:30 +0000
Subject: [New-bugs-announce] [issue33524] non-ascii characters in headers
 causes TypeError on email.policy.Policy.fold
Message-ID: <1526402130.36.0.682650639539.issue33524@psf.upfronthosting.co.za>


New submission from radical164 <tptlab at tuta.io>:

Python 3.6.5 has trouble on folding email messages:

Traceback (most recent call last):
  File "emailtest.py", line 7, in <module>
    policy.fold("Subject", msg["Subject"])
  File "/usr/lib/python3.6/email/policy.py", line 183, in fold
    return self._fold(name, value, refold_binary=True)
  File "/usr/lib/python3.6/email/policy.py", line 205, in _fold
    return value.fold(policy=self)
  File "/usr/lib/python3.6/email/headerregistry.py", line 258, in fold
    return header.fold(policy=policy)
  File "/usr/lib/python3.6/email/_header_value_parser.py", line 144, in fold
    return _refold_parse_tree(self, policy=policy)
  File "/usr/lib/python3.6/email/_header_value_parser.py", line 2651, in _refold_parse_tree
    part.ew_combine_allowed, charset)
  File "/usr/lib/python3.6/email/_header_value_parser.py", line 2728, in _fold_as_ew
    first_part = to_encode[:text_space]
TypeError: slice indices must be integers or None or have an __index__ method


The message has non-ascii characters in header and set max_line_length=0, regardless length of the header. Here is the code to reproduce.


from email.message import EmailMessage
from email.policy import default

policy = default.clone(max_line_length=0)
msg = EmailMessage()
msg["Subject"] = "?"
policy.fold("Subject", msg["Subject"])


I first found this issue on Maildir.add, which saves the message to a file without word wrap.

----------
components: email
messages: 316680
nosy: barry, r.david.murray, radical164
priority: normal
severity: normal
status: open
title: non-ascii characters in headers causes TypeError on email.policy.Policy.fold
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33524>
_______________________________________

From report at bugs.python.org  Tue May 15 15:14:21 2018
From: report at bugs.python.org (Mark Shannon)
Date: Tue, 15 May 2018 19:14:21 +0000
Subject: [New-bugs-announce] [issue33525] os.spawnvpe() returns error code
 127 instead of raising when env argument is invalid.
Message-ID: <1526411661.08.0.682650639539.issue33525@psf.upfronthosting.co.za>


New submission from Mark Shannon <mark at hotpy.org>:

>>> os.spawnvpe(os.P_WAIT, "python2", ["python2", "-c", "print 'Hello World!'"], {})
Hello World!
0
>>> os.spawnvpe(os.P_WAIT, "python2", ["python2", "-c", "print 'Hello World!'"], None)
127

The latter should raise an exception of some sort.


>From error found on lgtm.com

https://lgtm.com/projects/g/python/cpython/snapshot/404ccc0aa78cd896ecb025571633a68b7292e8d4/files/Lib/os.py?sort=name&dir=ASC&mode=heatmap&showExcluded=false#xf40b8a11dc3c558c:1

----------
components: Library (Lib)
messages: 316699
nosy: Mark.Shannon
priority: normal
severity: normal
stage: needs patch
status: open
title: os.spawnvpe() returns error code 127 instead of raising when env argument is invalid.
type: behavior
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33525>
_______________________________________

From report at bugs.python.org  Tue May 15 16:49:31 2018
From: report at bugs.python.org (Alexander Mohr)
Date: Tue, 15 May 2018 20:49:31 +0000
Subject: [New-bugs-announce] [issue33526] hashlib leak on import
Message-ID: <1526417371.88.0.682650639539.issue33526@psf.upfronthosting.co.za>


New submission from Alexander Mohr <thehesiod at gmail.com>:

I'm seeing a lot of leaks via valgrind against the hashlib module.  It appears that it's calling OpenSSL_add_all_digests(); on init, and never calling the corresponding EVP_Cleanup on free: https://www.openssl.org/docs/man1.1.0/crypto/OpenSSL_add_all_digests.html.  I see a ton of leaks like the following:

==27765== 24 bytes in 1 blocks are still reachable in loss record 13 of 10,294
==27765==    at 0x4C28C7B: malloc (vg_replace_malloc.c:299)
==27765==    by 0xA92E337: CRYPTO_malloc (in /usr/lib64/libcrypto.so.1.0.2k)
==27765==    by 0xA9E325A: lh_insert (in /usr/lib64/libcrypto.so.1.0.2k)
==27765==    by 0xA93103E: OBJ_NAME_add (in /usr/lib64/libcrypto.so.1.0.2k)
==27765==    by 0xA9F3559: OpenSSL_add_all_digests (in /usr/lib64/libcrypto.so.1.0.2k)
==27765==    by 0xA44CF02: PyInit__hashlib (_hashopenssl.c:998)
==27765==    by 0x506E627: _PyImport_LoadDynamicModuleWithSpec (importdl.c:154)
==27765==    by 0x506DBA7: _imp_create_dynamic_impl (import.c:2008)
==27765==    by 0x5067A2A: _imp_create_dynamic (import.c.h:289)
==27765==    by 0x4F3061A: PyCFunction_Call (methodobject.c:114)
==27765==    by 0x503E10C: do_call_core (ceval.c:5074)
==27765==    by 0x5035F30: _PyEval_EvalFrameDefault (ceval.c:3377)
==27765==    by 0x502280F: PyEval_EvalFrameEx (ceval.c:718)
==27765==    by 0x503A944: _PyEval_EvalCodeWithName (ceval.c:4139)
==27765==    by 0x503DA4D: fast_function (ceval.c:4950)
==27765==    by 0x503D3FC: call_function (ceval.c:4830)
==27765==    by 0x5035563: _PyEval_EvalFrameDefault (ceval.c:3295)
==27765==    by 0x502280F: PyEval_EvalFrameEx (ceval.c:718)
==27765==    by 0x503D70D: _PyFunction_FastCall (ceval.c:4891)
==27765==    by 0x503D922: fast_function (ceval.c:4926)
==27765==    by 0x503D3FC: call_function (ceval.c:4830)
==27765==    by 0x5035563: _PyEval_EvalFrameDefault (ceval.c:3295)
==27765==    by 0x502280F: PyEval_EvalFrameEx (ceval.c:718)
==27765==    by 0x503D70D: _PyFunction_FastCall (ceval.c:4891)
==27765==    by 0x503D922: fast_function (ceval.c:4926)
==27765==    by 0x503D3FC: call_function (ceval.c:4830)
==27765==    by 0x5035563: _PyEval_EvalFrameDefault (ceval.c:3295)
==27765==    by 0x502280F: PyEval_EvalFrameEx (ceval.c:718)
==27765==    by 0x503D70D: _PyFunction_FastCall (ceval.c:4891)
==27765==    by 0x503D922: fast_function (ceval.c:4926)
==27765==    by 0x503D3FC: call_function (ceval.c:4830)
==27765==    by 0x5035563: _PyEval_EvalFrameDefault (ceval.c:3295)
==27765==    by 0x502280F: PyEval_EvalFrameEx (ceval.c:718)
==27765==    by 0x503D70D: _PyFunction_FastCall (ceval.c:4891)
==27765==    by 0x503D922: fast_function (ceval.c:4926)

I'm not exactly sure how this is happening yet (I know the code I use does a __import__ and uses multiple threads).  It sounds like this call should be ref-counted or perhaps only done once for the life of the application.

----------
components: Extension Modules
messages: 316723
nosy: thehesiod
priority: normal
severity: normal
status: open
title: hashlib leak on import
type: resource usage
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33526>
_______________________________________

From report at bugs.python.org  Tue May 15 16:56:10 2018
From: report at bugs.python.org (gasokiw)
Date: Tue, 15 May 2018 20:56:10 +0000
Subject: [New-bugs-announce] [issue33527] Invalid child function scope
Message-ID: <1526417770.88.0.682650639539.issue33527@psf.upfronthosting.co.za>


New submission from gasokiw <gasokiw at stelliteop.info>:

When you try to reassign variable with same name as one of parameters/arguments of parent function to local scope in child function, even if it doesn't actually get ran, the argument/parameter is not passed to child function anymore. 
In practice this happens when you make decorator with arguments/parameters and make wrapper function inside it.

As workaround you can redeclare those arguments/parameters in new variables in parent function then rewrite them back in child function. ( as seen in child_function_scope_bodge.py )

Please view attachment to better understand the issue.

----------
files: example and workaround.zip
messages: 316725
nosy: gasokiw
priority: normal
severity: normal
status: open
title: Invalid child function scope
type: behavior
versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6
Added file: https://bugs.python.org/file47594/example and workaround.zip

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33527>
_______________________________________

From report at bugs.python.org  Tue May 15 18:08:43 2018
From: report at bugs.python.org (David Carlier)
Date: Tue, 15 May 2018 22:08:43 +0000
Subject: [New-bugs-announce] [issue33528] os.getentropy support
Message-ID: <1526422123.3.0.682650639539.issue33528@psf.upfronthosting.co.za>


Change by David Carlier <dcarlier at afilias.info>:


----------
components: Library (Lib)
nosy: David Carlier
priority: normal
pull_requests: 6558
severity: normal
status: open
title: os.getentropy support
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33528>
_______________________________________

From report at bugs.python.org  Tue May 15 20:12:28 2018
From: report at bugs.python.org (radical164)
Date: Wed, 16 May 2018 00:12:28 +0000
Subject: [New-bugs-announce] [issue33529] Infinite loop on folding email if
 headers has no spaces
Message-ID: <1526429548.63.0.682650639539.issue33529@psf.upfronthosting.co.za>


New submission from radical164 <tptlab at tuta.io>:

I just reported a bug about email folding at issue 33524, but this issue is more fatal in some languages like Chinese or Japanese, which does not insert spaces between each words.
Python 3.6.5 has this issue, while 3.6.4 does not.

Create an email with longer header than max_line_length set by its policy.  And the header contains non-ascii characters but no white spaces.
When try to fold it, python gets stuck and finally system hangs. There are no output unless I stop it with Ctrl-C.

^CTraceback (most recent call last):
  File "emailtest.py", line 7, in <module>
    policy.fold("Subject", msg["Subject"])
  File "/usr/lib/python3.6/email/policy.py", line 183, in fold
    return self._fold(name, value, refold_binary=True)
  File "/usr/lib/python3.6/email/policy.py", line 205, in _fold
    return value.fold(policy=self)
  File "/usr/lib/python3.6/email/headerregistry.py", line 258, in fold
    return header.fold(policy=policy)
  File "/usr/lib/python3.6/email/_header_value_parser.py", line 144, in fold
    return _refold_parse_tree(self, policy=policy)
  File "/usr/lib/python3.6/email/_header_value_parser.py", line 2651, in _refold_parse_tree
    part.ew_combine_allowed, charset)
  File "/usr/lib/python3.6/email/_header_value_parser.py", line 2735, in _fold_as_ew
    ew = _ew.encode(first_part)
  File "/usr/lib/python3.6/email/_encoded_words.py", line 215, in encode
    blen = _cte_encode_length['b'](bstring)
  File "/usr/lib/python3.6/email/_encoded_words.py", line 130, in len_b
    groups_of_3, leftover = divmod(len(bstring), 3)
KeyboardInterrupt


Code to reproduce:

from email.message import EmailMessage
from email.policy import default

policy = default # max_line_length = 78
msg = EmailMessage()
msg["Subject"] = "?"*100
policy.fold("Subject", msg["Subject"])


No problems in following cases:

1. If the header is shorter than max_line_length.
2. If the header can be split with spaces and the all chunk is shorter than max_line_length.
3. If the header is fully composed with ascii characters. In this case, there is no problem even if it is very long without spaces.

----------
components: email
messages: 316747
nosy: barry, r.david.murray, radical164
priority: normal
severity: normal
status: open
title: Infinite loop on folding email if headers has no spaces
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33529>
_______________________________________

From report at bugs.python.org  Tue May 15 22:56:28 2018
From: report at bugs.python.org (twisteroid ambassador)
Date: Wed, 16 May 2018 02:56:28 +0000
Subject: [New-bugs-announce] [issue33530] Implement Happy Eyeball in asyncio
Message-ID: <1526439388.6.0.682650639539.issue33530@psf.upfronthosting.co.za>


New submission from twisteroid ambassador <twisteroid.ambassador at gmail.com>:

Add a Happy Eyeballs implementation to asyncio, based on work in https://github.com/twisteroidambassador/async_stagger .

Current plans:

- Add 2 keyword arguments to loop.create_connection and asyncio.open_connection.

    * delay: Optional[float] = None. None disables happy eyeballs. A number >= 0 means the delay between starting new connections.

    * interleave: int = 1. Controls reordering of resolved IP addresses by address family.

- Optionally, expose the happy eyeballs scheduling helper function. 

    * It's currently called "staggered_race()". Suggestions for a better name welcome.

    * Should it belong to base_events.py, some other existing file or a new file?

----------
components: asyncio
messages: 316757
nosy: Yury.Selivanov, asvetlov, twisteroid ambassador, yselivanov
priority: normal
severity: normal
status: open
title: Implement Happy Eyeball in asyncio
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33530>
_______________________________________

From report at bugs.python.org  Wed May 16 02:29:52 2018
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 16 May 2018 06:29:52 +0000
Subject: [New-bugs-announce] [issue33531] test_asyncio: test_subprocess
 test_stdin_broken_pipe() failure on Travis CI
Message-ID: <1526452192.92.0.682650639539.issue33531@psf.upfronthosting.co.za>


New submission from STINNER Victor <vstinner at redhat.com>:

https://travis-ci.org/python/cpython/jobs/379560387

======================================================================
FAIL: test_stdin_broken_pipe (test.test_asyncio.test_subprocess.SubprocessFastWatcherTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/python/cpython/Lib/test/test_asyncio/test_subprocess.py", line 214, in test_stdin_broken_pipe
    self.loop.run_until_complete, coro)
AssertionError: (<class 'BrokenPipeError'>, <class 'ConnectionResetError'>) not raised by run_until_complete

======================================================================
FAIL: test_stdin_broken_pipe (test.test_asyncio.test_subprocess.SubprocessSafeWatcherTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/python/cpython/Lib/test/test_asyncio/test_subprocess.py", line 214, in test_stdin_broken_pipe
    self.loop.run_until_complete, coro)
AssertionError: (<class 'BrokenPipeError'>, <class 'ConnectionResetError'>) not raised by run_until_complete

It looks like bpo-30382 which has been closed because it disappears from "AMD64 FreeBSD CURRENT Non-Debug 3.x", but this bug was already random.

----------
components: Tests, asyncio
messages: 316770
nosy: asvetlov, vstinner, yselivanov
priority: normal
severity: normal
status: open
title: test_asyncio: test_subprocess test_stdin_broken_pipe() failure on Travis CI
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33531>
_______________________________________

From report at bugs.python.org  Wed May 16 02:30:24 2018
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 16 May 2018 06:30:24 +0000
Subject: [New-bugs-announce] [issue33532] test_multiprocessing_forkserver:
 TestIgnoreEINTR.test_ignore() fails on Travis CI
Message-ID: <1526452224.65.0.682650639539.issue33532@psf.upfronthosting.co.za>


New submission from STINNER Victor <vstinner at redhat.com>:

https://travis-ci.org/python/cpython/jobs/379560387

======================================================================
ERROR: test_ignore (test.test_multiprocessing_forkserver.TestIgnoreEINTR)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/python/cpython/Lib/test/_test_multiprocessing.py", line 4359, in test_ignore
    os.kill(p.pid, signal.SIGUSR1)
ProcessLookupError: [Errno 3] No such process

----------
components: Tests
messages: 316771
nosy: vstinner
priority: normal
severity: normal
status: open
title: test_multiprocessing_forkserver: TestIgnoreEINTR.test_ignore() fails on Travis CI
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33532>
_______________________________________

From report at bugs.python.org  Wed May 16 02:48:53 2018
From: report at bugs.python.org (=?utf-8?b?SHJ2b2plIE5pa8WhacSH?=)
Date: Wed, 16 May 2018 06:48:53 +0000
Subject: [New-bugs-announce] [issue33533] Provide an async-generator version
 of as_completed
Message-ID: <1526453333.0.0.682650639539.issue33533@psf.upfronthosting.co.za>


New submission from Hrvoje Nik?i? <hniksic at gmail.com>:

Judging by questions on the StackOverflow python-asyncio tag[1][2], it seems that users find it hard to understand how to use as_completed correctly. I have identified three issues:

* It's somewhat sparingly documented.

A StackOverflow user ([2]) didn't find it obvious that it runs the futures in parallel. Unless one is already aware of the meaning, the term "as completed" could suggest that they are executed and completed sequentially.

* Unlike its concurrent.futures counter-part, it's non-blocking.

This sounds like a good idea because it's usable from synchronous code, but it means that the futures it yields aren't completed, you have to await them first. This is confusing for a function with "completed" in the name, and is not the case with concurrent.futures.as_completed, nor with other waiting functions in asyncio (gather, wait, wait_for).

* It yields futures other than those that were passed in.

This prevents some usual patterns from working, e.g. associating the results with context data, such as Python docs itself uses for concurrent.futures.as_completed in https://docs.python.org/3/library/concurrent.futures.html#threadpoolexecutor-example .  See SO question [1] for a similar request in asyncio.


Here is my proposal to address the issues.

I believe the usage problems stem from as_completed predating the concept of async iterators. If we had async iterators back then, as_completed would have been an obvious candidate to be one. In that case it could be both "blocking" (but not for the event loop) and return the original futures. For example:

async def as_completed2(fs):
    pending = set(map(asyncio.ensure_future(fs)))
    while pending:
        done, pending = await asyncio.wait(pending, return_when=asyncio.FIRST_COMPLETED)
        yield from done

(It is straightforward to add support for a timeout argument.)

I propose to deprecate asyncio.as_completed and advertise the async-iterator version like the one presented here - under a nicer name, such as as_done(), or as_completed_async().



[1] https://stackoverflow.com/questions/50028465/python-get-reference-to-original-task-after-ordering-tasks-by-completion
[2] https://stackoverflow.com/questions/50355944/yield-from-async-generator-in-python-asyncio

----------
components: asyncio
messages: 316773
nosy: asvetlov, hniksic, yselivanov
priority: normal
severity: normal
status: open
title: Provide an async-generator version of as_completed
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33533>
_______________________________________

From report at bugs.python.org  Wed May 16 04:04:01 2018
From: report at bugs.python.org (Eric V. Smith)
Date: Wed, 16 May 2018 08:04:01 +0000
Subject: [New-bugs-announce] [issue33534] dataclasses: unneeded test in
 _is_classvar
Message-ID: <1526457841.32.0.682650639539.issue33534@psf.upfronthosting.co.za>


New submission from Eric V. Smith <eric at trueblade.com>:

There's an unnecessary test for "if typing" in _is_classvar. I added this function yesterday, and due to some refactoring it picked up some superfluous text. I'm going to remove it.

----------
assignee: eric.smith
components: Library (Lib)
messages: 316779
nosy: eric.smith, ned.deily
priority: release blocker
severity: normal
status: open
title: dataclasses: unneeded test in _is_classvar
type: behavior
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33534>
_______________________________________

From report at bugs.python.org  Wed May 16 04:15:36 2018
From: report at bugs.python.org (Berker Peksag)
Date: Wed, 16 May 2018 08:15:36 +0000
Subject: [New-bugs-announce] [issue33535] Consider quoting all values in
 Morsel objects
Message-ID: <1526458536.93.0.682650639539.issue33535@psf.upfronthosting.co.za>


New submission from Berker Peksag <berker.peksag at gmail.com>:

Continuing the discussion from bpo-991266. See msg315498 for Mark Williams' suggestion and https://github.com/python/cpython/pull/6555#discussion_r183176808 for another discussion between Alex and I on quoting all values unconditionally.

----------
components: Library (Lib)
messages: 316781
nosy: Mark.Williams, alex, berker.peksag
priority: normal
severity: normal
stage: test needed
status: open
title: Consider quoting all values in Morsel objects
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33535>
_______________________________________

From report at bugs.python.org  Wed May 16 08:27:58 2018
From: report at bugs.python.org (Eric V. Smith)
Date: Wed, 16 May 2018 12:27:58 +0000
Subject: [New-bugs-announce] [issue33536] dataclasses.make_dataclass does
 not validate fields for being valid identifiers
Message-ID: <1526473678.59.0.682650639539.issue33536@psf.upfronthosting.co.za>


Change by Eric V. Smith <eric at trueblade.com>:


----------
assignee: eric.smith
nosy: eric.smith
priority: high
severity: normal
status: open
title: dataclasses.make_dataclass does not validate fields for being valid identifiers
type: behavior
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33536>
_______________________________________

From report at bugs.python.org  Wed May 16 08:29:28 2018
From: report at bugs.python.org (Serhiy Storchaka)
Date: Wed, 16 May 2018 12:29:28 +0000
Subject: [New-bugs-announce] [issue33537] Help on importlib.resources
 outputs the builtin open description
Message-ID: <1526473768.61.0.682650639539.issue33537@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:

help(importlib.resources) outputs many noise. In particular it contains the description of builtin open() which is longer than descriptions of all functions defined in importlib.resources. It also contains references to typing classes in the DATA section.

There are two ways of solving this problem. First way: make all imported names underscored (for example _open instead of builtin_open). Second way: add __all__.

Actually I don't understand why buildin_open is used instead of just open. And _zipimport_get_resource_reader looks unused.

----------
components: Library (Lib)
messages: 316802
nosy: brett.cannon, eric.snow, ncoghlan, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Help on importlib.resources outputs the builtin open description
type: behavior
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33537>
_______________________________________

From report at bugs.python.org  Wed May 16 09:21:21 2018
From: report at bugs.python.org (TaoQingyun)
Date: Wed, 16 May 2018 13:21:21 +0000
Subject: [New-bugs-announce] [issue33538] Remove useless check in subprocess
Message-ID: <1526476881.16.0.682650639539.issue33538@psf.upfronthosting.co.za>


New submission from TaoQingyun <845767657 at qq.com>:

diff --git a/Lib/subprocess.py b/Lib/subprocess.py
index bafb501fcf..4a0bb33978 100644
--- a/Lib/subprocess.py
+++ b/Lib/subprocess.py
@@ -766,7 +766,7 @@ class Popen(object):
                   ResourceWarning, source=self)
         # In case the child hasn't been waited on, check if it's done.
         self._internal_poll(_deadstate=_maxsize)
-        if self.returncode is None and _active is not None:
+        if self.returncode is None:
             # Child is still running, keep us alive until we can wait on it.
             _active.append(self)
 

the `_active` is initialized with [] at the beginning of the module.

----------
components: Library (Lib)
messages: 316804
nosy: qingyunha
priority: normal
severity: normal
status: open
title: Remove useless check in subprocess
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33538>
_______________________________________

From report at bugs.python.org  Wed May 16 10:53:13 2018
From: report at bugs.python.org (Barry A. Warsaw)
Date: Wed, 16 May 2018 14:53:13 +0000
Subject: [New-bugs-announce] [issue33539] Remove `init` flag from @dataclass?
Message-ID: <1526482393.81.0.682650639539.issue33539@psf.upfronthosting.co.za>


New submission from Barry A. Warsaw <barry at python.org>:

In reading over the new dataclasses documentation, I'm unsure what the `init` flag is used for, given that:

* If you already have a __init__(), then dataclasses won't add one
* If you don't have a __init__(), why wouldn't you want dataclasses to add one?  Without that, how do your attributes get initialized?

If there is a valid use case for `init`, I'm happy to add it to the documentation.  If there isn't, can we call YAGNI and remove the flag?

I'm mildly of the same opinion with `repr` but I can see some value in wanting to defer to object.__repr__().  OTOH, it should be pretty easy to just write:

@dataclass
class Foo:
    def __repr__(self):
        return super.__repr__()

----------
assignee: eric.smith
messages: 316812
nosy: barry, eric.smith, gvanrossum
priority: normal
severity: normal
status: open
title: Remove `init` flag from @dataclass?
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33539>
_______________________________________

From report at bugs.python.org  Wed May 16 11:59:33 2018
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 16 May 2018 15:59:33 +0000
Subject: [New-bugs-announce] [issue33540] socketserver: Add an opt-in option
 to get Python 3.6 behaviour on server_close()
Message-ID: <1526486373.61.0.682650639539.issue33540@psf.upfronthosting.co.za>


New submission from STINNER Victor <vstinner at redhat.com>:

bpo-31151 changed ForkingMixIn and bpo-31233 changed ThreadingMixIn to block on server_closer() for processes/threads.

I propose to add a new opt-in option to get the Python 3.6 behaviour. The old behaviour was wrong, but I expect that some people rely on it.

----------
components: Library (Lib)
messages: 316825
nosy: vstinner
priority: normal
severity: normal
status: open
title: socketserver: Add an opt-in option to get Python 3.6 behaviour on server_close()
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33540>
_______________________________________

From report at bugs.python.org  Wed May 16 12:05:33 2018
From: report at bugs.python.org (Mario Corchero)
Date: Wed, 16 May 2018 16:05:33 +0000
Subject: [New-bugs-announce] [issue33541] Remove private and apparently
 unused __pad function
Message-ID: <1526486733.92.0.682650639539.issue33541@psf.upfronthosting.co.za>


New submission from Mario Corchero <mariocj89 at gmail.com>:

When checking on ways to improve coverage of datetime related functions I found this function that seems not to be used anyware.
It is private and mangled, should be safe to remove.

Creating the issue as requested in the PR:
https://github.com/python/cpython/pull/4377

----------
components: Library (Lib)
messages: 316827
nosy: mariocj89
priority: normal
severity: normal
status: open
title: Remove private and apparently unused __pad function
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33541>
_______________________________________

From report at bugs.python.org  Wed May 16 12:20:56 2018
From: report at bugs.python.org (Zvi Effron)
Date: Wed, 16 May 2018 16:20:56 +0000
Subject: [New-bugs-announce] [issue33542] _ipconfig_getnode incorrectly
 selects a DUID as a MAC address
Message-ID: <1526487656.13.0.682650639539.issue33542@psf.upfronthosting.co.za>


New submission from Zvi Effron <viz_skywalker at yahoo.com>:

The function _ipconfig_getnode() in the uuid module can incorrectly pick up an IPv6 DUID instead of a MAC address due to not validating an exact match for the regular expression. The following output from `ipconfig /all` on my machine can be used to illustrate the problem:
```
C:\Users\viz_s>ipconfig /all

Windows IP Configuration

   Host Name . . . . . . . . . . . . : DESKTOP-UOHALF9
   Primary Dns Suffix  . . . . . . . :
   Node Type . . . . . . . . . . . . : Hybrid
   IP Routing Enabled. . . . . . . . : No
   WINS Proxy Enabled. . . . . . . . : No

Ethernet adapter vEthernet (Default Switch):

   Connection-specific DNS Suffix  . :
   Description . . . . . . . . . . . : Hyper-V Virtual Ethernet Adapter
   Physical Address. . . . . . . . . : 02-15-0B-EF-A2-34
   DHCP Enabled. . . . . . . . . . . : Yes
   Autoconfiguration Enabled . . . . : Yes
   Link-local IPv6 Address . . . . . : fe80::d8b5:2fbd:4d82:2e95%12(Preferred)
   IPv4 Address. . . . . . . . . . . : 192.168.125.17(Preferred)
   Subnet Mask . . . . . . . . . . . : 255.255.255.240
   Default Gateway . . . . . . . . . :
   DHCPv6 IAID . . . . . . . . . . . : 452990301
   DHCPv6 Client DUID. . . . . . . . : 00-01-00-01-1F-5A-B1-27-98-5F-D3-C5-71-45
   DNS Servers . . . . . . . . . . . : fec0:0:0:ffff::1%1
                                       fec0:0:0:ffff::2%1
                                       fec0:0:0:ffff::3%1
   NetBIOS over Tcpip. . . . . . . . : Disabled

Ethernet adapter vEthernet (DockerNAT):

   Connection-specific DNS Suffix  . :
   Description . . . . . . . . . . . : Hyper-V Virtual Ethernet Adapter #2
   Physical Address. . . . . . . . . : 00-15-5D-00-0D-1C
   DHCP Enabled. . . . . . . . . . . : No
   Autoconfiguration Enabled . . . . : Yes
   Link-local IPv6 Address . . . . . : fe80::a8b0:4970:fe5c:7ad8%20(Preferred)
   IPv4 Address. . . . . . . . . . . : 10.0.75.1(Preferred)
   Subnet Mask . . . . . . . . . . . : 255.255.255.0
   Default Gateway . . . . . . . . . :
   DNS Servers . . . . . . . . . . . : fec0:0:0:ffff::1%1
                                       fec0:0:0:ffff::2%1
                                       fec0:0:0:ffff::3%1
   NetBIOS over Tcpip. . . . . . . . : Enabled

Ethernet adapter Npcap Loopback Adapter:

   Connection-specific DNS Suffix  . :
   Description . . . . . . . . . . . : Npcap Loopback Adapter
   Physical Address. . . . . . . . . : 02-00-4C-4F-4F-50
   DHCP Enabled. . . . . . . . . . . : Yes
   Autoconfiguration Enabled . . . . : Yes
   Link-local IPv6 Address . . . . . : fe80::c07f:d604:86b:87e0%26(Preferred)
   Autoconfiguration IPv4 Address. . : 169.254.135.224(Preferred)
   Subnet Mask . . . . . . . . . . . : 255.255.0.0
   Default Gateway . . . . . . . . . :
   DHCPv6 IAID . . . . . . . . . . . : 771883084
   DHCPv6 Client DUID. . . . . . . . : 00-01-00-01-1F-5A-B1-27-98-5F-D3-C5-71-45
   DNS Servers . . . . . . . . . . . : fec0:0:0:ffff::1%1
                                       fec0:0:0:ffff::2%1
                                       fec0:0:0:ffff::3%1
   NetBIOS over Tcpip. . . . . . . . : Enabled

Wireless LAN adapter Local Area Connection* 2:

   Media State . . . . . . . . . . . : Media disconnected
   Connection-specific DNS Suffix  . :
   Description . . . . . . . . . . . : Microsoft Wi-Fi Direct Virtual Adapter
   Physical Address. . . . . . . . . : 9A-5F-D3-C5-70-44
   DHCP Enabled. . . . . . . . . . . : Yes
   Autoconfiguration Enabled . . . . : Yes

Wireless LAN adapter Local Area Connection* 12:

   Media State . . . . . . . . . . . : Media disconnected
   Connection-specific DNS Suffix  . :
   Description . . . . . . . . . . . : Microsoft Wi-Fi Direct Virtual Adapter #2
   Physical Address. . . . . . . . . : 9A-5F-D3-C5-75-44
   DHCP Enabled. . . . . . . . . . . : Yes
   Autoconfiguration Enabled . . . . : Yes

Wireless LAN adapter Wi-Fi:

   Connection-specific DNS Suffix  . :
   Description . . . . . . . . . . . : Marvell AVASTAR Wireless-AC Network Controller
   Physical Address. . . . . . . . . : 98-5F-D3-C5-71-45
   DHCP Enabled. . . . . . . . . . . : Yes
   Autoconfiguration Enabled . . . . : Yes
   Link-local IPv6 Address . . . . . : fe80::f183:815e:4914:dfa%21(Preferred)
   IPv4 Address. . . . . . . . . . . : 100.69.161.39(Preferred)
   Subnet Mask . . . . . . . . . . . : 255.255.0.0
   Lease Obtained. . . . . . . . . . : Wednesday, May 16, 2018 8:51:32 AM
   Lease Expires . . . . . . . . . . : Wednesday, May 16, 2018 9:51:31 AM
   Default Gateway . . . . . . . . . : 100.69.0.5
   DHCP Server . . . . . . . . . . . : 100.69.0.6
   DHCPv6 IAID . . . . . . . . . . . : 144203731
   DHCPv6 Client DUID. . . . . . . . : 00-01-00-01-1F-5A-B1-27-98-5F-D3-C5-71-45
   DNS Servers . . . . . . . . . . . : 100.69.0.5
   NetBIOS over Tcpip. . . . . . . . : Enabled
```

----------
components: Library (Lib), Windows
messages: 316831
nosy: paul.moore, steve.dower, tim.golden, zach.ware, zeffron
priority: normal
severity: normal
status: open
title: _ipconfig_getnode incorrectly selects a DUID as a MAC address
type: behavior
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33542>
_______________________________________

From report at bugs.python.org  Wed May 16 12:42:41 2018
From: report at bugs.python.org (Barry A. Warsaw)
Date: Wed, 16 May 2018 16:42:41 +0000
Subject: [New-bugs-announce] [issue33543] `make html` broken
Message-ID: <1526488961.65.0.682650639539.issue33543@psf.upfronthosting.co.za>


New submission from Barry A. Warsaw <barry at python.org>:

Building the documentation in the master (3.8) branch is currently broken:

% make html
make html
mkdir -p build
Building NEWS from Misc/NEWS.d with blurb
PATH=./venv/bin:$PATH sphinx-build -b html -d build/doctrees -D latex_elements.papersize=  . build/html 
Running Sphinx v1.7.4
loading pickled environment... not yet created

Theme error:
no theme named 'python_docs_theme' found (missing theme.conf?)
make: *** [build] Error 2

----------
assignee: docs at python
components: Documentation
messages: 316838
nosy: barry, docs at python
priority: normal
severity: normal
status: open
title: `make html` broken
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33543>
_______________________________________

From report at bugs.python.org  Wed May 16 12:57:54 2018
From: report at bugs.python.org (Jason Fried)
Date: Wed, 16 May 2018 16:57:54 +0000
Subject: [New-bugs-announce] [issue33544] Asyncio Event.wait() and
 Condition.wait() is a hold over from before awaitable,
 and should be awaitable
Message-ID: <1526489874.13.0.682650639539.issue33544@psf.upfronthosting.co.za>


New submission from Jason Fried <me at jasonfried.info>:

wait is a very overloaded word in asyncio.  Events and Conditions are not consistent with the rest of asyncio.  

Why don't Future and Task have wait() methods? well because they are awaitable

Some subjective reasoning:
Every time I go to use one of these things, I attempt to await them like everything else in the Asyncio world and get a nice exception for it.

   await event

vs

   await event.wait()

I propose we make conditions and events awaitable and deprecate the .wait or at-least remove it from the documentation.

----------
components: asyncio
messages: 316842
nosy: asvetlov, fried, lukasz.langa, yselivanov
priority: normal
severity: normal
status: open
title: Asyncio Event.wait() and Condition.wait() is a hold over from before awaitable, and should be awaitable
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33544>
_______________________________________

From report at bugs.python.org  Wed May 16 13:32:45 2018
From: report at bugs.python.org (merelymoray)
Date: Wed, 16 May 2018 17:32:45 +0000
Subject: [New-bugs-announce] [issue33545] Docs for uuid don't mention that
 uuid1 can repeat in some circumstances
Message-ID: <1526491965.86.0.682650639539.issue33545@psf.upfronthosting.co.za>


New submission from merelymoray <arcadesrofocale at yahoo.co.uk>:

https://docs.python.org/3.6/library/uuid.html#uuid.UUID

uuid.uuid1(node=None, clock_seq=None)

Generate a UUID from a host ID, sequence number, and the current time. If node is not given, getnode() is used to obtain the hardware address. If clock_seq is given, it is used as the sequence number; otherwise a random 14-bit sequence number is chosen.


^^

This neglects to mention that if you call uuid1() in quick succession with no parameters, you can get a repeat uuid.

I had a bug because of this.

It would have been nice if the docs mentioned the fact directly. 

It's only a sentence to add, and would have saved a lot of time.

----------
assignee: docs at python
components: Documentation
messages: 316844
nosy: docs at python, merelymoray
priority: normal
severity: normal
status: open
title: Docs for uuid don't mention that uuid1 can repeat in some circumstances
type: enhancement
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33545>
_______________________________________

From report at bugs.python.org  Wed May 16 14:25:47 2018
From: report at bugs.python.org (Jason Fried)
Date: Wed, 16 May 2018 18:25:47 +0000
Subject: [New-bugs-announce] [issue33546] asyncio.Condition should become
 awaitable in 3.9
Message-ID: <1526495147.79.0.682650639539.issue33546@psf.upfronthosting.co.za>


New submission from Jason Fried <me at jasonfried.info>:

In 3.9 we can remove the deprecated pattern for accepting __enter__ and __exit__ for locks.  This will free up __await__ for Condition to use for replacing .wait() which is wart from before awaitables. 

My new proposed behavior is

await cond 

which would be equivalent of:

async with cond:
    await cond.wait()

----------
components: asyncio
messages: 316848
nosy: asvetlov, fried, lukasz.langa, yselivanov
priority: normal
severity: normal
status: open
title: asyncio.Condition should become awaitable in 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33546>
_______________________________________

From report at bugs.python.org  Wed May 16 16:46:59 2018
From: report at bugs.python.org (Rolf Campbell)
Date: Wed, 16 May 2018 20:46:59 +0000
Subject: [New-bugs-announce] [issue33547] Relative imports do not replace
 local variables
Message-ID: <1526503619.08.0.682650639539.issue33547@psf.upfronthosting.co.za>


New submission from Rolf Campbell <thats.unpossible at gmail.com>:

Relative imports do not replace local variables, but also don't fail.  This can cause some very strange outcomes like this simple example:

touch a.py; python3.6 -c 'a=7; b=5; from . import a as b; print(a,b)'

I would expect this to produce "7 <module ...>", but instead, it produces "7 7".

Tested in v3.6.2 and v3.6.5:
Python 3.6.5 (default, Mar 29 2018, 18:20:46) 
[GCC 8.0.1 20180317 (Red Hat 8.0.1-0.19)] on linux

----------
components: Interpreter Core
messages: 316864
nosy: Rolf Campbell
priority: normal
severity: normal
status: open
title: Relative imports do not replace local variables
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33547>
_______________________________________

From report at bugs.python.org  Wed May 16 17:05:19 2018
From: report at bugs.python.org (Steve Dower)
Date: Wed, 16 May 2018 21:05:19 +0000
Subject: [New-bugs-announce] [issue33548] tempfile._candidate_tempdir_list
 should consider common TEMP locations
Message-ID: <1526504719.45.0.682650639539.issue33548@psf.upfronthosting.co.za>


New submission from Steve Dower <steve.dower at python.org>:

In Lib/tempfile.py, we currently look at some non-standard Windows-specific locations to find potential TEMP directories, but we should also look in the more common user and system ones.

----------
assignee: steve.dower
components: Windows
messages: 316868
nosy: paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
pull_requests: 6590
severity: normal
stage: patch review
status: open
title: tempfile._candidate_tempdir_list should consider common TEMP locations
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33548>
_______________________________________

From report at bugs.python.org  Wed May 16 17:33:46 2018
From: report at bugs.python.org (Matthias Bussonnier)
Date: Wed, 16 May 2018 21:33:46 +0000
Subject: [New-bugs-announce] [issue33549] xmlbuilder's
 `_AsyncDeprecatedProperty` make no sens now that async is a keyword.
Message-ID: <1526506426.18.0.682650639539.issue33549@psf.upfronthosting.co.za>


New submission from Matthias Bussonnier <bussonniermatthias at gmail.com>:

`Lib/xml/dom/xmlbuilder's DocumentLS try to catch the use of `obj.asyc` and warn that it's now `obj.async_`. Though now that async is a proper keyword `obj.async` is anyway SyntaxError, so the warning will likely almost never be displayed. I guess one could still use getattr/setattr, but I doubt this is the most common use case this was meant to prevent.

----------
messages: 316871
nosy: mbussonn
priority: normal
severity: normal
status: open
title: xmlbuilder's `_AsyncDeprecatedProperty` make no sens now that async is a keyword.

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33549>
_______________________________________

From report at bugs.python.org  Wed May 16 19:55:52 2018
From: report at bugs.python.org (Alfred Perlstein)
Date: Wed, 16 May 2018 23:55:52 +0000
Subject: [New-bugs-announce] [issue33550] Sigpipe handling issue should be
 documented
Message-ID: <1526514952.03.0.682650639539.issue33550@psf.upfronthosting.co.za>


New submission from Alfred Perlstein <alfred at freebsd.org>:

A common anti-pattern in python used to get rid of "ugly" brokenpipe messages is to set the SIGPIPE handler to SIG_DFL, this however will cause your program to seemingly randomly exit if it makes any socket connections during its lifetime.  (see github PR for more info)

----------
assignee: docs at python
components: Documentation
messages: 316877
nosy: docs at python, splbio
priority: normal
pull_requests: 6597
severity: normal
status: open
title: Sigpipe handling issue should be documented
type: behavior
versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33550>
_______________________________________

From report at bugs.python.org  Thu May 17 02:25:08 2018
From: report at bugs.python.org (Aifu LIU)
Date: Thu, 17 May 2018 06:25:08 +0000
Subject: [New-bugs-announce] [issue33551] The string prefixes u and f can't
 used together
Message-ID: <1526538308.93.0.682650639539.issue33551@psf.upfronthosting.co.za>


New submission from Aifu LIU <laf163 at gmail.com>:

The string prefixes u and f can't used together, for example:

>>> age = 30
>>> s = uf'???{age}'
  File "<stdin>", line 1
    s = uf'???{age}'
                   ^
SyntaxError: invalid syntax
>>>

----------
components: Unicode
messages: 316883
nosy: Aifu LIU, ezio.melotti, vstinner
priority: normal
severity: normal
status: open
title: The string prefixes u and f can't used together
type: crash
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33551>
_______________________________________

From report at bugs.python.org  Thu May 17 03:03:06 2018
From: report at bugs.python.org (Serhiy Storchaka)
Date: Thu, 17 May 2018 07:03:06 +0000
Subject: [New-bugs-announce] [issue33552] f-strings and string annotations
Message-ID: <1526540586.49.0.682650639539.issue33552@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:

There are problems with converting annotations containing f-strings into strings (from __future__ import annotations). For example f'''{"'"}''' is converted to f'{"\'"}', but the latter is a syntax error.

>>> from __future__ import annotations
>>> x: f'''{"'"}'''
>>> print(__annotations__['x'])
f'{"\'"}'
>>> x: f'{"\'"}'
  File "<stdin>", line 1
SyntaxError: f-string expression part cannot include a backslash

More complex examples:

f"""{"'''"}"""
f"""{"'" '"'}"""
f"""{"'''"}""" f'''{'"""'}'''

----------
components: Interpreter Core
messages: 316886
nosy: eric.smith, lukasz.langa, serhiy.storchaka
priority: normal
severity: normal
status: open
title: f-strings and string annotations
type: behavior
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33552>
_______________________________________

From report at bugs.python.org  Thu May 17 03:15:18 2018
From: report at bugs.python.org (Derek Kim)
Date: Thu, 17 May 2018 07:15:18 +0000
Subject: [New-bugs-announce] [issue33553] possible documentation improvement
 proposal for multiprocessing
Message-ID: <1526541318.79.0.682650639539.issue33553@psf.upfronthosting.co.za>


New submission from Derek Kim <bluewhale8202 at gmail.com>:

>>> from multiprocessing import Pool
>>> p = Pool(5)
>>> def f(x):
...     return x*x
...
>>> p.map(f, [1,2,3])

This example in the document is confusing because this is already wrong code when run with fork method even if you run it as a script. With fork start method, f should be defined before p is created.

Also, just advising that one shouldn't use interactive shell and should run multiprocessing code in if __name__ == '__main__' is not informative becuase those conditions are generally no problem with fork method.

Can I try improving the documentation?

----------
assignee: docs at python
components: Documentation
messages: 316892
nosy: Derek Kim, docs at python
priority: normal
severity: normal
status: open
title: possible documentation improvement proposal for multiprocessing
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33553>
_______________________________________

From report at bugs.python.org  Thu May 17 06:05:44 2018
From: report at bugs.python.org (b@n)
Date: Thu, 17 May 2018 10:05:44 +0000
Subject: [New-bugs-announce] [issue33554] Optimize PyDictObject
Message-ID: <1526551544.61.0.682650639539.issue33554@psf.upfronthosting.co.za>


New submission from b at n <ban1314 at gmail.com>:

make_keys_shared reusing oldkeys memory

----------
components: Interpreter Core
messages: 316901
nosy: b at n
priority: normal
severity: normal
status: open
title: Optimize PyDictObject
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33554>
_______________________________________

From report at bugs.python.org  Thu May 17 07:33:26 2018
From: report at bugs.python.org (FHTMitchell)
Date: Thu, 17 May 2018 11:33:26 +0000
Subject: [New-bugs-announce] [issue33555] No SyntaxError raised for `return`
 with argument inside generator
Message-ID: <1526556806.68.0.682650639539.issue33555@psf.upfronthosting.co.za>


New submission from FHTMitchell <fergus.htm at gmail.com>:

In python 2.7 if you run the following code you get an error (as you would expect)

Python 2.7.14 | packaged by conda-forge | (default, Dec 25 2017, 01:17:32) [MSC v.1500 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.

>>> def f():
...     yield 1
...     return 2
...
  File "<stdin>", line 3
SyntaxError: 'return' with argument inside generator

However, in python 3.6 the error is silently ignored

Python 3.6.4 | packaged by conda-forge | (default, Dec 24 2017, 10:11:43) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> def f():
...     yield 1
...     return 2
...
>>> for i in f():
...     print(i)
...
1

and still is in 3.7

Python 3.7.0b2 (v3.7.0b2:b0ef5c979b, Feb 28 2018, 02:24:20) [MSC v.1912 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> def f():
...     yield 1
...     return 2
...
>>> for i in f():
...     print(i)
...
1

This is a source of confusion
 
https://stackoverflow.com/questions/47831240/why-is-no-value-returned-from-my-generator/
 
especially since the PEP says it is disallowed:

 https://www.python.org/dev/peps/pep-0255/#then-why-not-allow-an-expression-on-return-too

----------
components: Interpreter Core
messages: 316912
nosy: FHTMitchell
priority: normal
severity: normal
status: open
title: No SyntaxError raised for `return` with argument inside generator
type: behavior
versions: Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33555>
_______________________________________

From report at bugs.python.org  Thu May 17 09:53:10 2018
From: report at bugs.python.org (Skip Montanaro)
Date: Thu, 17 May 2018 13:53:10 +0000
Subject: [New-bugs-announce] [issue33556] leftover thread crumb in
 threading.ident docstring
Message-ID: <1526565190.58.0.682650639539.issue33556@psf.upfronthosting.co.za>


New submission from Skip Montanaro <skip.montanaro at gmail.com>:

There is a leftover reference to "thread.get_ident" in the docstring for threading.ident. I believe it needs a leading underscore.

Hopefully a PR isn't required for this. I'm not equipped to generate one at the moment, and this seems like a pretty trivial/obvious fix.

----------
components: Library (Lib)
keywords: easy
messages: 316931
nosy: skip.montanaro
priority: normal
severity: normal
stage: needs patch
status: open
title: leftover thread crumb in threading.ident docstring
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33556>
_______________________________________

From report at bugs.python.org  Thu May 17 11:38:07 2018
From: report at bugs.python.org (Jakub Wilk)
Date: Thu, 17 May 2018 15:38:07 +0000
Subject: [New-bugs-announce] [issue33557] Windows multiprocessing doesn't
 propagate tabcheck to children
Message-ID: <1526571487.53.0.682650639539.issue33557@psf.upfronthosting.co.za>


New submission from Jakub Wilk <jwilk at jwilk.net>:

Multiprocessing on Windows is supposed to start child processes using
the same sys.flags as the current process (see issue 12098).
However, at least sys.flags.tabcheck is not propagated.

I've attached small test program that reproduces this bug.
On Linux, as expected, it fails with TabError if you use -tt:

  $ python -tt test-tabcheck.py 
  Process Process-1:
  Traceback (most recent call last):
    File "/usr/lib/python2.7/multiprocessing/process.py", line 267, in _bootstrap
      self.run()
    File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
      self._target(*self._args, **self._kwargs)
    File "test-tabcheck.py", line 6, in test_ast_parse
      ast.parse('if 1:\n\t1\n' + '\x20' * 8 +'1')
    File "/usr/lib/python2.7/ast.py", line 37, in parse
      return compile(source, filename, mode, PyCF_ONLY_AST)
    File "<unknown>", line 3
      1
      ^
  TabError: inconsistent use of tabs and spaces in indentation

But it doesn't on Windows:

  >python -tt test-tabcheck.py
  Process Process-1:
  Traceback (most recent call last):
    File "C:\Python27\lib\multiprocessing\process.py", line 267, in _bootstrap
      self.run()
    File "C:\Python27\lib\multiprocessing\process.py", line 114, in run
      self._target(*self._args, **self._kwargs)
    File "...\test-tabcheck.py", line 7, in test_ast_parse
      assert sys.flags.tabcheck == expected_tabcheck, 'sys.flags.tabcheck == {0} != {1}'.format(sys.flags.tabcheck, expected_tabcheck)
  AssertionError: sys.flags.tabcheck == 0 != 2

This was tested with Python 2.7.15.

----------
components: Library (Lib), Windows
files: test-tabcheck.py
messages: 316949
nosy: jwilk, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Windows multiprocessing doesn't propagate tabcheck to children
versions: Python 2.7
Added file: https://bugs.python.org/file47598/test-tabcheck.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33557>
_______________________________________

From report at bugs.python.org  Thu May 17 11:53:36 2018
From: report at bugs.python.org (Filip)
Date: Thu, 17 May 2018 15:53:36 +0000
Subject: [New-bugs-announce] [issue33558] Python has no icon in taskbar and
 in start screen
Message-ID: <1526572416.53.0.682650639539.issue33558@psf.upfronthosting.co.za>


New submission from Filip <knazko.filip at gmail.com>:

I've just downloaded the latest version of Python (3.6.5) from python.org. After installing, I launched it and I found out the app has no icon. Then I tried to download Python using different link, but it was the same as the first try.

----------
files: bug.png
messages: 316952
nosy: e_l_e_c_t_r_i_f_y
priority: normal
severity: normal
status: open
title: Python has no icon in taskbar and in start screen
type: performance
versions: Python 3.5
Added file: https://bugs.python.org/file47599/bug.png

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33558>
_______________________________________

From report at bugs.python.org  Thu May 17 12:04:58 2018
From: report at bugs.python.org (=?utf-8?q?Miro_Hron=C4=8Dok?=)
Date: Thu, 17 May 2018 16:04:58 +0000
Subject: [New-bugs-announce] [issue33559] Exception's repr change not
 documented
Message-ID: <1526573098.08.0.682650639539.issue33559@psf.upfronthosting.co.za>


New submission from Miro Hron?ok <miro at hroncok.cz>:

Python 3.6.5 ...
>>> Exception('foo',)
Exception('foo',)

Python 3.7.0b4 ...
>>> Exception('foo',)
Exception('foo')

This is a change that might bit people who rely on doctesting.

It is not documented at https://docs.python.org/3.7/whatsnew/3.7.html

I'll send a PR.

----------
assignee: docs at python
components: Documentation
messages: 316953
nosy: Elvis.Pranskevichus, docs at python, hroncok, ned.deily, yselivanov
priority: normal
severity: normal
status: open
title: Exception's repr change not documented
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33559>
_______________________________________

From report at bugs.python.org  Thu May 17 12:20:20 2018
From: report at bugs.python.org (Cyril Roelandt)
Date: Thu, 17 May 2018 16:20:20 +0000
Subject: [New-bugs-announce] [issue33560] tuple.index() could return a more
 explicit error message
Message-ID: <1526574020.28.0.682650639539.issue33560@psf.upfronthosting.co.za>


New submission from Cyril Roelandt <croeland at redhat.com>:

The tuple.index() method returns an error message that does not allow users to know what element was being looked for inside the tuple:

>>> ().index(1)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: tuple.index(x): x not in tuple

The list.index() method has a much better error message:

>>> [].index(1)   
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: 1 is not in list

We could improve tuple.index() so that its behaviour becomes similar to that of list.index().

----------
messages: 316955
nosy: Cyril Roelandt
priority: normal
severity: normal
status: open
title: tuple.index() could return a more explicit error message
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33560>
_______________________________________

From report at bugs.python.org  Thu May 17 13:13:45 2018
From: report at bugs.python.org (Steven Vascellaro)
Date: Thu, 17 May 2018 17:13:45 +0000
Subject: [New-bugs-announce] [issue33561] Add .tostring() method to
 xml.etree.ElementTree.Element
Message-ID: <1526577225.21.0.682650639539.issue33561@psf.upfronthosting.co.za>


New submission from Steven Vascellaro <Stevoisiak at gmail.com>:

In Python 3.6, converting an xml `xml.etree.ElementTree.Element` to a string is done using `xml.etree.ElementTree.tostring()`.

```
from xml.etree import ElementTree

xml = ElementTree.Element('Person', Name='John')
print(ElementTree.tostring(xml, encoding='unicode', method='xml')
# Output: <Person Name="John" />
```

I would like to propose adding a `tostring()` function to the `Element` class, so that `ElementTree.tostring(xml)` could be replaced with the more intuitive `xml.tostring()`.

```
from xml.etree import ElementTree

xml = ElementTree.Element('Person', Name='John')
print(xml.tostring(encoding='unicode', method='xml'))
# Output: <Person Name="John" />
```

Benefits:

- Doesn't require importing `xml.etree.ElementTree`
- Allows writing more concise code
- Makes `tostring` part of the `Element` class
- Maintains backwards compatibility

----------
components: XML
messages: 316966
nosy: Stevoisiak
priority: normal
severity: normal
status: open
title: Add .tostring() method to xml.etree.ElementTree.Element
type: enhancement
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33561>
_______________________________________

From report at bugs.python.org  Thu May 17 16:49:30 2018
From: report at bugs.python.org (Brett Cannon)
Date: Thu, 17 May 2018 20:49:30 +0000
Subject: [New-bugs-announce] [issue33562] Check that the global settings for
 asyncio are not changed by tests
Message-ID: <1526590170.07.0.682650639539.issue33562@psf.upfronthosting.co.za>


New submission from Brett Cannon <brett at python.org>:

There were not checks in test.libregrtest.save_env to make sure tests were not mucking with the default event loop. The settings to check seem to be:

'asyncio.get_event_loop_policy', 'asyncio.get_event_loop',
'asyncio_get_exception_handler', 'asyncio_get_debug',
'asyncio_get_child_watcher',

I have a PR that I'm working on that checks these things and it looks like the following tests are being a little sloppy:

test_asyncgen
test_contextlib_async
test_coroutines
test_socket
test_sys_settrace

And test_asyncio never seems to complete.

----------
components: Tests
messages: 316985
nosy: asvetlov, brett.cannon, giampaolo.rodola, yselivanov
priority: normal
severity: normal
status: open
title: Check that the global settings for asyncio are not changed by tests
type: behavior
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33562>
_______________________________________

From report at bugs.python.org  Thu May 17 17:45:21 2018
From: report at bugs.python.org (Matthias Bussonnier)
Date: Thu, 17 May 2018 21:45:21 +0000
Subject: [New-bugs-announce] [issue33563] fileinput input's and Fileinput's
 bufsize=0 do not remit deprecationWarnings
Message-ID: <1526593521.99.0.682650639539.issue33563@psf.upfronthosting.co.za>


New submission from Matthias Bussonnier <bussonniermatthias at gmail.com>:

In Lib/fileinput.py, passing bufsize=0 does not raise any DeprecationWarning.

Also when using `input` the stacklevel could be better as it passes bufsize directly for FileInput.

in Lib/test/test_fileinput.py ; MockFileInput does not reflect the changes of FileInput. Presumably MockFielInput was made to make sure that `input` was passing the parameters as is and should be update accordingly.

----------
messages: 316988
nosy: mbussonn
priority: normal
severity: normal
status: open
title: fileinput input's and Fileinput's bufsize=0 do not remit deprecationWarnings

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33563>
_______________________________________

From report at bugs.python.org  Thu May 17 19:34:19 2018
From: report at bugs.python.org (Terry J. Reedy)
Date: Thu, 17 May 2018 23:34:19 +0000
Subject: [New-bugs-announce] [issue33564] IDLE: add 'async' to codecontext
 block openers
Message-ID: <1526600059.04.0.682650639539.issue33564@psf.upfronthosting.co.za>


New submission from Terry J. Reedy <tjreedy at udel.edu>:

I am just going to add 'async' to BLOCKOPENERS.  I don't think this needs a particular test, as most of the openers are not used in the new tests added by #32831.  We could eventually expand the sample to have one of each opener.

----------
assignee: terry.reedy
components: IDLE
messages: 316991
nosy: terry.reedy
priority: normal
severity: normal
stage: needs patch
status: open
title: IDLE: add 'async' to codecontext block openers
type: enhancement
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33564>
_______________________________________

From report at bugs.python.org  Thu May 17 22:40:46 2018
From: report at bugs.python.org (Alexander Mohr)
Date: Fri, 18 May 2018 02:40:46 +0000
Subject: [New-bugs-announce] [issue33565] strange tracemalloc results
Message-ID: <1526611246.92.0.682650639539.issue33565@psf.upfronthosting.co.za>


New submission from Alexander Mohr <thehesiod at gmail.com>:

while investigating https://github.com/boto/botocore/issues/1464 I used tracemalloc (like I've done before in 3.5.2) to try to figure out where the leak was.  To my surprise tracemalloc listed stacks that didn't make any sense.


Strangest example is the top result when running the attached script against python 3.6.5 in the following manner: PYTHONMALLOC=malloc /valgrind/bin/python3 /tmp/test.py head_object

The top hit is listed as:

21 memory blocks: 4.7 KiB
  File "/tmp/test.py", line 28
    raise
  File "/tmp/test.py", line 47
    test(s3_client)
  File "/tmp/test.py", line 65
    main()

how is it that the "raise" is a leak?  It doesn't make any sense to me specially given that no contexts are used in that call.  Further that line is never hit because the exception is not re-thrown.


Further a bunch of regular expression allocs don't make any sense either given that I've cleared the cache before doing snapshots.

if someone could shed some light on why this is/isn't a bug that would be great.  It seems to me that the callstacks are not related at all to the leak.

----------
components: Library (Lib)
files: tracemalloc_test.py
messages: 317002
nosy: thehesiod
priority: normal
severity: normal
status: open
title: strange tracemalloc results
versions: Python 3.6
Added file: https://bugs.python.org/file47600/tracemalloc_test.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33565>
_______________________________________

From report at bugs.python.org  Fri May 18 04:06:05 2018
From: report at bugs.python.org (Min)
Date: Fri, 18 May 2018 08:06:05 +0000
Subject: [New-bugs-announce] [issue33566] re.findall() dead locked whent the
 expected ending char not occur until end of string
Message-ID: <1526630765.64.0.682650639539.issue33566@psf.upfronthosting.co.za>


New submission from Min <mamamiaibm at gmail.com>:

Firstly, I wrote something like this:

patn = r"\bROW\s*\((\d+|\*)\)(.|\s)*?\)"
    newlines = re.sub(patn, "\nYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY\n", newlines)
but if the file(or string) ended without the expected ")" the code deadlock there, no progress, no exception, and no exit.

Then I changed it to :
 patn = r"\bROW\s*\((\d+|\*)\)(.|\s)*?(\)|$)"
    newlines = re.sub(patn, "\nYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY\n", newlines) to enforce the rule of  end of file. then everything ok.

I felt this is a but, coz RE should not die, it should exit if can't match.

it is Py3.5 on ubuntu. Thanks!

----------
messages: 317013
nosy: mamamiaibm
priority: normal
severity: normal
status: open
title: re.findall() dead locked whent the expected ending char not occur until end of string
type: behavior
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33566>
_______________________________________

From report at bugs.python.org  Fri May 18 06:41:41 2018
From: report at bugs.python.org (Martijn Pieters)
Date: Fri, 18 May 2018 10:41:41 +0000
Subject: [New-bugs-announce] [issue33567] Explicitly mention bytes and other
 buffers in the documentation for float()
Message-ID: <1526640101.14.0.682650639539.issue33567@psf.upfronthosting.co.za>


New submission from Martijn Pieters <mj at python.org>:

float(bytesobject) treats the contents of the bytesobject as a sequence of ASCII characters, and converts those to a float value as if you used float(bytesobject.decode('ASCII')). The same support is extended to other objects implementing the buffer protocol.

The documentation, however, doesn't mention this:

> Return a floating point number constructed from a number or string x.

Everywhere else in the functions documentation, "string" refers to an object of type `str`. Please make it explicit that `bytes` is also acceptedable, like it does for the int() documentation.

----------
messages: 317022
nosy: mjpieters
priority: normal
severity: normal
status: open
title: Explicitly mention bytes and other buffers in the documentation for float()

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33567>
_______________________________________

From report at bugs.python.org  Fri May 18 08:12:02 2018
From: report at bugs.python.org (Licht Takeuchi)
Date: Fri, 18 May 2018 12:12:02 +0000
Subject: [New-bugs-announce] [issue33568] Inconsistent behavior of non-ascii
 handling in EmailPolicy.fold
Message-ID: <1526645522.46.0.682650639539.issue33568@psf.upfronthosting.co.za>


New submission from Licht Takeuchi <licht-t at outlook.jp>:

policy.utf8 is False, but non-ascii are not well-handled.
Repro. code.

```
Python 3.6.4 |Anaconda, Inc.| (default, Jan 16 2018, 18:10:19) 
[GCC 7.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from email.message import EmailMessage
>>> from email.policy import default
>>> 
>>> policy = default.clone()
>>> policy.utf8
False
>>> msg = EmailMessage()
>>> msg["Subject"] = "?"
>>> policy.fold("Subject", msg["Subject"])
'Subject: =?utf-8?q?=C3=A1?=\n'
>>> policy.fold("Subject", '?')
'Subject: ?\n'
```

----------
components: Library (Lib)
messages: 317025
nosy: licht-t
priority: normal
severity: normal
status: open
title: Inconsistent behavior of non-ascii handling in EmailPolicy.fold
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33568>
_______________________________________

From report at bugs.python.org  Fri May 18 08:39:55 2018
From: report at bugs.python.org (reinhrst)
Date: Fri, 18 May 2018 12:39:55 +0000
Subject: [New-bugs-announce] [issue33569] dataclasses InitVar does not
 maintain any type info
Message-ID: <1526647195.4.0.682650639539.issue33569@psf.upfronthosting.co.za>


New submission from reinhrst <pythonbugs at claude.nl>:

Right now dataclasses.InitVar[something] is dataclasses.InitVar. This means that any type-information is removed, and it will (for instance) be impossible to do (runtime) type info checks on the generated __init__, or find out anything about the type of the variable.

----------
messages: 317026
nosy: reinhrst
priority: normal
severity: normal
status: open
title: dataclasses InitVar does not maintain any type info
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33569>
_______________________________________

From report at bugs.python.org  Fri May 18 09:00:38 2018
From: report at bugs.python.org (Christian Heimes)
Date: Fri, 18 May 2018 13:00:38 +0000
Subject: [New-bugs-announce] [issue33570] OpenSSL 1.1.1 / TLS 1.3 cipher
 suite changes
Message-ID: <1526648438.42.0.682650639539.issue33570@psf.upfronthosting.co.za>


New submission from Christian Heimes <lists at cheimes.de>:

The definition and configuration of TLS 1.3 cipher suites has changed during the development phase of OpenSSL 1.1.1. The cipher suites are no longer prefixed with "TLS13-". TLS 1.3 are always enabled and can no longer be disabled with SSLContext.set_ciphers() / SSL_CTX_set_cipher_list(). Instead the suites are now configured with SSL_CTX_set_ciphersuites(). See https://github.com/openssl/openssl/pull/5392

For now I'm not going to expose the new API. Instead I'll update the documentation and tests for 2.7 to 3.8 with new names. I'll also mention that TLS 1.3 suites will be always available with OpenSSL 1.1.1.

----------
assignee: christian.heimes
components: SSL
messages: 317027
nosy: alex, benjamin.peterson, christian.heimes, dstufft, janssen, ned.deily
priority: high
severity: normal
stage: test needed
status: open
title: OpenSSL 1.1.1 / TLS 1.3 cipher suite changes
type: behavior
versions: Python 2.7, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33570>
_______________________________________

From report at bugs.python.org  Fri May 18 09:49:38 2018
From: report at bugs.python.org (=?utf-8?q?Andr=C3=A9s_Delfino?=)
Date: Fri, 18 May 2018 13:49:38 +0000
Subject: [New-bugs-announce] [issue33571] Add triple quotes to list of
 delimiters that trigger '...' prompt
Message-ID: <1526651378.97.0.682650639539.issue33571@psf.upfronthosting.co.za>


Change by Andr?s Delfino <adelfino at gmail.com>:


----------
assignee: docs at python
components: Documentation
nosy: adelfino, docs at python
priority: normal
severity: normal
status: open
title: Add triple quotes to list of delimiters that trigger '...' prompt
type: enhancement
versions: Python 2.7, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33571>
_______________________________________

From report at bugs.python.org  Fri May 18 11:19:50 2018
From: report at bugs.python.org (Janusz Harkot)
Date: Fri, 18 May 2018 15:19:50 +0000
Subject: [New-bugs-announce] [issue33572] False/True as dictionary keys
 treated as integers
Message-ID: <1526656790.96.0.682650639539.issue33572@psf.upfronthosting.co.za>


New submission from Janusz Harkot <janusz.harkot at gmail.com>:

using boolean (True/False) as dictionary keys, coerce them to integers - is this behavior documented somewhere?

I know that asking to fix this is not easy fix, but shouldn't this be highlighted everywhere with red flags and warnings, so people will know that this is expected?


Python 3.6.5 (default, Mar 29 2018, 03:28:50)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> dta = {False: 'false', True: 'true', 0: 'zero', 1: 'one'}
>>> print(dta[False])
zero
>>>

----------
components: Interpreter Core
messages: 317032
nosy: Janusz Harkot
priority: normal
severity: normal
status: open
title: False/True as dictionary keys treated as integers
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33572>
_______________________________________

From report at bugs.python.org  Fri May 18 15:29:45 2018
From: report at bugs.python.org (W deW)
Date: Fri, 18 May 2018 19:29:45 +0000
Subject: [New-bugs-announce] [issue33573] statistics.median does not work
 with ordinal scale
Message-ID: <1526671785.76.0.682650639539.issue33573@psf.upfronthosting.co.za>


New submission from W deW <felixfilicibus at gmail.com>:

The 0.5-quantile or median is defined for ordinal, interval, and ratio scales. An Enumerator as derived from Enum and extended with rich comparison methods implements an ordinal scale. Therefore calculating the median over a list of such enum-elements ought to be possible.

The current implementation tries to interpolate the median value by averaging the two middle observations. This is allowed for interval and ratio scales, but since this interpolation involves an addition, not so for ordinal scales. Although computationally it is possible to do this for numeric ordinal variables, logically it is non-sense for the distance between ordinal values is - by definition - unknown. On non-numeric ordinal values it is even computationally impossible.

The correct return value would be: the first value in an ordered set where al least half the number of observations is smaller or equal than it. This is observation[len(observation)//2] for odd and even length ordered lists of values.

Whether the same applies to interval and ratio scales is a matter of opinion. The currently implemented algorith definitely is more popular these days.

----------
components: Library (Lib)
files: testMedian.py
messages: 317048
nosy: W deW
priority: normal
severity: normal
status: open
title: statistics.median does not work with ordinal scale
type: crash
versions: Python 3.4
Added file: https://bugs.python.org/file47601/testMedian.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33573>
_______________________________________

From report at bugs.python.org  Fri May 18 15:30:27 2018
From: report at bugs.python.org (Khalid Moh'd.)
Date: Fri, 18 May 2018 19:30:27 +0000
Subject: [New-bugs-announce] [issue33574] Conversion of Number to
 String(str(number))
Message-ID: <1526671827.02.0.682650639539.issue33574@psf.upfronthosting.co.za>


New submission from Khalid Moh'd. <khld.163 at gmail.com>:

Consider conversion of an integer to string:
a=5 #number
str #outputs <class 'str'>
str(a) #works perfectly and prints '5'

Now, consider:
str="Hello World" #reads the string
str #prints "Hello World"
str(5) #gives an error


Interpreter considers variable str before parenthesis not str from <class 'str'>
There must be a way to distinguish user defined variable str and conversion operation str().

----------
messages: 317049
nosy: Khalid Moh'd.
priority: normal
severity: normal
status: open
title: Conversion of Number to String(str(number))
type: compile error
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33574>
_______________________________________

From report at bugs.python.org  Fri May 18 20:09:21 2018
From: report at bugs.python.org (Gregory P. Smith)
Date: Sat, 19 May 2018 00:09:21 +0000
Subject: [New-bugs-announce] [issue33575] Python relies on C undefined
 behavior float-cast-overflow
Message-ID: <1526688561.12.0.682650639539.issue33575@psf.upfronthosting.co.za>


New submission from Gregory P. Smith <greg at krypto.org>:

Clang's undefined behavior sanitizer is flagging several places in CPython where it is relying on float-cast-overflow behavior.  Typically exposed where an out of bounds floating point value is cast to another type.

The clang compiler is about to start applying optimizations that alters the previous version of the undefined behavior on some platforms.  We need to make CPython clean for float-cast-overflow errors.

examples:
 _PyTime_DoubleToDenominator https://github.com/python/cpython/blob/master/Python/pytime.c#L159
 _PyTime_FromFloatObject - https://github.com/python/cpython/blob/master/Python/pytime.c#L389
 getargs double cast to a float - https://github.com/python/cpython/blob/master/Python/getargs.c#L864
 _PyFloat_Pack4 double cast to a float - https://github.com/python/cpython/blob/master/Objects/floatobject.c#L2234

These are found by running a ubsan build with this checker enabled on test_datetime, test_getargs2, test_struct, and test_thread.

There are probably others, but our own test suite happens to trigger these.

In many cases we should use correct conversion code instead of the cast that does what we want when the value is out of bounds and without a defined conversion.  In others we might want an OverflowError or ValueError.  But preserving the existing compilers up until now behavior makes more sense from a code compatibility standpoint (ie: it is not expecting an OverflowError when we make a CPython API that takes a float as input but behind the scenes uses a C API that operates on an int64 - that is an implementation detail no user should care about).

----------
components: Extension Modules, Interpreter Core
messages: 317074
nosy: gregory.p.smith
priority: normal
severity: normal
stage: needs patch
status: open
title: Python relies on C undefined behavior float-cast-overflow
type: behavior
versions: Python 2.7, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33575>
_______________________________________

From report at bugs.python.org  Sat May 19 01:41:19 2018
From: report at bugs.python.org (Nick Coghlan)
Date: Sat, 19 May 2018 05:41:19 +0000
Subject: [New-bugs-announce] [issue33576] Remove exception wrapping from
 __set_name__ calls
Message-ID: <1526708479.7.0.682650639539.issue33576@psf.upfronthosting.co.za>


New submission from Nick Coghlan <ncoghlan at gmail.com>:

Type creation currently wraps all exceptions raised by __set_name__ calls with a generic RuntimeError: https://github.com/python/cpython/blob/master/Objects/typeobject.c#L7263

Unfortunately, this makes it difficult to use __set_name__ for descriptor validation operations, since it means the specific exception type gets lost, and it makes the traceback much harder to read (since the generic error messages appears at the end, while the actual error appears somewhere in the middle).

See https://bugs.python.org/issue21145#msg317097 for a specific example.

----------
components: Interpreter Core
messages: 317098
nosy: ncoghlan
priority: normal
severity: normal
stage: needs patch
status: open
title: Remove exception wrapping from __set_name__ calls
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33576>
_______________________________________

From report at bugs.python.org  Sat May 19 01:41:49 2018
From: report at bugs.python.org (Carl Meyer)
Date: Sat, 19 May 2018 05:41:49 +0000
Subject: [New-bugs-announce] [issue33577] remove wrapping of __set_name__
 exceptions in RuntimeError
Message-ID: <1526708509.94.0.682650639539.issue33577@psf.upfronthosting.co.za>


New submission from Carl Meyer <carl at oddbird.net>:

Per Nick Coghlan in discussion on issue21145:

"I think it would make sense to remove the exception wrapping from the __set_name__ calls - I don't think we're improving the ease of understanding the tracebacks by converting everything to a generic RuntimeError, and we're hurting the UX of descriptor validation cases like this one."

https://github.com/python/cpython/blob/master/Objects/typeobject.c#L7263

----------
components: Interpreter Core
messages: 317099
nosy: carljm
priority: normal
severity: normal
status: open
title: remove wrapping of __set_name__ exceptions in RuntimeError
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33577>
_______________________________________

From report at bugs.python.org  Sat May 19 03:00:06 2018
From: report at bugs.python.org (Christopher Thorne)
Date: Sat, 19 May 2018 07:00:06 +0000
Subject: [New-bugs-announce] [issue33578] cjkcodecs missing getstate and
 setstate implementations
Message-ID: <1526713206.06.0.682650639539.issue33578@psf.upfronthosting.co.za>


New submission from Christopher Thorne <libcthorne at gmail.com>:

The cjkcodecs module provides support for various Chinese/Japanese/Korean codecs through its MultibyteIncrementalDecoder and MultibyteIncrementalEncoder classes.

While working with one of these cjkcodecs (euc-jp), I came across an issue where calling TextIOWrapper.tell() was causing a decode error on a valid euc-jp file:

>>> with open("/tmp/test", "w", encoding="euc-jp") as f:
...     f.write("AB\n??\n")
...
6
>>> with open("/tmp/test", "r", encoding="euc-jp") as f:
...     f.readline()
...     f.tell()
...     f.readline()
...     f.readline()
...
'AB\n'
4
'??\n'
Traceback (most recent call last):
  File "<stdin>", line 5, in <module>
UnicodeDecodeError: 'euc_jp' codec can't decode byte 0xa4 in position 0: incomplete multibyte sequence

Without the tell(), there is no problem:

>>> with open("/tmp/test", "w", encoding="euc-jp") as f:
...     f.write("AB\n??\n")
...
6
>>> with open("/tmp/test", "r", encoding="euc-jp") as f:
...     f.readline()
...     f.readline()
...     f.readline()
...
'AB\n'
'??\n'
''

After looking at _io_TextIOWrapper_tell_impl in textio.c, I understood that tell() is not as trivial as one might expect as it is using a "reconstruction algorithm" [1] to account for buffered chunk reads. By default, TextIOWrapper reads from its underlying stream in chunks of 8092 bytes and then decodes and buffers this data for speed efficiency. As a result, TextIOWrapper.tell() can't just call tell() on the underlying stream because the stream's tell() will usually be too far ahead. Thus, a reconstruction algorithm is used to calculate how many bytes of the buffered chunk have actually been read so far by the user.

The reconstruction algorithm could be trivial for single byte codecs - just track the number of characters read so far, e.g. each time read() is called. However, for multibyte codecs where the number of bytes representing a character is not uniform nor reported by the decoder, the character count alone isn't sufficient. What CPython does for this is jump to a "snapshot" point where the decoder is in a clean state (i.e. not halfway through a multibyte read) and feeds it bytes to generate characters, counting as it goes, until the number of characters it tracked from user reads are generated. From this, it knows the number of bytes it took to reach the point the user is at and can calculate the correct tell() value.

Now this is where the problem comes in: the reconstruction algorithm depends on a reliable way to detect when the decoder is in a "clean state". The getstate method [2], which returns any pending data the decoder has, is used for this. However, in the case of cjkcodecs, the getstate implementation is missing. This can be seen in the following example:

>>> import codecs
>>> decoder = codecs.getincrementaldecoder('euc_jp')()
... decoder.decode(b'\xa4\xa6') # Decode a complete input sequence
'?'
>>> decoder.getstate() # There should be no state here (working)
(b'', 0)
>>> decoder.decode(b'\xa4') # Decode first half of a partial input sequence
''
>>> decoder.getstate() # There should be state here (not working)
(b'', 0)

Internally, however, the cjkcodecs do hold state. They just don't expose it.

This leads to unexpected results in the reconstruction algorithm, such as the tell() bug demonstrated above.

It appears decoder.setstate [3], encoder.getstate [4], encoder.setstate [5], are also not implemented for the cjkcodecs. This leads to other issues in code that assumes their implementaton is present (e.g. TextIOWrapper.seek).

I will attach a PR shortly with some tests and proposed implementations. This is my first time working with the CPython codebase, so apologies if I've overlooked anything.

Finally, here is a somewhat related issue with a mention of the same tell() bug at the end: https://bugs.python.org/issue25863
However, the main problem identified in that report requires further changes than just adding getstate as it's caused by the more fundamental issue of encoder and decoder state not being kept in sync.
(Also, I have added the reporter of that issue to the nosy list for this one as I assume they have some familiarity with this bug)

[1] https://docs.python.org/3/library/io.html#id3
[2] https://docs.python.org/3/library/codecs.html#codecs.IncrementalDecoder.getstate
[3] https://docs.python.org/3/library/codecs.html#codecs.IncrementalDecoder.setstate
[4] https://docs.python.org/3/library/codecs.html#codecs.IncrementalEncoder.getstate
[5] https://docs.python.org/3/library/codecs.html#codecs.IncrementalEncoder.setstate

----------
components: IO, Tests
messages: 317106
nosy: libcthorne, martin.panter
priority: normal
severity: normal
status: open
title: cjkcodecs missing getstate and setstate implementations
type: behavior
versions: Python 2.7, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33578>
_______________________________________

From report at bugs.python.org  Sat May 19 06:41:39 2018
From: report at bugs.python.org (Eitan Adler)
Date: Sat, 19 May 2018 10:41:39 +0000
Subject: [New-bugs-announce] [issue33579] calendar.timegm not always an
 inverse of time.gmtime
Message-ID: <1526726499.81.0.682650639539.issue33579@psf.upfronthosting.co.za>


New submission from Eitan Adler <lists at eitanadler.com>:

How to reproduce:
?cat bad.py
import time
import calendar

one = time.gmtime(1234567899)
two = calendar.timegm(time.gmtime(1234567899))
three = time.gmtime(two)

print(one)
print(two)
print(three)
print(one == three)


Expected behavior:
the functions behave as documented: they are inverses


Actual behavior:

?/srv/src/python/cpython/python bad.py
time.struct_time(tm_year=2009, tm_mon=2, tm_mday=13, tm_hour=23, tm_min=31, tm_sec=15, tm_wday=4, tm_yday=44, tm_isdst=0)
1234567875
time.struct_time(tm_year=2009, tm_mon=2, tm_mday=13, tm_hour=23, tm_min=30, tm_sec=51, tm_wday=4, tm_yday=44, tm_isdst=0)
False

Details: python built from f65e31fee3 under debug mode

----------
messages: 317111
nosy: eitan.adler
priority: normal
severity: normal
status: open
title: calendar.timegm not always an inverse of time.gmtime

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33579>
_______________________________________

From report at bugs.python.org  Sat May 19 16:13:09 2018
From: report at bugs.python.org (=?utf-8?q?Andr=C3=A9s_Delfino?=)
Date: Sat, 19 May 2018 20:13:09 +0000
Subject: [New-bugs-announce] [issue33580] Make binary/text file glossary
 entries follow most common "see also" style
Message-ID: <1526760789.23.0.682650639539.issue33580@psf.upfronthosting.co.za>


New submission from Andr?s Delfino <adelfino at gmail.com>:

While most entries don't show "see also" as a separate block, binary/text file entries do.

I'm proposing to change this.

----------
assignee: docs at python
components: Documentation
messages: 317135
nosy: adelfino, docs at python
priority: normal
severity: normal
status: open
title: Make binary/text file glossary entries follow most common "see also" style
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33580>
_______________________________________

From report at bugs.python.org  Sat May 19 18:00:07 2018
From: report at bugs.python.org (Antony Lee)
Date: Sat, 19 May 2018 22:00:07 +0000
Subject: [New-bugs-announce] [issue33581] Document "optional components that
 are commonly included in Python distributions."
Message-ID: <1526767207.27.0.682650639539.issue33581@psf.upfronthosting.co.za>


New submission from Antony Lee <anntzer.lee at gmail.com>:

The stdlib docs intro include the following sentences:

    It also describes some of the optional components that are commonly included in Python distributions.

    For Unix-like operating systems Python is normally provided as a collection of packages, so it may be necessary to use the packaging tools provided with the operating system to obtain some or all of the optional components.

However, as far as I can tell, there is no easy way to actually know what parts of the stdlib are "optional".  The _thread module's doc does state "The module is optional.", and the threading module's doc also implies that it is optional ("The dummy_threading module is provided for situations where threading cannot be used because _thread is missing.") (yes, I know support for threadless builds is going away in 3.7; that's irrelevant here.), but even ensurepip's docs don't state that it may be missing (even though I believe(?) that it is indeed removed (and intended to be so) from some linux distribution's Pythons).

A master list of "optional" features would thus be welcome.  Its usefulness is to know what parts of the stdlib we can actually rely on; e.g. in https://github.com/matplotlib/matplotlib/issues/10866 a Matplotlib user noted that bz2 is not present in all Python installs.

----------
assignee: docs at python
components: Documentation
messages: 317140
nosy: Antony.Lee, docs at python
priority: normal
severity: normal
status: open
title: Document "optional components that are commonly included in Python distributions."

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33581>
_______________________________________

From report at bugs.python.org  Sat May 19 18:55:49 2018
From: report at bugs.python.org (Matthias Bussonnier)
Date: Sat, 19 May 2018 22:55:49 +0000
Subject: [New-bugs-announce] [issue33582] formatargspec deprecated but does
 nto emit DeprecationWarning.
Message-ID: <1526770549.95.0.682650639539.issue33582@psf.upfronthosting.co.za>


New submission from Matthias Bussonnier <bussonniermatthias at gmail.com>:

Documentation says formatargspec is deprecated since 3.5, but no DeprecationWarnings are emitted, the docstring also does not mention this fact.

----------
assignee: docs at python
components: Documentation
messages: 317142
nosy: docs at python, mbussonn
priority: normal
severity: normal
status: open
title: formatargspec deprecated but does nto emit DeprecationWarning.
versions: Python 3.5, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33582>
_______________________________________

From report at bugs.python.org  Sun May 20 03:42:31 2018
From: report at bugs.python.org (INADA Naoki)
Date: Sun, 20 May 2018 07:42:31 +0000
Subject: [New-bugs-announce] [issue33583] PyObject_GC_Resize() doesn't
 relink GCHead
Message-ID: <1526802151.18.0.682650639539.issue33583@psf.upfronthosting.co.za>


New submission from INADA Naoki <songofacandy at gmail.com>:

https://github.com/python/cpython/blob/master/Modules/gcmodule.c#L1750-L1763

PyObject_GC_Resize() calls PyObject_REALLOC(), which can change object's address.
But it doesn't relinking:

 g->gc.gc_prev->gc.gc_next = g
 g->gc.gc_next->gc.gc_prev = g

So this API can't be used for tracked object.
In CPython code, all callers use this API for non tracked objects.

But if some extension module author used this API for tracked objects, it will cause hard to debug behavior.

Which should we do?

1. Add relinking to PyObject_GC_Resize()
2. Add note in the document

If 1, should we backport it to 3.7 and 3.6?

----------
components: Interpreter Core
messages: 317158
nosy: inada.naoki
priority: normal
severity: normal
status: open
title: PyObject_GC_Resize() doesn't relink GCHead
type: behavior

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33583>
_______________________________________

From report at bugs.python.org  Sun May 20 05:15:22 2018
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sun, 20 May 2018 09:15:22 +0000
Subject: [New-bugs-announce] [issue33584] Fix several minor bugs in asyncio
Message-ID: <1526807722.12.0.682650639539.issue33584@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:

* get_future_loop() silenced all exceptions raised when look up the get_loop attribute, not just an AttributeError.

* enter_task() silenced all exceptions raised when look up the current task, not just a KeyError.

* repr() was called for a borrowed link in enter_task() and task_step_impl().

* str() was used instead of repr() in formatting one error message.

* There where few reference leaks.

----------
components: Extension Modules, asyncio
messages: 317160
nosy: asvetlov, serhiy.storchaka, yselivanov
priority: normal
severity: normal
status: open
title: Fix several minor bugs in asyncio
type: behavior
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33584>
_______________________________________

From report at bugs.python.org  Sun May 20 09:11:22 2018
From: report at bugs.python.org (=?utf-8?q?Ville_Skytt=C3=A4?=)
Date: Sun, 20 May 2018 13:11:22 +0000
Subject: [New-bugs-announce] [issue33585] re.sub calls repl function one
 time too many for catch-all regex
Message-ID: <1526821882.25.0.682650639539.issue33585@psf.upfronthosting.co.za>


New submission from Ville Skytt? <ville.skytta at iki.fi>:

(I'm fairly certain that the title doesn't describe the actual underlying issue all that well, however it is what I'm seeing so going with that for now.)

Compared to Python 3.6, 3.7 appears to call the repl function for re.sub one time too many, when given a catch-all regex. The extra call is made with a match consisting of an empty string. I think this is quite unexpected, and think it's a bug that I hope could be fixed before 3.7 is out.

Demonstration code:

    import re
    def repl(match):
        print(f"Called with match '{match.group(0)}'")
    re.sub(".*", repl, "foo")

3.6.3 produces the expected output:

    Called with match 'foo'

3.7.0b4+ (current git) demonstrates the extra call:

    Called with match 'foo'
    Called with match ''

----------
messages: 317166
nosy: scop
priority: normal
severity: normal
status: open
title: re.sub calls repl function one time too many for catch-all regex
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33585>
_______________________________________

From report at bugs.python.org  Sun May 20 12:28:28 2018
From: report at bugs.python.org (Eric N. Vander Weele)
Date: Sun, 20 May 2018 16:28:28 +0000
Subject: [New-bugs-announce] [issue33586] 2.7.15 missing release notes on
 download page
Message-ID: <1526833708.19.0.682650639539.issue33586@psf.upfronthosting.co.za>


New submission from Eric N. Vander Weele <ericvw at gmail.com>:

When visiting https://www.python.org/downloads/ and attempting to look at the 2.7.15 release notes, the 'Release Notes' link on the far right of the table, it resolves to https://www.python.org/downloads/.

----------
assignee: docs at python
components: Documentation
messages: 317185
nosy: docs at python, ericvw
priority: normal
severity: normal
status: open
title: 2.7.15 missing release notes on download page
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33586>
_______________________________________

From report at bugs.python.org  Sun May 20 12:38:49 2018
From: report at bugs.python.org (Pankaj Pandey)
Date: Sun, 20 May 2018 16:38:49 +0000
Subject: [New-bugs-announce] [issue33587] inspect.getsource performs
 unnecessary filesystem stat call which is ignored in some circumstances
Message-ID: <1526834329.07.0.682650639539.issue33587@psf.upfronthosting.co.za>


New submission from Pankaj Pandey <pankaj86 at gmail.com>:

The specific os.path.exists() call is shown here: https://github.com/python/cpython/pull/6805

If the filename is already in linecache or the module has a PEP 302 loader, then the os.path.exists() call is ignored. Hence it is better to call it only if its value is needed.

Attached a very simple case which shows the impact of the change:

----------
files: inspect_stack_perf.py
messages: 317186
nosy: Pankaj Pandey
priority: normal
pull_requests: 6667
severity: normal
status: open
title: inspect.getsource performs unnecessary filesystem stat call which is ignored in some circumstances
Added file: https://bugs.python.org/file47607/inspect_stack_perf.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33587>
_______________________________________

From report at bugs.python.org  Sun May 20 17:51:51 2018
From: report at bugs.python.org (Mat Leonard)
Date: Sun, 20 May 2018 21:51:51 +0000
Subject: [New-bugs-announce] [issue33588] Unicode function arguments aren't
 preserved
Message-ID: <1526853111.38.0.682650639539.issue33588@psf.upfronthosting.co.za>


New submission from Mat Leonard <leonard.mat at gmail.com>:

Unicode symbols used as function arguments aren't preserved in the variable names available from .__code__.co_varnames. Example shown below.

def func(?, ?, ?, ?):
    pass

varnames = func.__code__.co_varnames
print(varnames)
print('?' == varnames[0])
print('?' == varnames[1])
print('?' == varnames[2])
print('?' == varnames[3])

>> ('?', '?', '?', '?')
>> False
>> True
>> True
>> False

I wrote some code dependent on using function arguments obtained from .__code__.co_varnames in a dictionary. Since the unicode arguments aren't preserved from defining the function and .__code__.co_varnames, the lookup in the dictionary fails.

Looks like same thing happens with the inspect module (maybe .__code__.co_varnames comes from inspect)

inspect.signature(func)
>> <Signature (?, ?, ?, ?)>

----------
components: Unicode
files: Screen Shot 2018-05-20 at 2.32.05 PM.png
messages: 317201
nosy: ezio.melotti, mcleonard, vstinner
priority: normal
severity: normal
status: open
title: Unicode function arguments aren't preserved
type: behavior
versions: Python 3.6
Added file: https://bugs.python.org/file47608/Screen Shot 2018-05-20 at 2.32.05 PM.png

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33588>
_______________________________________

From report at bugs.python.org  Sun May 20 20:23:25 2018
From: report at bugs.python.org (INADA Naoki)
Date: Mon, 21 May 2018 00:23:25 +0000
Subject: [New-bugs-announce] [issue33589] Remove dummy member in GCHead
Message-ID: <1526862205.85.0.682650639539.issue33589@psf.upfronthosting.co.za>


New submission from INADA Naoki <songofacandy at gmail.com>:

https://github.com/python/cpython/commit/e348c8d154cf6342c79d627ebfe89dfe9de23817

> We can probably get rid of the double and this union hack all together today.
> That is a slightly more invasive change that can be left for later.

Can we do it for now (Python 3.8)?

----------
components: Interpreter Core
messages: 317212
nosy: inada.naoki, pitrou
priority: normal
severity: normal
status: open
title: Remove dummy member in GCHead
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33589>
_______________________________________

From report at bugs.python.org  Mon May 21 01:43:33 2018
From: report at bugs.python.org (sahilmn)
Date: Mon, 21 May 2018 05:43:33 +0000
Subject: [New-bugs-announce] [issue33590] sched.enter priority has no impact
 on execution
Message-ID: <1526881413.19.0.682650639539.issue33590@psf.upfronthosting.co.za>


New submission from sahilmn <sahilmanocha1994 at gmail.com>:

`sched.enter` doesn't work as expected. If two events are scheduled with the same delay, then their order of execution seems to be dictated by the order of `enter` statements for the events instead of the priority order.

Ref attached file with example code. `sched.enterabs` works as expected.

----------
files: sched.enter.bug.py
messages: 317215
nosy: sahilmn
priority: normal
severity: normal
status: open
title: sched.enter priority has no impact on execution
versions: Python 3.6
Added file: https://bugs.python.org/file47609/sched.enter.bug.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33590>
_______________________________________

From report at bugs.python.org  Mon May 21 06:56:08 2018
From: report at bugs.python.org (Robert)
Date: Mon, 21 May 2018 10:56:08 +0000
Subject: [New-bugs-announce] [issue33591] ctypes does not support fspath
 protocol
Message-ID: <1526900168.47.0.682650639539.issue33591@psf.upfronthosting.co.za>


New submission from Robert <lma at posteo.de>:

Passing a non-str path which fulfills the new fspath-protcol (like a Path() object) to CDLL (or WinDLL, PyDLL, ...) a exception is returned.

----------
components: ctypes
messages: 317230
nosy: mrh1997
priority: normal
severity: normal
status: open
title: ctypes does not support fspath protocol
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33591>
_______________________________________

From report at bugs.python.org  Mon May 21 13:28:43 2018
From: report at bugs.python.org (Elvis Pranskevichus)
Date: Mon, 21 May 2018 17:28:43 +0000
Subject: [New-bugs-announce] [issue33592] Document contextvars C API
Message-ID: <1526923723.99.0.682650639539.issue33592@psf.upfronthosting.co.za>


New submission from Elvis Pranskevichus <elvis at magic.io>:

The C API for PEP 567 is currently missing.

----------
assignee: docs at python
components: Documentation
messages: 317244
nosy: Elvis.Pranskevichus, docs at python, yselivanov
priority: normal
severity: normal
status: open
title: Document contextvars C API
type: enhancement
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33592>
_______________________________________

From report at bugs.python.org  Mon May 21 17:11:04 2018
From: report at bugs.python.org (Diego Argueta)
Date: Mon, 21 May 2018 21:11:04 +0000
Subject: [New-bugs-announce] [issue33593] Support heapq on typed arrays?
Message-ID: <1526937064.06.0.682650639539.issue33593@psf.upfronthosting.co.za>


New submission from Diego Argueta <diego.argueta at gmail.com>:

It'd be really great if we could have support for using the `heapq` module on typed arrays from `array`. For example:


```
import array
import heapq
import random

a = array.array('I', (random.randrange(10) for _ in range(10)))
heapq.heapify(a)
```

Right now this code throws a TypeError:

    TypeError: heap argument must be a list


I suppose I could use `bisect` to insert items one by one but I imagine a single call to heapify() would be more efficient, especially if I'm loading the array from a byte string.

>From what I can tell the problem lies in the C implementation, since removing the _heapq imports at the end of the heapq module (in 3.6) makes it work.

----------
components: Library (Lib)
messages: 317250
nosy: da
priority: normal
severity: normal
status: open
title: Support heapq on typed arrays?
type: enhancement
versions: Python 2.7, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33593>
_______________________________________

From report at bugs.python.org  Mon May 21 17:34:32 2018
From: report at bugs.python.org (Matthias Bussonnier)
Date: Mon, 21 May 2018 21:34:32 +0000
Subject: [New-bugs-announce] [issue33594] add deprecation since 3.5 for a
 few methods of inspect.
Message-ID: <1526938472.55.0.682650639539.issue33594@psf.upfronthosting.co.za>


New submission from Matthias Bussonnier <bussonniermatthias at gmail.com>:

inspect's getargspec, as well as Signature's from_function, and from_builtin are deprecated. 

They all emits DeprecationWarning but: 

The deprecation warning does not say since which version since are deprecated. 
For the two Signature's methods, there is no documentation available, so finding this information is difficult. 

Many tool rely on introspection to provide object help via docstring, which also did not include this deprecation notice. 

Adding the deprecation informations to these 3 function/methods would be good to foster migration to newer ways.

----------
assignee: docs at python
components: Documentation
messages: 317252
nosy: docs at python, mbussonn
priority: normal
severity: normal
status: open
title: add deprecation since 3.5 for a few methods of inspect.
versions: Python 3.5, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33594>
_______________________________________

From report at bugs.python.org  Mon May 21 18:45:53 2018
From: report at bugs.python.org (=?utf-8?q?Andr=C3=A9s_Delfino?=)
Date: Mon, 21 May 2018 22:45:53 +0000
Subject: [New-bugs-announce] [issue33595] FIx references to lambda
 "arguments"
Message-ID: <1526942753.14.0.682650639539.issue33595@psf.upfronthosting.co.za>


New submission from Andr?s Delfino <adelfino at gmail.com>:

There a couple of references to lambda parameters mentioned as arguments. I'm proposing fixing them.

----------
assignee: docs at python
components: Documentation
messages: 317255
nosy: adelfino, docs at python
priority: normal
severity: normal
status: open
title: FIx references to lambda "arguments"
type: enhancement
versions: Python 2.7, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33595>
_______________________________________

From report at bugs.python.org  Tue May 22 02:55:34 2018
From: report at bugs.python.org (yzongyue)
Date: Tue, 22 May 2018 06:55:34 +0000
Subject: [New-bugs-announce] [issue33596] fix memory leak in
 lib/json/scanner.py py_make_scanner
Message-ID: <1526972134.7.0.682650639539.issue33596@psf.upfronthosting.co.za>


New submission from yzongyue <yzongyue at gmail.com>:

in function py_make_scanner, the last return statements is "return _scan_once"
it should be "return scan_once", or  the memo never clear.
in fact, make_scanner use c_make_scanner rather  than py_make_scanner, so this bug never occur in default

----------
components: Library (Lib)
files: ??_2018-05-22_14-53-27.png
messages: 317261
nosy: yzongyue
priority: normal
severity: normal
status: open
title: fix memory leak in lib/json/scanner.py py_make_scanner
versions: Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8
Added file: https://bugs.python.org/file47610/??_2018-05-22_14-53-27.png

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33596>
_______________________________________

From report at bugs.python.org  Tue May 22 03:06:22 2018
From: report at bugs.python.org (INADA Naoki)
Date: Tue, 22 May 2018 07:06:22 +0000
Subject: [New-bugs-announce] [issue33597] Compact PyGC_Head
Message-ID: <1526972782.65.0.682650639539.issue33597@psf.upfronthosting.co.za>


New submission from INADA Naoki <songofacandy at gmail.com>:

Currently, PyGC_Head takes three words; gc_prev, gc_next, and gc_refcnt.

gc_refcnt is used when collecting, for trial deletion.
gc_prev is used for tracking and untracking.

So if we can avoid tracking/untracking while trial deletion, gc_prev and gc_refcnt can share same memory space.

This idea reduces PyGC_Head size to two words.

----------
components: Interpreter Core
messages: 317262
nosy: inada.naoki, pitrou
priority: normal
severity: normal
status: open
title: Compact PyGC_Head
type: resource usage
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33597>
_______________________________________

From report at bugs.python.org  Tue May 22 05:33:46 2018
From: report at bugs.python.org (Alex Walters)
Date: Tue, 22 May 2018 09:33:46 +0000
Subject: [New-bugs-announce] [issue33598] ActiveState Recipes links in docs,
 and the apparent closure of Recipes
Message-ID: <1526981626.86.0.682650639539.issue33598@psf.upfronthosting.co.za>


New submission from Alex Walters <tritium-list at sdamon.com>:

ActiveState has stopped accepting new recipes on their website, and migrated all current recipes to a Github repo.  I have seen no official announcement of a shutdown date for the code.activestate.com website, but it's future has to be in question considering the migration.

I propose we go through the docs for all the recipes, and either rescue them to a section of the docs (perhaps to a new "Code Examples" section), incorporate the recipes into the doc pages where appropriate, or excise them entirely.

Another option would be to update the links to the GitHub repo, but I am less enthusiastic about that option - GitHub repos come and go, and mutate structure easily.  I don't think it would be a good idea to deep link into a repository not controlled by the Python project.

It would stink to have a bunch of suddenly dead links in the docs.

----------
assignee: docs at python
components: Documentation
messages: 317265
nosy: docs at python, rhettinger, tritium
priority: normal
severity: normal
status: open
title: ActiveState Recipes links in docs, and the apparent closure of Recipes

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33598>
_______________________________________

From report at bugs.python.org  Tue May 22 05:56:10 2018
From: report at bugs.python.org (Sascha)
Date: Tue, 22 May 2018 09:56:10 +0000
Subject: [New-bugs-announce] [issue33599] Copying objects subclassed from
 SimpleNamespace doesn't work
Message-ID: <1526982970.55.0.682650639539.issue33599@psf.upfronthosting.co.za>


New submission from Sascha <sascha.schlemmer at me.com>:

Try 
from types import SimpleNamespace
import copy
class Person(SimpleNamespace):
    def __init__(self, name, **kwargs):
        self.name = name
        super().__init__(**kwargs)

bob = Person('Bob', job='tester')
clone = copy.copy(bob)

For me this results in 
TypeError: __init__() missing 1 required positional argument: 'name'

----------
components: Library (Lib)
messages: 317266
nosy: stereobutter
priority: normal
severity: normal
status: open
title: Copying objects subclassed from SimpleNamespace doesn't work
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33599>
_______________________________________

From report at bugs.python.org  Tue May 22 09:05:21 2018
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 22 May 2018 13:05:21 +0000
Subject: [New-bugs-announce] [issue33600] [EASY DOC] Python 2: document that
 platform.linux_distribution() has been removed
Message-ID: <1526994321.44.0.682650639539.issue33600@psf.upfronthosting.co.za>


New submission from STINNER Victor <vstinner at redhat.com>:

The platform.linux_distribution() function has been removed from the future Python 3.8. The removal should be documented in Python 2.7 documentation, and mention the alternative.

Python 2 documenation:
https://docs.python.org/2/library/platform.html#platform.linux_distribution

Python 3 documentation with the deprecation warning:
https://docs.python.org/3/library/platform.html#platform.linux_distribution

IMHO it's too late to deprecate the function from Python 2. We should only add a quick note.

Function removed in: bpo-28167.

----------
assignee: docs at python
components: Documentation
messages: 317275
nosy: docs at python, vstinner
priority: normal
severity: normal
status: open
title: [EASY DOC] Python 2: document that platform.linux_distribution() has been removed
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33600>
_______________________________________

From report at bugs.python.org  Tue May 22 09:49:41 2018
From: report at bugs.python.org (Serhiy Storchaka)
Date: Tue, 22 May 2018 13:49:41 +0000
Subject: [New-bugs-announce] [issue33601] Py_UTF8Mode is not documented
Message-ID: <1526996981.47.0.682650639539.issue33601@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:

Py_UTF8Mode was added to the limited API in 3.7, but it is not documented anywhere.

----------
assignee: docs at python
components: Documentation
messages: 317286
nosy: docs at python, ncoghlan, serhiy.storchaka, vstinner
priority: normal
severity: normal
status: open
title: Py_UTF8Mode is not documented
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33601>
_______________________________________

From report at bugs.python.org  Tue May 22 10:28:08 2018
From: report at bugs.python.org (=?utf-8?q?Andr=C3=A9s_Delfino?=)
Date: Tue, 22 May 2018 14:28:08 +0000
Subject: [New-bugs-announce] [issue33602] Remove set and queue references
 from Data Types
Message-ID: <1526999288.77.0.682650639539.issue33602@psf.upfronthosting.co.za>


New submission from Andr?s Delfino <adelfino at gmail.com>:

Data Types mentions sets (which are now built-in) and synchronized queues (now mentioned in Concurrent Execution). I'm proposing fixing this.

PR also adds mention to bytearray.

----------
assignee: docs at python
components: Documentation
messages: 317293
nosy: adelfino, docs at python
priority: normal
severity: normal
status: open
title: Remove set and queue references from Data Types
type: enhancement
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33602>
_______________________________________

From report at bugs.python.org  Tue May 22 10:59:21 2018
From: report at bugs.python.org (GranPrego)
Date: Tue, 22 May 2018 14:59:21 +0000
Subject: [New-bugs-announce] [issue33603] Subprocess Thread handles grow
 with each call and aren't released until script ends
Message-ID: <1527001161.43.0.682650639539.issue33603@psf.upfronthosting.co.za>


New submission from GranPrego <josephdeller at googlemail.com>:

On windows 7 / 10 I'm using subprocess to launch a dos cmdline executable and returning the results, which is all working fine.
However, each time I make a call, the Python handle count is gradually increasing, jumping up , back a few, then jumping up and so on.

All the handles are released when the script exits, but quite often python just hangs after a few hours.  If I use process explorer to investigate I can see that python has an increasing number of Thread handles, even though I can see the process being called and cleanly exiting.

Unfortunately I'm stuck with the dos executable and it's always a one shot of sending it a single command each time and the script calls it a lot. The executable is just taking a string cmdline and returning a couple of lines of text and then exiting.  It only runs for a couple of seconds at most.

I've tried two variants of calling the process, I was hoping that the with variant would clean up, but there is no difference.

Each handle object that gets left behind has a single reference and a non paged quota of 1192, 0 paged.

The script is long running and I've seen the handle count reach 46K.


result = ""
with Popen ([fcptool, parameters],  stdout=PIPE, universal_newlines=True, bufsize=1) as process:
        for line in process.stdout:
            result = result + line

return result
or 
p = subprocess.run([fcptool, parameters], stdout=subprocess.PIPE,              stderr=subprocess.STDOUT, universal_newlines=True, shell=True).stdout


return p

I can reproduce this on 3 different machines, 2 windows 7 and one windows 10, all Python 3.6.   I can't see a way around this at the moment and as far as I can tell, I'm using the call to subprocess correctly.

----------
components: Interpreter Core, Windows
messages: 317296
nosy: GranPrego, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Subprocess Thread handles  grow with each call and aren't released until script ends
type: resource usage
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33603>
_______________________________________

From report at bugs.python.org  Tue May 22 14:46:00 2018
From: report at bugs.python.org (Matthias Bussonnier)
Date: Tue, 22 May 2018 18:46:00 +0000
Subject: [New-bugs-announce] [issue33604] HMAC default to MD5 marked as to
 be removed in 3.6
Message-ID: <1527014760.1.0.682650639539.issue33604@psf.upfronthosting.co.za>


New submission from Matthias Bussonnier <bussonniermatthias at gmail.com>:

HMAC docs says digestmod=md5 default will be removed in 3.6, but was not. 

We should likely bumpt that to 3.8 in the documentation, and actually remove it in 3.8

----------
messages: 317322
nosy: mbussonn
priority: normal
severity: normal
status: open
title: HMAC default to MD5 marked as to be removed in 3.6
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33604>
_______________________________________

From report at bugs.python.org  Tue May 22 15:00:04 2018
From: report at bugs.python.org (=?utf-8?b?SHJ2b2plIE5pa8WhacSH?=)
Date: Tue, 22 May 2018 19:00:04 +0000
Subject: [New-bugs-announce] [issue33605] Detect accessing event loop from a
 different thread outside of _debug
Message-ID: <1527015604.6.0.682650639539.issue33605@psf.upfronthosting.co.za>


New submission from Hrvoje Nik?i? <hniksic at gmail.com>:

Looking at StackOverflow's python-asyncio tag[1], it appears that it's a very common mistake for users to invoke asyncio functions or methods from a thread other than the event loop thread. In some cases this happens because the user is careless with threads and hasn't read the documentation. But in many cases what happens is that a third-party library invoked a callback in a different thread without making it transparent that that's what it's doing.

The trouble is that in many cases doing these things, e.g. calling loop.call_soon() or loop.create_task() from the wrong thread, will *appear to work*. The typical event loop is busy servicing different coroutines, so a function or task enqueued without a proper wakeup gets picked up soon enough. This is, of course, a disaster waiting to happen because it could easily lead to corruption of event loop's data structures. But users don't know that, and many of them become aware of the problem only after wondering "why does my code start working when I add a coroutine that does nothing but asyncio.sleep(0.1) in an infinite loop?" Some may never even fix their code, just assuming that asyncio takes a long time to process a new task or something like that.

I suggest that asyncio should be stricter about this error and that methods and functions that operate on the event loop, such as call_soon, call_later, create_task, ensure_future, and close, should all call _check_thread() even when not in debug mode. _check_thread() warns that it "should only be called when self._debug == True", hinting at "performance reasons", but that doesn't seem justified. threading.get_ident() is efficiently implemented in C, and comparing that integer to another cached integer is about as efficient an operation as it gets.

The added benefit would be a vast improvement of robustness of asyncio-based programs, saving many hours of debugging.


[1]
Here is an incomplete list of questions where the users stumbled on this problem, and that's only from the last three months or so:

https://stackoverflow.com/questions/49906034/python-asyncio-run-forever-and-tasks
https://stackoverflow.com/questions/49851514/python-websockets-and-gtk-confused-about-asyncio-queue
https://stackoverflow.com/questions/49533612/using-asyncio-loop-reference-in-another-file
https://stackoverflow.com/questions/49093623/strange-behaviour-when-task-added-to-empty-loop-in-different-thread
https://stackoverflow.com/questions/48836285/python-asyncio-event-wait-not-responding-to-event-set
https://stackoverflow.com/questions/48833644/how-to-wait-for-asynchronous-callback-in-the-background-i-e-not-invoked-by-us
https://stackoverflow.com/questions/48695670/running-asynchronous-code-synchronously-in-separate-thread

----------
components: asyncio
messages: 317324
nosy: asvetlov, hniksic, yselivanov
priority: normal
severity: normal
status: open
title: Detect accessing event loop from a different thread outside of _debug
type: enhancement
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33605>
_______________________________________

From report at bugs.python.org  Tue May 22 15:15:59 2018
From: report at bugs.python.org (Vinay Sajip)
Date: Tue, 22 May 2018 19:15:59 +0000
Subject: [New-bugs-announce] [issue33606] Improve logging performance when
 logger disabled
Message-ID: <1527016559.5.0.682650639539.issue33606@psf.upfronthosting.co.za>


New submission from Vinay Sajip <vinay_sajip at yahoo.co.uk>:

If a logger is disabled (by setting it's disabled attribute to True), the check for this is done late in the dispatch of the logging event - during the handle() call - rather than isEnabledFor(), which would short-circuit some processing. So the check for logger.disabled should be moved to isEnabledFor().

Credit to Abhijit Gadgil for raising this:

https://stackoverflow.com/questions/50453121/logger-disabled-check-much-later-in-python-logging-module-whats-the-rationale

----------
assignee: vinay.sajip
components: Library (Lib)
messages: 317328
nosy: vinay.sajip
priority: normal
severity: normal
stage: needs patch
status: open
title: Improve logging performance when logger disabled
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33606>
_______________________________________

From report at bugs.python.org  Tue May 22 15:19:38 2018
From: report at bugs.python.org (Eric Snow)
Date: Tue, 22 May 2018 19:19:38 +0000
Subject: [New-bugs-announce] [issue33607] Explicitly track object ownership
 (and allocator).
Message-ID: <1527016778.74.0.682650639539.issue33607@psf.upfronthosting.co.za>


New submission from Eric Snow <ericsnowcurrently at gmail.com>:

When an object is created it happens relative to the current
thread (ergo interpreter) and the current allocator (part of
global state).  We do not track either of these details for
the object.  It may make sense to start doing so (reasons next).

Regarding tracking the interpreter, that originating interpreter
can be thought of as the owner.  Any lifecycle operations should
happen relative to that interpreter.  Furthermore, the object
should be used in C-API calls only in that interpreter (i.e.
when the current thread's Py_ThreadState belongs to that
interpreter).  This hasn't been an issue since currently all
interpreters in the process share the GIL, as well as the fact
that subinterpreters haven't been heavily used historically.
However, the possibility of no longer sharing the GIL suggests
that tracking the owning interpreter (and perhaps even other
"sharing" interpreters) would be important.  Furthermore,
in the last few years subinterpreters have seen increasing usage
(see Openstack Ceph), and knowing the originating interpreter
for an object can be useful there.  Regardless, even in the
single interpreter case knowing the owning interpreter is
important during runtime finalization (which is currently
slightly broken), which impacts CPython embedders.

Regarding the allocator, there used to be just a single global
one that the runtime used from start to finish.  Now the C-API
offers a way to switch the allocator, so there's no guarantee
that the right allocator is used in PyMem_Free().  This has
already had a negative impact on efforts to clean up CPython's
runtime initialization.  It also results in problems during
finalization.  Additionally, we are looking into moving the
allocator from the global runtime state to the per-interpreter
(or even per-thread or per-context) state value.  In that world
it would be essential to know which allocator was used when
creating the object.  There are other possible applications
based on knowing an object's allocator, but I'll stop there.

To sort all this out we would need to track per-object:

* originating allocator (pointer or id)
* owning interpreter (pointer or id)
* (possibly) "sharing" interpreters (linked list?)

Either we'd add 2 pointer-size fields to PyObject or we would
keep a separate hash table (or two) pointing from each object
to the info (similar to how we've considered doing for
refcounts).  To alleviate impact on the common case (not
embedded, single interpreter, same allocator), we could default
to not tracking interpreter/allocator and take a lookup failure
to mean "main interpreter, default allocator".

----------
messages: 317330
nosy: eric.snow, ncoghlan, vstinner
priority: normal
severity: normal
status: open
title: Explicitly track object ownership (and allocator).
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33607>
_______________________________________

From report at bugs.python.org  Tue May 22 15:34:41 2018
From: report at bugs.python.org (Eric Snow)
Date: Tue, 22 May 2018 19:34:41 +0000
Subject: [New-bugs-announce] [issue33608] Add a cross-interpreter-safe
 mechanism to indicate that an object may be destroyed.
Message-ID: <1527017681.69.0.682650639539.issue33608@psf.upfronthosting.co.za>


New submission from Eric Snow <ericsnowcurrently at gmail.com>:

In order to keep subinterpreters properly isolated, objects
from one interpreter should not be used in C-API calls in
another interpreter.  That is relatively straight-forward
except in one case: indicating that the other interpreter
doesn't need the object to exist any more (similar to
PyBuffer_Release() but more general).  I consider the
following solutions to be the most viable.  Both make use
of recounts to protect cross-interpreter usage (with incref
before sharing).

1. manually switch interpreters (new private function)
  a. acquire the GIL
  b. if refcount > 1 then decref and release the GIL
  c. switch
  d. new thread (or re-use dedicated one)
  e. decref
  f. kill thread
  g. switch back
  h. release the GIL
2. change pending call mechanism (see Py_AddPendingCall) to
   per-interpreter instead of global (add "interp" arg to
   signature of new private C-API function)
  a. queue a function that decrefs the object
3. new cross-interpreter-specific private C-API function
  a. queue the object for decref (a la Py_AddPendingCall)
     in owning interpreter

I favor #2, since it is more generally applicable.  #3 would
probably be built on #2 anyway.  #1 is relatively inefficient.
With #2, Py_AddPendingCall() would become a simple wrapper
around the new private function.

----------
messages: 317333
nosy: eric.snow, ncoghlan, serhiy.storchaka, vstinner, yselivanov
priority: normal
severity: normal
stage: needs patch
status: open
title: Add a cross-interpreter-safe mechanism to indicate that an object may be destroyed.
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33608>
_______________________________________

From report at bugs.python.org  Tue May 22 17:07:30 2018
From: report at bugs.python.org (Yury Selivanov)
Date: Tue, 22 May 2018 21:07:30 +0000
Subject: [New-bugs-announce] [issue33609] Document that dicts preserve
 insertion order
Message-ID: <1527023250.35.0.682650639539.issue33609@psf.upfronthosting.co.za>


New submission from Yury Selivanov <yselivanov at gmail.com>:

I don't see it documented that dicts preserve insertion order. 3.7 what's new points to [1], but that section doesn't have a "version changed" tag.

IMO, [1] should have two version changed tags: one for 3.6, and one for 3.7.

Also, it would be great if we could document how the order is preserved when a key is deleted from the dict etc.

[1] https://docs.python.org/3.7/library/stdtypes.html#mapping-types-dict

----------
assignee: docs at python
components: Documentation
messages: 317346
nosy: docs at python, inada.naoki, ned.deily, vstinner, yselivanov
priority: normal
severity: normal
status: open
title: Document that dicts preserve insertion order
type: enhancement
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33609>
_______________________________________

From report at bugs.python.org  Wed May 23 01:18:12 2018
From: report at bugs.python.org (Terry J. Reedy)
Date: Wed, 23 May 2018 05:18:12 +0000
Subject: [New-bugs-announce] [issue33610] IDLE: Improve CodeContext (various
 issues)
Message-ID: <1527052692.02.0.682650639539.issue33610@psf.upfronthosting.co.za>


New submission from Terry J. Reedy <tjreedy at udel.edu>:

There are a number of possible improvements to CodeContext.  They be separate issues (and PRs) that are dependencies of this master index issue. Some should be fairly easy now that we have the new tests.

1. #32831 added docstrings and tests.  Review has notes.
2. Follow-up may revise and do some user-invisible code cleanups.
3. #31493 cancelled the event loops when instances are deleted.
4. Spinoff from above: 1 or 2 events loops for class, not each instance.
5. #22703: separate initial context state of new window from toggling the state of current windows.  Current behavior is buggy.
6. Gray-out Options => Code Context on non-editor windows. (This would have to be revised again if windows became panes in master window.)
7. Change fixed # of lines to variable # of lines as needed, up to limit.  About 15 is limit for 4-space indents in 80 char lines. 
8. Click on context line jumps to line. Show it at top of window.
9. Reenable user config of context colors?
10. Somehow mark blocks in editor.  Subtle background color change?
Or tag just the indents, or, if add line numbers, the line numbers?

----------
assignee: terry.reedy
components: IDLE
messages: 317357
nosy: cheryl.sabella, terry.reedy
priority: normal
severity: normal
stage: test needed
status: open
title: IDLE: Improve CodeContext (various issues)
type: enhancement

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33610>
_______________________________________

From report at bugs.python.org  Wed May 23 03:01:25 2018
From: report at bugs.python.org (s)
Date: Wed, 23 May 2018 07:01:25 +0000
Subject: [New-bugs-announce] [issue33611] Fatal Python error: Py_Initialize:
 unable to load the file system codec
Message-ID: <1527058885.03.0.682650639539.issue33611@psf.upfronthosting.co.za>


New submission from s <slavamas at gmail.com>:

Fatal Python error: Py_Initialize: unable to load the file system codec
ModuleNotFoundError: No module named 'encodings'
###########################################
Process:               Python [1889]
Path:                  /Library/Frameworks/Python.framework/Versions/3.6/Resources/Python.app/Contents/MacOS/Python
Identifier:            Python
Version:               3.6.5 (3.6.5)
Code Type:             X86-64 (Native)
Parent Process:        bash [722]
Responsible:           Python [1889]
User ID:               501

Date/Time:             2018-05-22 23:52:40.997 -0700
OS Version:            Mac OS X 10.13.4 (17E202)
Report Version:        12
Anonymous UUID:        F667AFCE-E0F9-BF0F-FB74-372904C44DB5


Time Awake Since Boot: 2000 seconds

System Integrity Protection: disabled

Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_CRASH (SIGABRT)
Exception Codes:       0x0000000000000000, 0x0000000000000000
Exception Note:        EXC_CORPSE_NOTIFY

Application Specific Information:
abort() called

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   libsystem_kernel.dylib        	0x00000001018eeb6e __pthread_kill + 10
1   libsystem_pthread.dylib       	0x0000000101928080 pthread_kill + 333
2   libsystem_c.dylib             	0x00000001015c51ae abort + 127
3   org.python.python             	0x0000000100877611 Py_FatalError + 65
4   org.python.python             	0x00000001008793c1 _Py_InitializeEx_Private + 1281
5   org.python.python             	0x0000000100896b40 Py_Main + 1360
6   org.python.python             	0x0000000100000dfe 0x100000000 + 3582
7   org.python.python             	0x0000000100000c34 0x100000000 + 3124

Thread 0 crashed with X86 Thread State (64-bit):
  rax: 0x0000000000000000  rbx: 0x000000010192e380  rcx: 0x00007fff5bffef58  rdx: 0x0000000000000000
  rdi: 0x0000000000000307  rsi: 0x0000000000000006  rbp: 0x00007fff5bffef90  rsp: 0x00007fff5bffef58
   r8: 0x0000000000000000   r9: 0x0000000000000006  r10: 0x0000000000000000  r11: 0x0000000000000202
  r12: 0x0000000000000307  r13: 0x0000000000000002  r14: 0x0000000000000006  r15: 0x000000000000002d
  rip: 0x00000001018eeb6e  rfl: 0x0000000000000202  cr2: 0x0000000101bd7020
  
Logical CPU:     0
Error Code:      0x02000148
Trap Number:     133


Binary Images:
       0x100000000 -        0x100000ff7 +org.python.python (3.6.5 - 3.6.5) <C539EB14-B454-D151-7B93-6A12255582B2> /Library/Frameworks/Python.framework/Versions/3.6/Resources/Python.app/Contents/MacOS/Python
       0x100003000 -        0x10049dff7  com.apple.CoreFoundation (6.9 - 1452.23) <358C547D-E227-3228-8218-62327F4605C8> /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation
       0x100750000 -        0x100977fef +org.python.python (3.6.5, [c] 2001-2018 Python Software Foundation. - 3.6.5) <EBF198CD-A37E-A0F1-1E7C-19B9F912D34A> /Library/Frameworks/Python.framework/Versions/3.6/Python
       0x100ac0000 -        0x100ac1ffb  libSystem.B.dylib (1252.50.4) <F64430FD-30A6-328D-A476-0AED41D371C6> /usr/lib/libSystem.B.dylib
       0x100ac8000 -        0x100ac9ff3  libDiagnosticMessagesClient.dylib (104) <9712E980-76EE-3A89-AEA6-DF4BAF5C0574> /usr/lib/libDiagnosticMessagesClient.dylib
       0x100acf000 -        0x100cf6ffb  libicucore.A.dylib (59173.0.1) <CECAD5E5-3EFD-3AAC-AD9B-E355B2DD5E9C> /usr/lib/libicucore.A.dylib
       0x100dc8000 -        0x1011b93b7  libobjc.A.dylib (723) <BEEA51E2-1C4D-3672-87F6-F267B4AC6221> /usr/lib/libobjc.A.dylib
       0x10136a000 -        0x10137cffb  libz.1.dylib (70) <48C67CFC-940D-3857-8DAD-857774605352> /usr/lib/libz.1.dylib
       0x101382000 -        0x101386ff7  libcache.dylib (80) <092479CB-1008-3A83-BECF-E115F24D13C1> /usr/lib/system/libcache.dylib
       0x10138c000 -        0x101396ff3  libcommonCrypto.dylib (60118.50.1) <029F5985-9B6E-3DCB-9B96-FD007678C6A7> /usr/lib/system/libcommonCrypto.dylib
       0x1013a4000 -        0x1013abfff  libcompiler_rt.dylib (62) <968B8E3F-3681-3230-9D78-BB8732024F6E> /usr/lib/system/libcompiler_rt.dylib
       0x1013b9000 -        0x1013c2ffb  libcopyfile.dylib (146.50.5) <3885083D-50D8-3EEC-B481-B2E605180D7F> /usr/lib/system/libcopyfile.dylib
       0x1013c9000 -        0x10144efff  libcorecrypto.dylib (562.50.17) <67007279-24E1-3F30-802D-A55CD5C27946> /usr/lib/system/libcorecrypto.dylib
       0x10146b000 -        0x1014a4ff7  libdispatch.dylib (913.50.12) <848EEE57-4235-3A61-9A52-C0097DD2AB5E> /usr/lib/system/libdispatch.dylib
       0x1014df000 -        0x1014fcff7  libdyld.dylib (551.3) <CF59A5A5-288B-30E6-BD42-9056B4E4139A> /usr/lib/system/libdyld.dylib
       0x101517000 -        0x101517ffb  libkeymgr.dylib (28) <E34E283E-90FA-3C59-B48E-1277CDB9CDCE> /usr/lib/system/libkeymgr.dylib
       0x10151c000 -        0x10151cff7  liblaunch.dylib (1205.50.76) <4D52BB64-C568-3A36-8935-2480427EE2A2> /usr/lib/system/liblaunch.dylib
       0x101523000 -        0x101527ffb  libmacho.dylib (906) <1902A611-081A-3452-B11E-EBD1B166E831> /usr/lib/system/libmacho.dylib
       0x10152e000 -        0x101530ff3  libquarantine.dylib (86) <26C0BA22-8F93-3A07-9A4E-C8D53D2CE42E> /usr/lib/system/libquarantine.dylib
       0x101537000 -        0x101538ff3  libremovefile.dylib (45) <711E18B2-5BBE-3211-A916-56740C27D17A> /usr/lib/system/libremovefile.dylib
       0x10153e000 -        0x101555fff  libsystem_asl.dylib (356.50.1) <3B24F2D1-B578-359D-ADB2-0ED19A364C38> /usr/lib/system/libsystem_asl.dylib
       0x101563000 -        0x101563fff  libsystem_blocks.dylib (67) <17303FDF-0D2D-3963-B05E-B4DF63052D47> /usr/lib/system/libsystem_blocks.dylib
       0x101568000 -        0x1015f1ff7  libsystem_c.dylib (1244.50.9) <1187BFE8-4576-3247-8177-481554E1F9E7> /usr/lib/system/libsystem_c.dylib
       0x10161a000 -        0x10161dffb  libsystem_configuration.dylib (963.50.8) <DF6B5287-203E-30CB-9947-78DF446C72B8> /usr/lib/system/libsystem_configuration.dylib
       0x101624000 -        0x101627ffb  libsystem_coreservices.dylib (51) <486000D3-D8CB-3BE7-8EE5-8BF380DE6DF7> /usr/lib/system/libsystem_coreservices.dylib
       0x10162d000 -        0x10162efff  libsystem_darwin.dylib (1244.50.9) <09C21A4A-9EE0-388B-A9D9-DFF8F6758791> /usr/lib/system/libsystem_darwin.dylib
       0x101634000 -        0x10163aff7  libsystem_dnssd.dylib (878.50.17) <9033B909-BCF7-37EB-A040-ADE8081611D6> /usr/lib/system/libsystem_dnssd.dylib
       0x101641000 -        0x10168aff7  libsystem_info.dylib (517.30.1) <AB634A98-B8AA-3804-8436-38261FC8EC4D> /usr/lib/system/libsystem_info.dylib
       0x1016a2000 -        0x1016edfcb  libsystem_m.dylib (3147.50.1) <FC2E58BA-E6D5-3D2A-89CA-67F212294136> /usr/lib/system/libsystem_m.dylib
       0x1016fc000 -        0x10171bfff  libsystem_malloc.dylib (140.50.6) <7FD43735-9DDD-300E-8C4A-F909A74BDF49> /usr/lib/system/libsystem_malloc.dylib
       0x101728000 -        0x101858ff3  libsystem_network.dylib (1229.51.2) <F65ED095-8ABA-3355-93F8-B9D0505932F4> /usr/lib/system/libsystem_network.dylib
       0x101899000 -        0x1018a3ffb  libsystem_networkextension.dylib (767.50.25) <758F1414-796D-3422-83C9-92D2915A06CE> /usr/lib/system/libsystem_networkextension.dylib
       0x1018b0000 -        0x1018b9ff3  libsystem_notify.dylib (172) <08012EC0-2CD2-34BE-BF93-E7F56491299A> /usr/lib/system/libsystem_notify.dylib
       0x1018c1000 -        0x1018c4fff  libsystem_sandbox.dylib (765.50.51) <B48D256E-D7DB-3D4F-BC95-34557170C7AA> /usr/lib/system/libsystem_sandbox.dylib
       0x1018cb000 -        0x1018ccff3  libsystem_secinit.dylib (30) <DE8D14E8-A276-3FF8-AE13-77F7040F33C1> /usr/lib/system/libsystem_secinit.dylib
       0x1018d2000 -        0x1018f8ff7  libsystem_kernel.dylib (4570.51.2) <7FF1E390-2FAC-33A7-A545-60D9BB6692D4> /usr/lib/system/libsystem_kernel.dylib
       0x101912000 -        0x101919ff7  libsystem_platform.dylib (161.50.1) <6355EE2D-5456-3CA8-A227-B96E8F1E2AF8> /usr/lib/system/libsystem_platform.dylib
       0x101922000 -        0x10192dfff  libsystem_pthread.dylib (301.50.1) <0E51CCBA-91F2-34E1-BF2A-FEEFD3D321E4> /usr/lib/system/libsystem_pthread.dylib
       0x10193a000 -        0x101941ff7  libsystem_symptoms.dylib (820.50.37) <ED7350E0-B68B-374A-9AB7-DC1F195A8327> /usr/lib/system/libsystem_symptoms.dylib
       0x101948000 -        0x10195bfff  libsystem_trace.dylib (829.50.17) <6568D68B-1D4C-38EE-90A9-54821D6403C0> /usr/lib/system/libsystem_trace.dylib
       0x10196a000 -        0x10196fff7  libunwind.dylib (35.3) <BEF3FB49-5604-3B5F-82B5-332B80023AC3> /usr/lib/system/libunwind.dylib
       0x101977000 -        0x1019a4fff  libxpc.dylib (1205.50.76) <25DB244E-217D-3CE0-A8F2-0C4255783B42> /usr/lib/system/libxpc.dylib
       0x1019c7000 -        0x1019faff7  libclosured.dylib (551.3) <DC3DA678-9C40-339C-A9C6-32AB74FCC682> /usr/lib/closure/libclosured.dylib
       0x101a16000 -        0x101a3aff7  libc++abi.dylib (400.8.2) <EF5E37D7-11D9-3530-BE45-B986612D13E2> /usr/lib/libc++abi.dylib
       0x101a4a000 -        0x101aa0fff  libc++.1.dylib (400.9) <7D3DACCC-3804-393C-ABC1-1A580FD00CB6> /usr/lib/libc++.1.dylib
    0x7fff61e04000 -     0x7fff61e4e9df  dyld (551.3) <AFAB4EFA-7020-34B1-BBEF-0F26C6D3CA36> /usr/lib/dyld

External Modification Summary:
  Calls made by other processes targeting this process:
    task_for_pid: 0
    thread_create: 0
    thread_set_state: 0
  Calls made by this process:
    task_for_pid: 0
    thread_create: 0
    thread_set_state: 0
  Calls made by all processes on this machine:
    task_for_pid: 950
    thread_create: 0
    thread_set_state: 0

VM Region Summary:
ReadOnly portion of Libraries: Total=22.8M resident=0K(0%) swapped_out_or_unallocated=22.8M(100%)
Writable regions: Total=26.4M written=0K(0%) resident=0K(0%) swapped_out=0K(0%) unallocated=26.4M(100%)
 
                                VIRTUAL   REGION 
REGION TYPE                        SIZE    COUNT (non-coalesced) 
===========                     =======  ======= 
Kernel Alloc Once                    8K        2 
MALLOC                             9.9M        9 
MALLOC guard page                   16K        5 
__DATA                            4332K       61 
__LINKEDIT                        4716K       47 
__TEXT                            18.2M       47 
__UNICODE                          560K        2 
__UNIXSTACK                       16.0M        2 
shared memory                        8K        3 
===========                     =======  ======= 
TOTAL                             53.6M      169

----------
components: macOS
messages: 317361
nosy: ned.deily, ronaldoussoren, sm
priority: normal
severity: normal
status: open
title: Fatal Python error: Py_Initialize: unable to load the file system codec
type: crash
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33611>
_______________________________________

From report at bugs.python.org  Wed May 23 03:17:40 2018
From: report at bugs.python.org (Serhiy Storchaka)
Date: Wed, 23 May 2018 07:17:40 +0000
Subject: [New-bugs-announce] [issue33612] Assertion failure in
 PyThreadState_Clear
Message-ID: <1527059860.79.0.682650639539.issue33612@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:

I got the following output when run tests in the huntrleaks mode.

$ ./python -We -m test -R 3:3 -x test_builtin -x test_urlparse
...
1:24:54 load avg: 2.48 [225/414] test_multiprocessing_fork
beginning 6 repetitions
123456
..python: Python/pystate.c:589: PyThreadState_Clear: Assertion `tstate->exc_info->previous_item == NULL' failed.
Fatal Python error: Aborted

Current thread 0x00007fdd3e07e700 (most recent call first):
  File "/home/serhiy/py/cpython/Lib/multiprocessing/popen_fork.py", line 70 in _launch
  File "/home/serhiy/py/cpython/Lib/multiprocessing/popen_fork.py", line 20 in __init__
  File "/home/serhiy/py/cpython/Lib/multiprocessing/context.py", line 277 in _Popen
  File "/home/serhiy/py/cpython/Lib/multiprocessing/process.py", line 112 in start
  File "/home/serhiy/py/cpython/Lib/multiprocessing/pool.py", line 241 in _repopulate_pool
  File "/home/serhiy/py/cpython/Lib/multiprocessing/pool.py", line 248 in _maintain_pool
  File "/home/serhiy/py/cpython/Lib/multiprocessing/pool.py", line 412 in _handle_workers
  File "/home/serhiy/py/cpython/Lib/threading.py", line 865 in run
  File "/home/serhiy/py/cpython/Lib/threading.py", line 917 in _bootstrap_inner
  File "/home/serhiy/py/cpython/Lib/threading.py", line 885 in _bootstrap
..python: Python/pystate.c:589: PyThreadState_Clear: Assertion `tstate->exc_info->previous_item == NULL' failed.
Fatal Python error: Aborted

Current thread 0x00007fdd6cb6c700 (most recent call first):
  File "/home/serhiy/py/cpython/Lib/multiprocessing/popen_fork.py", line 70 in _launch
  File "/home/serhiy/py/cpython/Lib/multiprocessing/popen_fork.py", line 20 in __init__
  File "/home/serhiy/py/cpython/Lib/multiprocessing/context.py", line 277 in _Popen
  File "/home/serhiy/py/cpython/Lib/multiprocessing/process.py", line 112 in start
  File "/home/serhiy/py/cpython/Lib/multiprocessing/pool.py", line 241 in _repopulate_pool
  File "/home/serhiy/py/cpython/Lib/multiprocessing/pool.py", line 248 in _maintain_pool
  File "/home/serhiy/py/cpython/Lib/multiprocessing/pool.py", line 412 in _handle_workers
  File "/home/serhiy/py/cpython/Lib/threading.py", line 865 in run
  File "/home/serhiy/py/cpython/Lib/threading.py", line 917 in _bootstrap_inner
  File "/home/serhiy/py/cpython/Lib/threading.py", line 885 in _bootstrap
..
1:34:07 load avg: 1.01 [226/414] test_multiprocessing_forkserver -- test_multiprocessing_fork passed in 9 min 13 sec
...

It happens after 1.5 hours of running tests. Running the test_multiprocessing_fork test only doesn't expose it. Likely there are already opened related issues, but the assertion failure in PyThreadState_Clear is new, it was added in 3.7. It may help to identify the problem.

----------
messages: 317363
nosy: Mark.Shannon, davin, pitrou, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Assertion failure in PyThreadState_Clear
type: crash
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33612>
_______________________________________

From report at bugs.python.org  Wed May 23 06:04:09 2018
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 23 May 2018 10:04:09 +0000
Subject: [New-bugs-announce] [issue33613] test_multiprocessing_fork:
 test_semaphore_tracker_sigint() fails with -W error
Message-ID: <1527069849.43.0.682650639539.issue33613@psf.upfronthosting.co.za>


New submission from STINNER Victor <vstinner at redhat.com>:

test_semaphore_tracker_sigint() emits a warning. If the test is run with -W error, the test fails.

vstinner at apu$ ./python -m test test_multiprocessing_fork -v -m test_semaphore_tracker_sigint
(...)
test_semaphore_tracker_sigint (test.test_multiprocessing_fork.TestSemaphoreTracker) ... /home/vstinner/prog/python/master/Lib/multiprocessing/semaphore_tracker.py:55: UserWarning: semaphore_tracker: process died unexpectedly, relaunching.  Some semaphores might leak.
  warnings.warn('semaphore_tracker: process died unexpectedly, '
ok
(...)
Tests result: SUCCESS

vstinner at apu$ ./python  -Werror  -m test test_multiprocessing_fork -v -m test_semaphore_tracker_sigint
(...)
======================================================================
ERROR: test_semaphore_tracker_sigint (test.test_multiprocessing_fork.TestSemaphoreTracker)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/vstinner/prog/python/master/Lib/test/_test_multiprocessing.py", line 4533, in test_semaphore_tracker_sigint
    self.check_semaphore_tracker_death(signal.SIGINT, False)
  File "/home/vstinner/prog/python/master/Lib/test/_test_multiprocessing.py", line 4521, in check_semaphore_tracker_death
    sem = ctx.Semaphore()
  File "/home/vstinner/prog/python/master/Lib/multiprocessing/context.py", line 82, in Semaphore
    return Semaphore(value, ctx=self.get_context())
  File "/home/vstinner/prog/python/master/Lib/multiprocessing/synchronize.py", line 127, in __init__
    SemLock.__init__(self, SEMAPHORE, value, SEM_VALUE_MAX, ctx=ctx)
  File "/home/vstinner/prog/python/master/Lib/multiprocessing/synchronize.py", line 81, in __init__
    register(self._semlock.name)
  File "/home/vstinner/prog/python/master/Lib/multiprocessing/semaphore_tracker.py", line 83, in register
    self._send('REGISTER', name)
  File "/home/vstinner/prog/python/master/Lib/multiprocessing/semaphore_tracker.py", line 90, in _send
    self.ensure_running()
  File "/home/vstinner/prog/python/master/Lib/multiprocessing/semaphore_tracker.py", line 55, in ensure_running
    warnings.warn('semaphore_tracker: process died unexpectedly, '
UserWarning: semaphore_tracker: process died unexpectedly, relaunching.  Some semaphores might leak.
(...)
Tests result: FAILURE

----------
components: Tests
messages: 317372
nosy: davin, pitrou, vstinner
priority: normal
severity: normal
status: open
title: test_multiprocessing_fork: test_semaphore_tracker_sigint() fails with -W error
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33613>
_______________________________________

From report at bugs.python.org  Wed May 23 08:20:55 2018
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 23 May 2018 12:20:55 +0000
Subject: [New-bugs-announce] [issue33614] Compilation of Python fails on
 AMD64 Windows8.1 Refleaks 3.x
Message-ID: <1527078055.25.0.682650639539.issue33614@psf.upfronthosting.co.za>


New submission from STINNER Victor <vstinner at redhat.com>:

http://buildbot.python.org/all/#/builders/80/builds/244

       (Link target) -> 
         python3_d.def : error LNK2001: unresolved external symbol PyImport_GetModule [D:\buildarea\3.x.ware-win81-release.refleak\build\PCbuild\python3dll.vcxproj]
         python3_d.def : error LNK2001: unresolved external symbol Py_UTF8Mode [D:\buildarea\3.x.ware-win81-release.refleak\build\PCbuild\python3dll.vcxproj]
         D:\buildarea\3.x.ware-win81-release.refleak\build\PCbuild\amd64\python3_d.lib : fatal error LNK1120: 2 unresolved externals [D:\buildarea\3.x.ware-win81-release.refleak\build\PCbuild\python3dll.vcxproj]

Maybe it's related to this recent change:

commit 4e29f566e8821c09d8274eadcdd355e8b1284b8b
Author: Serhiy Storchaka <storchaka at gmail.com>
Date:   Tue May 22 20:59:42 2018 +0300

    Add missed details of the C API introduced in 3.7. (GH-7047)
    
    * Set the limited API version for PyImport_GetModule and PyOS_*Fork
      functions.
    * Add PyImport_GetModule and Py_UTF8Mode in PC/python3.def.
    * Add several functions in Doc/data/refcounts.dat.

The two missing symbols are defined by:

#if !defined(Py_LIMITED_API) || Py_LIMITED_API+0 >= 0x03070000
PyAPI_FUNC(PyObject *) PyImport_GetModule(PyObject *name);
#endif

and

#if !defined(Py_LIMITED_API) || Py_LIMITED_API+0 >= 0x03070000
PyAPI_DATA(int) Py_UTF8Mode;
#endif

They are not part of the stable ABI.

----------
messages: 317392
nosy: vstinner
priority: normal
severity: normal
status: open
title: Compilation of Python fails on AMD64 Windows8.1 Refleaks 3.x

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33614>
_______________________________________

From report at bugs.python.org  Wed May 23 08:25:16 2018
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 23 May 2018 12:25:16 +0000
Subject: [New-bugs-announce] [issue33615] test__xxsubinterpreters crashed on
 x86 Gentoo Refleaks 3.x
Message-ID: <1527078316.31.0.682650639539.issue33615@psf.upfronthosting.co.za>


New submission from STINNER Victor <vstinner at redhat.com>:

http://buildbot.python.org/all/#/builders/1/builds/232

(...)
3:28:16 load avg: 3.67 [401/416/3] test__xxsubinterpreters crashed (Exit code -6) -- running: test_asyncio (4631 sec)
python: Modules/gcmodule.c:277: visit_decref: Assertion `_PyGCHead_REFS(gc) != 0' failed.
Fatal Python error: Aborted

Current thread 0xb748a700 (most recent call first):
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/unittest/case.py", line 615 in run
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/unittest/case.py", line 663 in __call__
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/unittest/suite.py", line 122 in run
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/unittest/suite.py", line 84 in __call__
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/unittest/suite.py", line 122 in run
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/unittest/suite.py", line 84 in __call__
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/unittest/suite.py", line 122 in run
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/unittest/suite.py", line 84 in __call__
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/support/__init__.py", line 1781 in run
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/support/__init__.py", line 1882 in _run_suite
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/support/__init__.py", line 1972 in run_unittest
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/libregrtest/runtest.py", line 175 in test_runner
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/libregrtest/runtest.py", line 176 in runtest_inner
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/libregrtest/runtest.py", line 140 in runtest
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/libregrtest/runtest_mp.py", line 67 in run_tests_slave
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/libregrtest/main.py", line 517 in _main
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/libregrtest/main.py", line 510 in main
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/libregrtest/main.py", line 585 in main
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/regrtest.py", line 46 in _main
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/regrtest.py", line 50 in <module>
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/runpy.py", line 85 in _run_code
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/runpy.py", line 193 in _run_module_as_main
(...)
Re-running test 'test__xxsubinterpreters' in verbose mode
(...)
test_recv_not_found (test.test__xxsubinterpreters.ChannelTests) ... ok
test_run_string_arg_resolved (test.test__xxsubinterpreters.ChannelTests) ... python: Modules/gcmodule.c:277: visit_decref: Assertion `_PyGCHead_REFS(gc) != 0' failed.
Fatal Python error: Aborted
Current thread 0xb74b9700 (most recent call first):
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/unittest/case.py", line 615 in run
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/unittest/case.py", line 663 in __call__
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/unittest/suite.py", line 122 in run
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/unittest/suite.py", line 84 in __call__
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/unittest/suite.py", line 122 in run
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/unittest/suite.py", line 84 in __call__
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/unittest/suite.py", line 122 in run
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/unittest/suite.py", line 84 in __call__
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/unittest/runner.py", line 176 in run
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/support/__init__.py", line 1882 in _run_suite
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/support/__init__.py", line 1972 in run_unittest
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/libregrtest/runtest.py", line 175 in test_runner
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/libregrtest/runtest.py", line 176 in runtest_inner
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/libregrtest/runtest.py", line 140 in runtest
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/libregrtest/main.py", line 291 in rerun_failed_tests
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/libregrtest/main.py", line 540 in _main
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/libregrtest/main.py", line 510 in main
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/libregrtest/main.py", line 585 in main
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/__main__.py", line 2 in <module>
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/runpy.py", line 85 in _run_code
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/runpy.py", line 193 in _run_module_as_main

----------
components: Interpreter Core, Tests
messages: 317394
nosy: eric.snow, vstinner
priority: normal
severity: normal
status: open
title: test__xxsubinterpreters crashed on x86 Gentoo Refleaks 3.x
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33615>
_______________________________________

From report at bugs.python.org  Wed May 23 08:33:08 2018
From: report at bugs.python.org (Sebastian Rittau)
Date: Wed, 23 May 2018 12:33:08 +0000
Subject: [New-bugs-announce] [issue33616] typing.NoReturn is undocumented
Message-ID: <1527078788.95.0.682650639539.issue33616@psf.upfronthosting.co.za>


New submission from Sebastian Rittau <srittau at rittau.biz>:

This exists at least in Python 3.6.5's typing module. https://github.com/python/typing/issues/165 has background on why it was added.

----------
assignee: docs at python
components: Documentation
messages: 317395
nosy: docs at python, srittau
priority: normal
severity: normal
status: open
title: typing.NoReturn is undocumented
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33616>
_______________________________________

From report at bugs.python.org  Wed May 23 10:14:04 2018
From: report at bugs.python.org (Kyle Altendorf)
Date: Wed, 23 May 2018 14:14:04 +0000
Subject: [New-bugs-announce] [issue33617] subprocess.Popen etc do not accept
 os.PathLike in passed sequence on Windows
Message-ID: <1527084844.2.0.682650639539.issue33617@psf.upfronthosting.co.za>


New submission from Kyle Altendorf <sda at fstab.net>:

PS C:\Users\FSTAB\Desktop\g\20\grid-tied> py -3.6 -c "import os, pathlib, subprocess; subprocess.run([pathlib.Path()])"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Program Files\Python36\lib\subprocess.py", line 403, in run
    with Popen(*popenargs, **kwargs) as process:
  File "C:\Program Files\Python36\lib\subprocess.py", line 707, in __init__
    restore_signals, start_new_session)
  File "C:\Program Files\Python36\lib\subprocess.py", line 964, in _execute_child
    args = list2cmdline(args)
  File "C:\Program Files\Python36\lib\subprocess.py", line 461, in list2cmdline
    needquote = (" " in arg) or ("\t" in arg) or not arg
TypeError: argument of type 'WindowsPath' is not iterable


PR to follow soon.

----------
messages: 317408
nosy: altendky
priority: normal
severity: normal
status: open
title: subprocess.Popen etc do not accept os.PathLike in passed sequence on Windows

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33617>
_______________________________________

From report at bugs.python.org  Wed May 23 10:38:43 2018
From: report at bugs.python.org (Christian Heimes)
Date: Wed, 23 May 2018 14:38:43 +0000
Subject: [New-bugs-announce] [issue33618] Support TLS 1.3
Message-ID: <1527086323.78.0.682650639539.issue33618@psf.upfronthosting.co.za>


New submission from Christian Heimes <lists at cheimes.de>:

Epic for various TLS 1.3 related tasks and improvements

TLS 1.3 requires some changes to the SSL module and tests. The TLS 1.3 handshake behaves slightly differently, which causes some tests to fail. The new handshake and deferred non-application data also affect applications. 

* TLS 1.3 cipher suites are now set by SSL_CTX_set_ciphersuites(), while SSL to TLS 1.2 cipher suites are still set by SSL_CTX_set_cipher_list(). Therefore SSLContext.set_ciphers() no longer fails with invalid cipher suites, because TLS 1.3 are still available. TLS 1.3 cipher suites also cannot be changed or disabled by SSLContext.set_ciphers().

* TLS client cert authentication occurs after SSL_do_handshake() has finished. SSLSocket.connect() / handshake no longer fail, when the server requests a client cert or the available client cert is invalid. The actual authentication occurs when the client performs the first SSL_read() / SSL_write().

* Session tickets are exchanged after the handshake, too. On the client side, the session ticket is only available after the first SSL_read() or other operations that perform a read(). The session ticket class and code no longer works with TLS 1.3.

* TLS 1.3 sends two session tickets instead of one.

* Server-side handshake can fail with ConnectionResetError or BrokenPipeError, when the client closes the fd while the server is still send non-application data like new session ticket or client cert request.

* Client-side unwrap() / shutdown used to fail when a session ticket was stuck on the wire. This problem will be fixed by OpenSSL 1.1.1-pre7, see https://github.com/openssl/openssl/pull/6340 


I'll add a TLS 1.3 section to the ssl module documentation. TLS 1.3 will be a tech-preview and not production-ready until at least OpenSSL 1.1.1-final and Python 3.7.1. Ned, Benjamin, are you OK with that?

----------
assignee: christian.heimes
messages: 317413
nosy: alex, benjamin.peterson, christian.heimes, dstufft, janssen, ned.deily
priority: high
severity: normal
stage: needs patch
status: open
title: Support TLS 1.3
type: enhancement
versions: Python 2.7, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33618>
_______________________________________

From report at bugs.python.org  Wed May 23 12:34:24 2018
From: report at bugs.python.org (Chih-Hsuan Yen)
Date: Wed, 23 May 2018 16:34:24 +0000
Subject: [New-bugs-announce] [issue33619] libffi detection via pkg-config is
 broken
Message-ID: <1527093264.92.0.682650639539.issue33619@psf.upfronthosting.co.za>


New submission from Chih-Hsuan Yen <yan16037741 at hotmail.com.tw>:

The cause is that ./configure contains unexpanded m4 macro PKG_PROG_PKG_CONFIG:

https://github.com/python/cpython/blob/3055c94/configure#L9909

For unknown reasons, after GH-6850 the PKG_PROG_PKG_CONFIG macro is removed from aclocal.m4, and thus the macro is kept unexpanded in ./configure. As a result, $PKG_CONFIG is not correctly set, libffi can't be detected and thus _ctypes fails to build.

Run autoreconf again fixes the issue on my machine. I have pkg-config 0.29.2-1, autoconf 2.69-4 and automake 1.15.1-1 on Arch Linux.

cc the author of GH-6850

----------
components: Build
messages: 317424
nosy: benjamin.peterson, yan12125
priority: normal
severity: normal
status: open
title: libffi detection via pkg-config is broken
type: compile error
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33619>
_______________________________________

From report at bugs.python.org  Wed May 23 12:46:21 2018
From: report at bugs.python.org (Jonathan Lynch)
Date: Wed, 23 May 2018 16:46:21 +0000
Subject: [New-bugs-announce] [issue33620] requests.Session doesn't properly
 handle closed keep-alive sessions
Message-ID: <1527093981.46.0.682650639539.issue33620@psf.upfronthosting.co.za>


New submission from Jonathan Lynch <jlynch at tenable.com>:

When a server reaps a keep-alive session it sends a FIN packet to the client. Normally, requests handles this fine and rebuilds the session on the next request. However, there is an edge case involving network latency that is not properly handled:

If python sends a request at roughly the same time as the server closes the session, then the server will send a RST (as the session is closed). Python receives this RST on what it thought was a valid session and throws an error:

requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response',))

The reason I consider this a bug is because python received the FIN packet before it received the RST. As a result, it shouldn't be surprised when the connection is subsequently aborted. It is an edge case, but the client has enough information available to it that it could have handled it correctly.

The workaround is to set max_retries on the Session via an HTTPAdaptor, but I believe the correct behavior when the FIN is received is to rebuild the session and re-send any requests that were in-flight (rather than throwing an error). Requests correctly handles the FIN packet if there are no in-flight requests, but if there are in-flight requests it ignores it and instead throws an error.

----------
components: Library (Lib)
messages: 317425
nosy: Jonathan Lynch
priority: normal
severity: normal
status: open
title: requests.Session doesn't properly handle closed keep-alive sessions
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33620>
_______________________________________

From report at bugs.python.org  Wed May 23 13:39:42 2018
From: report at bugs.python.org (Fabio Zadrozny)
Date: Wed, 23 May 2018 17:39:42 +0000
Subject: [New-bugs-announce] [issue33621] repr(threading._DummyThread)
 always fails.
Message-ID: <1527097182.41.0.682650639539.issue33621@psf.upfronthosting.co.za>


New submission from Fabio Zadrozny <fabioz at users.sourceforge.net>:

Doing the following throws an exception:

import threading
repr(threading._DummyThread())


Or, in a more contrived example (I actually had this in a QThread, so, reproducing using getting the current_thread using a thread created with the _thread module):

import threading
import traceback
finished = threading.Event()
worked = []
def method():
    try:
        repr(threading.current_thread())
        worked.append(True)
    except:
        traceback.print_exc()
        worked.append(False)
    finally:
        finished.set()
    
import _thread
_thread.start_new_thread(method, ())

finished.wait()
assert worked[0]

----------
components: Library (Lib)
messages: 317430
nosy: fabioz
priority: normal
severity: normal
status: open
title: repr(threading._DummyThread) always fails.
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33621>
_______________________________________

From report at bugs.python.org  Wed May 23 13:41:20 2018
From: report at bugs.python.org (Serhiy Storchaka)
Date: Wed, 23 May 2018 17:41:20 +0000
Subject: [New-bugs-announce] [issue33622] Fix and improve errors handling in
 the garbage collector
Message-ID: <1527097280.42.0.682650639539.issue33622@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:

There are following bugs in the garbage collector.

* If the garbage collector fails to add an object with __del__ or referenced by an object with __del__ to gc.garbage (in handle_legacy_finalizers()), it leaks it and other not added objects with __del__ and referenced by them. They become no longer accessible by the garbage collector.

* PyGC_Collect() is not documented, but it is a public C API. And it can be called by user with an exception set. PyGC_Collect() then can either crash or just silent the exception. It is safer to safe possible exception and restore it after collecting.

* A pointer to released member can be used (compared with NULL) in invoke_gc_callback(). This is an undefined behavior.

----------
components: Interpreter Core
messages: 317431
nosy: pitrou, serhiy.storchaka, vstinner
priority: normal
severity: normal
status: open
title: Fix and improve errors handling in the garbage collector
type: behavior
versions: Python 2.7, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33622>
_______________________________________

From report at bugs.python.org  Wed May 23 14:55:46 2018
From: report at bugs.python.org (Yury Selivanov)
Date: Wed, 23 May 2018 18:55:46 +0000
Subject: [New-bugs-announce] [issue33623] Fix possible SIGSGV when
 asyncio.Future is created in __del__
Message-ID: <1527101746.49.0.682650639539.issue33623@psf.upfronthosting.co.za>


New submission from Yury Selivanov <yselivanov at gmail.com>:

Originally reported in https://github.com/MagicStack/uvloop/issues/143

Future.__init__ shouldn't try to capture the current traceback if the interpreter is being finalized.

----------
messages: 317437
nosy: yselivanov
priority: normal
severity: normal
status: open
title: Fix possible SIGSGV when asyncio.Future is created in __del__
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33623>
_______________________________________

From report at bugs.python.org  Wed May 23 15:18:18 2018
From: report at bugs.python.org (Andrew Svetlov)
Date: Wed, 23 May 2018 19:18:18 +0000
Subject: [New-bugs-announce] [issue33624] Implement subclass hooks for
 asyncio abstract classes
Message-ID: <1527103098.65.0.682650639539.issue33624@psf.upfronthosting.co.za>


New submission from Andrew Svetlov <andrew.svetlov at gmail.com>:

To make clean inheritance hierarchy of objects implemented by third-party loop implementations.
Now is impossible to implement AbstractServer in C or Cython.

See also https://github.com/MagicStack/uvloop/issues/131

----------
components: asyncio
messages: 317438
nosy: asvetlov, yselivanov
priority: normal
severity: normal
status: open
title: Implement subclass hooks for asyncio abstract classes
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33624>
_______________________________________

From report at bugs.python.org  Wed May 23 16:03:01 2018
From: report at bugs.python.org (William Grzybowski)
Date: Wed, 23 May 2018 20:03:01 +0000
Subject: [New-bugs-announce] [issue33625] Disable GIL on getpwnam and
 getpwuid
Message-ID: <1527105781.7.0.682650639539.issue33625@psf.upfronthosting.co.za>


New submission from William Grzybowski <william88 at gmail.com>:

Hello,

Currently the GIL is not disabled when calling pwd.getpwnam nor pwd.getpwuid.

It could be the C library call may take some time for completion, especially when using third-party modules on the system (nss-ldap, nss-pgsql, sss, etc).

Disabling GIL before calling them makes sure other threads can run in the meantime.

----------
components: Extension Modules
messages: 317445
nosy: wg
priority: normal
severity: normal
status: open
title: Disable GIL on getpwnam and getpwuid
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33625>
_______________________________________

From report at bugs.python.org  Wed May 23 19:23:53 2018
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 23 May 2018 23:23:53 +0000
Subject: [New-bugs-announce] [issue33626]
 test_sendfile_fallback_close_peer_in_middle_of_receiving() fails on x86-64
 Sierra 3.x
Message-ID: <1527117833.42.0.682650639539.issue33626@psf.upfronthosting.co.za>


New submission from STINNER Victor <vstinner at redhat.com>:

http://buildbot.python.org/all/#/builders/14/builds/1023

======================================================================
FAIL: test_sendfile_fallback_close_peer_in_middle_of_receiving (test.test_asyncio.test_events.KqueueEventLoopTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/buildbot/buildarea/3.x.billenstein-sierra/build/Lib/test/test_asyncio/test_events.py", line 2474, in test_sendfile_fallback_close_peer_in_middle_of_receiving
    self.loop.sendfile(cli_proto.transport, self.file))
AssertionError: ConnectionError not raised

======================================================================
FAIL: test_sendfile_fallback_close_peer_in_middle_of_receiving (test.test_asyncio.test_events.PollEventLoopTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/buildbot/buildarea/3.x.billenstein-sierra/build/Lib/test/test_asyncio/test_events.py", line 2474, in test_sendfile_fallback_close_peer_in_middle_of_receiving
    self.loop.sendfile(cli_proto.transport, self.file))
AssertionError: ConnectionError not raised

======================================================================
FAIL: test_sendfile_fallback_close_peer_in_middle_of_receiving (test.test_asyncio.test_events.SelectEventLoopTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/buildbot/buildarea/3.x.billenstein-sierra/build/Lib/test/test_asyncio/test_events.py", line 2474, in test_sendfile_fallback_close_peer_in_middle_of_receiving
    self.loop.sendfile(cli_proto.transport, self.file))
AssertionError: ConnectionError not raised

----------
components: Tests, asyncio
messages: 317472
nosy: asvetlov, vstinner, yselivanov
priority: normal
severity: normal
status: open
title: test_sendfile_fallback_close_peer_in_middle_of_receiving() fails on x86-64 Sierra 3.x
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33626>
_______________________________________

From report at bugs.python.org  Wed May 23 19:30:49 2018
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 23 May 2018 23:30:49 +0000
Subject: [New-bugs-announce] [issue33627] test_numeric_tower.test_complex()
 crashed on
Message-ID: <1527118249.12.0.682650639539.issue33627@psf.upfronthosting.co.za>


New submission from STINNER Victor <vstinner at redhat.com>:

http://buildbot.python.org/all/#/builders/88/builds/1044

0:07:59 load avg: 1.58 [107/416/1] test_numeric_tower crashed (Exit code -11)
Fatal Python error: Segmentation fault

Current thread 0x405c4dc0 (most recent call first):
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/test_numeric_tower.py", line 184 in test_complex
(...)
Re-running test 'test_numeric_tower' in verbose mode
test_complex (test.test_numeric_tower.ComparisonTest) ... ok
test_mixed_comparisons (test.test_numeric_tower.ComparisonTest) ... ok
test_binary_floats (test.test_numeric_tower.HashTest) ... ok
test_bools (test.test_numeric_tower.HashTest) ... ok
test_complex (test.test_numeric_tower.HashTest) ... ok
test_decimals (test.test_numeric_tower.HashTest) ... ok
test_fractions (test.test_numeric_tower.HashTest) ... ok
test_hash_normalization (test.test_numeric_tower.HashTest) ... ok
test_integers (test.test_numeric_tower.HashTest) ... ok

----------------------------------------------------------------------
Ran 9 tests in 0.573s

OK

----------
messages: 317474
nosy: vstinner
priority: normal
severity: normal
status: open
title: test_numeric_tower.test_complex() crashed on

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33627>
_______________________________________

From report at bugs.python.org  Wed May 23 19:48:33 2018
From: report at bugs.python.org (Cheryl Sabella)
Date: Wed, 23 May 2018 23:48:33 +0000
Subject: [New-bugs-announce] [issue33628] IDLE: Code cleanup on codecontext
Message-ID: <1527119313.11.0.682650639539.issue33628@psf.upfronthosting.co.za>


New submission from Cheryl Sabella <cheryl.sabella at gmail.com>:

Based on the github comments on PR5638, some code cleanup issues identified for codecontext.

codecontext
* getspacesfirstword - function and param1 name
* get_line_info - make into helper function and change parm
* get_context - call to get_line_info sends text instead of line number
* toggle_code_context_event - move border calc to same loop at padx
* cleanup some comments

test_codecontext
* test_get_line_info - move test to helper test class and change parm from line number to line
* test_get_line_info - add test for tab (I noticed 'indent' is 1 when the regex matches to a tab.  Even though IDLE doesn't allow tabs in the editor, I didn't want to lose track of this).
* test_getspacesfirstword - function name
* test_del - remove call to after info

----------
assignee: terry.reedy
components: IDLE
messages: 317479
nosy: cheryl.sabella, terry.reedy
priority: normal
severity: normal
status: open
title: IDLE: Code cleanup on codecontext
type: enhancement
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33628>
_______________________________________

From report at bugs.python.org  Wed May 23 21:38:15 2018
From: report at bugs.python.org (STINNER Victor)
Date: Thu, 24 May 2018 01:38:15 +0000
Subject: [New-bugs-announce] [issue33629] test_importlib creates a coredump
 on AMD64 FreeBSD 10.x Shared 3.7
Message-ID: <1527125895.06.0.682650639539.issue33629@psf.upfronthosting.co.za>


New submission from STINNER Victor <vstinner at redhat.com>:

http://buildbot.python.org/all/#/builders/124/builds/315

Warning -- files was modified by test_importlib
  Before: []
  After:  ['python.core']

----------
components: FreeBSD, Tests
messages: 317497
nosy: koobs, vstinner
priority: normal
severity: normal
status: open
title: test_importlib creates a coredump on AMD64 FreeBSD 10.x Shared 3.7
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33629>
_______________________________________

From report at bugs.python.org  Wed May 23 22:18:53 2018
From: report at bugs.python.org (STINNER Victor)
Date: Thu, 24 May 2018 02:18:53 +0000
Subject: [New-bugs-announce] [issue33630] test_posix: TestPosixSpawn fails
 on PPC64 Fedora 3.x
Message-ID: <1527128333.92.0.682650639539.issue33630@psf.upfronthosting.co.za>


New submission from STINNER Victor <vstinner at redhat.com>:

http://buildbot.python.org/all/#/builders/53/builds/1062

(...)
test_close_file (test.test_posix.TestPosixSpawn) ... ok
test_dup2 (test.test_posix.TestPosixSpawn) ... ok
test_empty_file_actions (test.test_posix.TestPosixSpawn) ... ok
test_multiple_file_actions (test.test_posix.TestPosixSpawn) ... FAIL
test_no_such_executable (test.test_posix.TestPosixSpawn) ... ok
test_open_file (test.test_posix.TestPosixSpawn) ... ERROR
test_returns_pid (test.test_posix.TestPosixSpawn) ... ok
Warning -- files was modified by test_posix
  Before: []
  After:  ['\udcdb\udcdb\udcdb\udcdb\udcdb\udcdb\udcdb\udcdb\udcdb\udcdb\udcdb\udcdb\udcdb\udcdb\udcdb\udcdb\udcdb\udcdb\udcdb\udcdb\udcdb\udcdb\udcdb\udcdb\udcdb\udcdb\udcdb\udcdb\udcdb\udcdb\udcdb\udcdb'] 
test test_posix failed
test_specify_environment (test.test_posix.TestPosixSpawn) ... ok

======================================================================
ERROR: test_open_file (test.test_posix.TestPosixSpawn)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/shager/cpython-buildarea/3.x.edelsohn-fedora-ppc64/build/Lib/test/test_posix.py", line 1542, in test_open_file
    with open(outfile) as f:
FileNotFoundError: [Errno 2] No such file or directory: '@test_42482_tmp'

======================================================================
FAIL: test_multiple_file_actions (test.test_posix.TestPosixSpawn)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/shager/cpython-buildarea/3.x.edelsohn-fedora-ppc64/build/Lib/test/test_posix.py", line 1488, in test_multiple_file_actions
    self.assertEqual(os.waitpid(pid, 0), (pid, 0))
AssertionError: Tuples differ: (31319, 32512) != (31319, 0)

First differing element 1:
32512
0

- (31319, 32512)
?         ^^^^^

+ (31319, 0)
?         ^


----------------------------------------------------------------------
Ran 101 tests in 2.175s

FAILED (failures=1, errors=1, skipped=9)
1 test failed again:
    test_posix

----------
components: Tests
messages: 317509
nosy: vstinner
priority: normal
severity: normal
status: open
title: test_posix: TestPosixSpawn fails on PPC64 Fedora 3.x
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33630>
_______________________________________

From report at bugs.python.org  Thu May 24 03:29:39 2018
From: report at bugs.python.org (CatPaw Freed)
Date: Thu, 24 May 2018 07:29:39 +0000
Subject: [New-bugs-announce] [issue33631] [ValueError] _strptime.py can't
 handle 12-hr format strings using '0' instead of '12' for noon and midnight
Message-ID: <1527146979.67.0.682650639539.issue33631@psf.upfronthosting.co.za>


New submission from CatPaw Freed <prawarit888 at gmail.com>:

I have to process some data which date was specified in 12-hour format, eg. 9/24/2017 0:39:41 PM.
I call datetime.datetime.strptime with "%m/%d/%Y %I:%M:%S %p" format.
However, it throws me an error [File "C:\Python27\lib\_strptime.py", line 332, in _strptime (data_string, format)) ValueError: time data '9/24/2017 0:39:41 PM' does not match format '%m/%d/%Y %I:%M:%S %p'].
Then, I found out that, in _strptime.py, the regex patterns are defined as below.

            'd': r"(?P<d>3[0-1]|[1-2]\d|0[1-9]|[1-9]| [1-9])",
            'f': r"(?P<f>[0-9]{1,6})",
            'H': r"(?P<H>2[0-3]|[0-1]\d|\d)",
            'I': r"(?P<I>1[0-2]|0[1-9]|[1-9])",
            'j': r"(?P<j>36[0-6]|3[0-5]\d|[1-2]\d\d|0[1-9]\d|00[1-9]|[1-9]\d|0[1-9]|[1-9])",
            'm': r"(?P<m>1[0-2]|0[1-9]|[1-9])",
            'M': r"(?P<M>[0-5]\d|\d)",
            'S': r"(?P<S>6[0-1]|[0-5]\d|\d)",
            'U': r"(?P<U>5[0-3]|[0-4]\d|\d)",
            'w': r"(?P<w>[0-6])",

I'm a newbie in python world. Is it a bug?
I changed regex pattern of 'I' to r"(?P<I>1[0-2]|0[0-9]|[0-9])" and it works.
I would like to know why 0 is excluded.
May be, I have missed something.

----------
messages: 317541
nosy: CatPaw Freed
priority: normal
severity: normal
status: open
title: [ValueError] _strptime.py can't handle 12-hr format strings using '0' instead of '12' for noon and midnight
type: crash

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33631>
_______________________________________

From report at bugs.python.org  Thu May 24 04:39:55 2018
From: report at bugs.python.org (Antoine Pitrou)
Date: Thu, 24 May 2018 08:39:55 +0000
Subject: [New-bugs-announce] [issue33632] undefined behaviour: signed
 integer overflow in threadmodule.c
Message-ID: <1527151195.18.0.682650639539.issue33632@psf.upfronthosting.co.za>


New submission from Antoine Pitrou <pitrou at free.fr>:

Modules/_threadmodule.c:52:47: runtime error: signed integer overflow: 2387971499048 + 9223372036000000000 cannot be represented in type 'long'

----------
components: Library (Lib)
messages: 317545
nosy: pitrou
priority: normal
severity: normal
status: open
title: undefined behaviour: signed integer overflow in threadmodule.c
type: behavior
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33632>
_______________________________________

From report at bugs.python.org  Thu May 24 06:50:33 2018
From: report at bugs.python.org (Xavier Bonaventura)
Date: Thu, 24 May 2018 10:50:33 +0000
Subject: [New-bugs-announce] [issue33633] smtplib msg['To] = appends instead
 of assigning
Message-ID: <1527159033.13.0.682650639539.issue33633@psf.upfronthosting.co.za>


New submission from Xavier Bonaventura <xavibonaventura at gmail.com>:

The behavior when you assign an email to 'To' is counter intuitive.
When you do:

msg['To'] = 'foo at mail.com'

this appends the email instead of really assigning it.
This is because the assignment operator is overwritten.

Imagine that you have code like this:

msg = MIMEText("The report at *link* has been updated")
for recipient in recipient_list:
    msg['To'] = recipient_email
    server.sendmail(from_address, recipient_email, msg.as_string())

This will send the email to the first person N times, to the second N-1, etc.

In case that you want to debug, it is also a problem. In case that the append is the expected behaviour, one would expect that at least if you do a print like this you see the full list.

print(msg['To'])

Instead of that, in this example of code:
msg['To'] = 'foo at mail.com'
msg['To'] = 'foo at mail.com'
print(msg['To'])

It will print:
foo at mail.com

But the message will be sent two times.

----------
components: Library (Lib)
messages: 317550
nosy: Xavier Bonaventura
priority: normal
severity: normal
status: open
title: smtplib msg['To] = appends instead of assigning
type: behavior
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33633>
_______________________________________

From report at bugs.python.org  Thu May 24 07:31:44 2018
From: report at bugs.python.org (STINNER Victor)
Date: Thu, 24 May 2018 11:31:44 +0000
Subject: [New-bugs-announce] [issue33634] Buildbot configuration issue on
 Windows7 buildbots
Message-ID: <1527161504.95.0.682650639539.issue33634@psf.upfronthosting.co.za>


New submission from STINNER Victor <vstinner at redhat.com>:

http://buildbot.python.org/all/#/builders/90/builds/342

D:\cygwin\home\db3l\buildarea\3.6.bolen-windows7\build>"D:\cygwin\home\db3l\buildarea\3.6.bolen-windows7\build\PCbuild\win32\python_d.exe"  -u -Wd -E -bb -m test  -uall -rwW --slowest --timeout 1200 -j1 -j2 -uall -cpu -largefile --timeout 900 
usage: python -m test [options] [test_name1 [test_name2 ...]]
python path/to/Lib/test/regrtest.py [options] [test_name1 [test_name2 ...]]
regrtest.py: error: argument -l/--findleaks: ignored explicit argument 'argefile'
Pass -h or --help for complete help.

----------
components: Tests
messages: 317558
nosy: ned.deily, vstinner, zach.ware
priority: normal
severity: normal
status: open
title: Buildbot configuration issue on Windows7 buildbots
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33634>
_______________________________________

From report at bugs.python.org  Thu May 24 08:15:27 2018
From: report at bugs.python.org (Victor Domingos)
Date: Thu, 24 May 2018 12:15:27 +0000
Subject: [New-bugs-announce] [issue33635] OSError when using
 pathlib.Path.rglob() to list device files
Message-ID: <1527164127.59.0.682650639539.issue33635@psf.upfronthosting.co.za>


New submission from Victor Domingos <editor.arcosonline at gmail.com>:

This method fails with an error when it finds a character or block device (like those found in /dev on macOS). However, in the same machine, with the same Python version, the alternative os.walk() performs basically the same job with no errors. I am not sure if that error is the expected behaviour, but I wasn't able to find a clear explanation in the docs. Anyway, it seems to me, as a user, that the os.walk() error-less behaviour is more desirable.

```
$ python3
Python 3.7.0b4 (v3.7.0b4:eb96c37699, May  2 2018, 04:13:13) 
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> from pathlib import Path
>>> [f for f in Path(os.path.expanduser('/dev')).rglob("*") if f.is_file()]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 1, in <listcomp>
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/pathlib.py", line 1344, in is_file
    return S_ISREG(self.stat().st_mode)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/pathlib.py", line 1140, in stat
    return self._accessor.stat(self)
OSError: [Errno 9] Bad file descriptor: '/dev/fd/3'
>>> 
```

----------
components: Library (Lib)
messages: 317566
nosy: Victor Domingos
priority: normal
severity: normal
status: open
title: OSError when using pathlib.Path.rglob() to list device files
type: crash
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33635>
_______________________________________

From report at bugs.python.org  Thu May 24 10:12:24 2018
From: report at bugs.python.org (nathan rogers)
Date: Thu, 24 May 2018 14:12:24 +0000
Subject: [New-bugs-announce] [issue33636] Unexpected behavior with * and
 arrays
Message-ID: <1527171144.64.0.682650639539.issue33636@psf.upfronthosting.co.za>


New submission from nathan rogers <nathan.rogers.d at gmail.com>:

https://repl.it/repls/ColorfulFlusteredPercent

Here you can see the unexpected behavior I was speaking of. This behavior is NOT useful compared to the expected behavior. If I reference position 0 in the array, I expect position 0 to be appended. The sensible behavior, from my view, would be to make n unique values, not n duplicates.

----------
messages: 317572
nosy: nanthil
priority: normal
severity: normal
status: open
title: Unexpected behavior with * and arrays
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33636>
_______________________________________

From report at bugs.python.org  Thu May 24 13:10:47 2018
From: report at bugs.python.org (Ivan Pozdeev)
Date: Thu, 24 May 2018 17:10:47 +0000
Subject: [New-bugs-announce] [issue33637] pip cannot build extensions for
 debug Python
Message-ID: <1527181847.69.0.682650639539.issue33637@psf.upfronthosting.co.za>


New submission from Ivan Pozdeev <ivan_pozdeev at mail.ru>:

(the output is for 2.7 head; for master, it's the same save for paths, library versions and DeprecationWarning's)

>python.bat -m ensurepip
Running Debug|x64 interpreter...
Looking in links: c:\users\sasha\appdata\local\temp\tmpiysoby
Collecting setuptools
Collecting pip
Installing collected packages: setuptools, pip
Successfully installed pip-10.0.1 setuptools-39.0.1
>python.bat -m pip install scandir
Running Debug|x64 interpreter...
Collecting scandir
  Using cached https://files.pythonhosted.org/packages/13/bb/e541b74230bbf7a20a3949a2ee6631be299378a784f5445aa5d0047c192b/scandir-1.7.tar.gz
Installing collected packages: scandir
  Running setup.py install for scandir ... error
    Complete output from command C:\Users\Sasha\Documents\cpython\PCBuild\amd64\python_d.exe -u -c "import setuptools, tokenize;__file__='c:\\users\\sasha\\appdata\\local\\temp\\pip-install-xafnbn\\scandir\\setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record c:\users\sasha\appdata\local\temp\pip-record-gbe0kk\install-record.txt --single-version-externally-managed --compile:
    running install
    running build
    running build_py
    creating build
    creating build\lib.win-amd64-2.7-pydebug
    copying scandir.py -> build\lib.win-amd64-2.7-pydebug
    running build_ext
    building '_scandir' extension
    creating build\temp.win-amd64-2.7-pydebug
    creating build\temp.win-amd64-2.7-pydebug\Release
    c:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\BIN\amd64\cl.exe /c /nologo /Ox /MD /W3 /GS- /DNDEBUG -IC:\Users\Sasha\Documents\cpython\include -IC:\Users\Sasha\Documents\cpython\PC /Tc_scandir.c /Fobuild\temp.win-amd64-2.7-pydebug\Release\_scandir.obj
    _scandir.c
    c:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\BIN\amd64\link.exe /DLL /nologo /INCREMENTAL:NO /LIBPATH:C:\Users\Sasha\Documents\cpython\libs /LIBPATH:C:\Users\Sasha\Documents\cpython\PCbuild\amd64 /LIBPATH:C:\Users\Sasha\Documents\cpython\PC\VS9.0\amd64 /EXPORT:init_scandir build\temp.win-amd64-2.7-pydebug\Release\_scandir.obj /OUT:build\lib.win-amd64-2.7-pydebug\_scandir.pyd /IMPLIB:build\temp.win-amd64-2.7-pydebug\Release\_scandir.lib /MANIFESTFILE:build\temp.win-amd64-2.7-pydebug\Release\_scandir.pyd.manifest
    LINK : fatal error LNK1104: cannot open file 'python27.lib'
    error: command 'c:\\Program Files (x86)\\Microsoft Visual Studio 9.0\\VC\\BIN\\amd64\\link.exe' failed with exit status 1104

    ----------------------------------------
Command "C:\Users\Sasha\Documents\cpython\PCBuild\amd64\python_d.exe -u -c "import setuptools, tokenize;__file__='c:\\users\\sasha\\appdata\\local\\temp\\pip-install-xafnbn\\scandir\\setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record c:\users\sasha\appdata\local\temp\pip-record-gbe0kk\install-record.txt --single-version-externally-managed --compile" failed with error code
1 in c:\users\sasha\appdata\local\temp\pip-install-xafnbn\scandir\



The cause is that distutils' `build_ext` looks at the --debug switch instead of the running Python to decide which Python runtime to link against.

A workaround is thus to run setup.py by hand with the --debug switch.


Suggested fix is to split compiler options into optimization options and linking options and:

* for debug Python, build without optimizations and against debug C runtime + Python runtime regardless of --debug
* for release Python,
    * build without optimizations but against release C runtime + Python runtime with --debug


Rationale:

* It doesn't make sense to try to build against a different runtime than the running Python -- this will always fail because include/lib paths, preprocessor directives and such that build_ext generates is for the running Python.
* Must build against the same C runtime that Python uses in any case since using two different runtimes will break things, especially in 2.7 that relies on stdio.
* Building an extension without optimizations will allow to diagnose problems in it. Debug facilities of the runtime like malloc tracking and probably tracing will not work, but at least one will see all the variables and call hierarchy.

----------
components: Distutils
messages: 317589
nosy: Ivan.Pozdeev, dstufft, eric.araujo
priority: normal
severity: normal
status: open
title: pip cannot build extensions for debug Python
type: behavior
versions: Python 2.7, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33637>
_______________________________________

From report at bugs.python.org  Thu May 24 15:10:23 2018
From: report at bugs.python.org (christof)
Date: Thu, 24 May 2018 19:10:23 +0000
Subject: [New-bugs-announce] [issue33638] condition lock not re-acquired
Message-ID: <1527189023.9.0.682650639539.issue33638@psf.upfronthosting.co.za>


New submission from christof <chbailly at gmail.com>:

Hello,

I have a simple code which triggers a timeout if a task did not complete

import asyncio

async def task_timeout():
    condition = asyncio.Condition()
    with await condition:
        try:
            await asyncio.wait_for(condition.wait(), timeout=4)
        except asyncio.TimeoutError as e:
 
            print("timeout reached")
            # uncomment this line to make the code work
            # await asyncio.sleep(0)


f = asyncio.ensure_future(task_timeout())

loop= asyncio.get_event_loop()
loop.run_until_complete(f)

It throws an exception when leaving the scope for the condition because it expects the lock to be acquired:

RuntimeError: Lock is not acquired.

If you uncomment the line to sleep, it will work because it will continue in the coroutine Condition.wait and call:
 yield from self.acquire() => locks.py line 355

I think this is a bug and that this behaviour is not the expected one.

----------
components: asyncio
messages: 317604
nosy: asvetlov, christof, yselivanov
priority: normal
severity: normal
status: open
title: condition lock not re-acquired
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33638>
_______________________________________

From report at bugs.python.org  Thu May 24 16:15:29 2018
From: report at bugs.python.org (Giampaolo Rodola')
Date: Thu, 24 May 2018 20:15:29 +0000
Subject: [New-bugs-announce] [issue33639] Use high-performance os.sendfile()
 in shutil.copy*
Message-ID: <1527192929.89.0.682650639539.issue33639@psf.upfronthosting.co.za>


New submission from Giampaolo Rodola' <g.rodola at gmail.com>:

This is a follow up of #25063 and similar to socket.sendfile() (#17552). It provides a 20/25% speedup when copying files with shutil.copyfile(), shutil.copy() and shutil.copy2(). Differently from #25063 this is used for filesystem files only and copyfileobj() is left alone.

Unmerged #26826 is also related to this. I applied #26826 patch and built a wrapper around copy_file_range() and the speedup is basically the same. Nevertheless, even when #26826 gets merged it probably makes sense to rely on sendfile() in case copy_file_range() is not available (it was introduced in 2016) or the UNIX platform supports file-to-file copy via sendfile(2) (even though I'm not aware of any, so this would basically be Linux-only).

Some benchmarks:

    $ dd if=/dev/urandom of=/tmp/f1 bs=1K count=128
    $ time ./python -m timeit -s 'import shutil; p1 = "/tmp/f1"; p2 = "/tmp/f2"' 'shutil.copyfile(p1, p2)'

128K copy
=========

--- without patch:

    2000 loops, best of 5: 160 usec per loop

    real    0m2.353s
    user    0m0.454s
    sys     0m1.435s

--- with patch:

    2000 loops, best of 5: 187 usec per loop

    real    0m2.724s
    user    0m0.627s
    sys     0m1.634s

8MB copy
========

$ dd if=/dev/urandom of=/tmp/f1 bs=1M count=8

--- without patch:

    50 loops, best of 5: 9.51 msec per loop

    real    0m3.392s
    user    0m0.343s
    sys     0m2.478s

--- with patch:

    50 loops, best of 5: 7.75 msec per loop

    real    0m2.878s
    user    0m0.105s
    sys     0m2.187s

512MB copy
==========

--- without patch:

    1 loop, best of 5: 872 msec per loop

    real    0m5.574s
    user    0m0.402s
    sys     0m3.115s

--- with patch:

    1 loop, best of 5: 646 msec per loop

    real    0m5.475s
    user    0m0.037s
    sys     0m2.959s

----------
components: Library (Lib)
messages: 317611
nosy: StyXman, desbma, facundobatista, giampaolo.rodola, martin.panter, ncoghlan, neologix, petr.viktorin, python-dev, r.david.murray, vstinner
priority: normal
severity: normal
stage: patch review
status: open
title: Use high-performance os.sendfile() in shutil.copy*
type: performance
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33639>
_______________________________________

From report at bugs.python.org  Thu May 24 17:08:45 2018
From: report at bugs.python.org (STINNER Victor)
Date: Thu, 24 May 2018 21:08:45 +0000
Subject: [New-bugs-announce] [issue33640] uuid: endian of the bytes argument
 is not documented
Message-ID: <1527196125.95.0.682650639539.issue33640@psf.upfronthosting.co.za>


New submission from STINNER Victor <vstinner at redhat.com>:

I don't understand what is the endian of the 'bytes' parameter of the uuid.UUID constructor:
https://docs.python.org/dev/library/uuid.html#uuid.UUID

According to examples, it seems like the endian is big endian, but I'm not sure.

Related issue: bpo-32493.

----------
assignee: docs at python
components: Documentation
messages: 317626
nosy: docs at python, vstinner
priority: normal
severity: normal
status: open
title: uuid: endian of the bytes argument is not documented
versions: Python 2.7, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33640>
_______________________________________

From report at bugs.python.org  Thu May 24 18:14:54 2018
From: report at bugs.python.org (Serhiy Storchaka)
Date: Thu, 24 May 2018 22:14:54 +0000
Subject: [New-bugs-announce] [issue33641] Add links to RFCs
Message-ID: <1527200094.65.0.682650639539.issue33641@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:

The proposed PR replaces "RFC XYZ" with ":rfc:`XYZ`". This adds hyperlinks to RFC documents. Actually 85% of all RFC references already use this markup, the PR change the rest 15% written as plain text.

----------
assignee: docs at python
components: Documentation
messages: 317629
nosy: docs at python, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Add links to RFCs
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33641>
_______________________________________

From report at bugs.python.org  Thu May 24 20:41:45 2018
From: report at bugs.python.org (Cheryl Sabella)
Date: Fri, 25 May 2018 00:41:45 +0000
Subject: [New-bugs-announce] [issue33642] IDLE: Use variable number of lines
 in CodeContext
Message-ID: <1527208905.36.0.682650639539.issue33642@psf.upfronthosting.co.za>


New submission from Cheryl Sabella <cheryl.sabella at gmail.com>:

Item 7 from #33610:
Change fixed # of lines to variable # of lines as needed, up to limit.  About 15 is limit for 4-space indents in 80 char lines. 

* Change default number of lines to be 15 in config.
* Label in config dialog.
* Initialize context to be one blank line to show it's turned on.
* Remove padding of blank lines.
* Modify tests to show variable number of lines in context text instead of lines scrolling out of view.

Tip: For manual tests, help.py works really well in viewing 15 lines.

----------
assignee: terry.reedy
components: IDLE
messages: 317649
nosy: cheryl.sabella, terry.reedy
priority: normal
severity: normal
status: open
title: IDLE: Use variable number of lines in CodeContext
type: enhancement
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33642>
_______________________________________

From report at bugs.python.org  Fri May 25 03:39:19 2018
From: report at bugs.python.org (Piotr Dybowski)
Date: Fri, 25 May 2018 07:39:19 +0000
Subject: [New-bugs-announce] [issue33643] Mock functions with autospec STILL
 don't support assert_called_once, assert_called, assert_not_called
Message-ID: <1527233959.88.0.682650639539.issue33643@psf.upfronthosting.co.za>


New submission from Piotr Dybowski <piotr.p.dybowski at gmail.com>:

This is the same issue as: https://bugs.python.org/issue28380.
It seems that fix has never been merged,

----------
components: Library (Lib)
messages: 317659
nosy: dybi
priority: normal
severity: normal
status: open
title: Mock functions with autospec STILL don't support assert_called_once, assert_called, assert_not_called
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33643>
_______________________________________

From report at bugs.python.org  Fri May 25 04:49:30 2018
From: report at bugs.python.org (Serhiy Storchaka)
Date: Fri, 25 May 2018 08:49:30 +0000
Subject: [New-bugs-announce] [issue33644] Fix signatures of tp_finalize
 handlers in testing code.
Message-ID: <1527238170.64.0.682650639539.issue33644@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:

tp_finalize handlers in Modules/_testmultiphase.c and Modules/_testmultiphase.c return int. This signature is incompatible with the signature of the tp_finalize slot. They should not return any value.

----------
components: Tests
messages: 317662
nosy: serhiy.storchaka
priority: normal
severity: normal
status: open
title: Fix signatures of tp_finalize handlers in testing code.
type: behavior
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33644>
_______________________________________

From report at bugs.python.org  Fri May 25 05:17:42 2018
From: report at bugs.python.org (Florian Schulze)
Date: Fri, 25 May 2018 09:17:42 +0000
Subject: [New-bugs-announce] [issue33645] Importing bs4 fails with -3 option
 in Python 2.7.15
Message-ID: <1527239862.03.0.682650639539.issue33645@psf.upfronthosting.co.za>


New submission from Florian Schulze <florian.schulze at gmx.net>:

Since Python 2.7.15 import bs4 (BeautifulSoup4) fails when using the -3 option on the Python binary.

...
    'You are trying to run the Python 2 version of Beautiful Soup under Python 3. This will not work.'<>'You need to convert the code, either by installing it (`python setup.py install`) or by running 2to3 (`2to3 -w bs4`).'
                                                                                                       ^
SyntaxError: unknown parsing error

With 2.7.14 this works fine and I get the expected deprecation warning.

A workaround is to import the package without -3 first, so the .pyc file is generated and used in subsequent imports.

----------
components: Interpreter Core
messages: 317664
nosy: fschulze
priority: normal
severity: normal
status: open
title: Importing bs4 fails with -3 option in Python 2.7.15
type: behavior
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33645>
_______________________________________

From report at bugs.python.org  Fri May 25 06:00:01 2018
From: report at bugs.python.org (Joseph Martinot-Lagarde)
Date: Fri, 25 May 2018 10:00:01 +0000
Subject: [New-bugs-announce] [issue33646] os.fspath() bypasses __fspath__
 for str subclasses
Message-ID: <1527242401.32.0.682650639539.issue33646@psf.upfronthosting.co.za>


New submission from Joseph Martinot-Lagarde <contrebasse at gmail.com>:

os.fspath() returns its argument if it is a str. That means that it bypasses __fspath__ for str subclasses.

This is the case for the library path.py for example.

This is a corner case that was discovered while trying to fix https://github.com/matplotlib/matplotlib/issues/11306

Minimal example:

```
import os

class MyPath(str):
    def __fspath__(self):
        print("Returns a pure string")
        return str(self)

os.fspath(MyPath())  # Prints nothing
```

----------
components: Library (Lib)
messages: 317666
nosy: contrebasse
priority: normal
severity: normal
status: open
title: os.fspath() bypasses __fspath__ for str subclasses
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33646>
_______________________________________

From report at bugs.python.org  Fri May 25 08:52:17 2018
From: report at bugs.python.org (Paal Pedersen)
Date: Fri, 25 May 2018 12:52:17 +0000
Subject: [New-bugs-announce] [issue33647] Make string.replace accept
 **kwargs instead of two arguments
Message-ID: <1527252737.56.0.682650639539.issue33647@psf.upfronthosting.co.za>


New submission from Paal Pedersen <paalped at gmail.com>:

It would be really nice if the python core function string.replace is modifed to accept **kwargs instead of 2 arguments.

then you could just do:

kwargs = {'my': 'yours', 'string': 'car'}

>>>'this is my string'.replace(kwargs) 
'this is your car'

----------
messages: 317672
nosy: paalped
priority: normal
severity: normal
status: open
title: Make string.replace accept **kwargs instead of two arguments
type: enhancement
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33647>
_______________________________________

From report at bugs.python.org  Fri May 25 10:11:51 2018
From: report at bugs.python.org (Eitan Adler)
Date: Fri, 25 May 2018 14:11:51 +0000
Subject: [New-bugs-announce] [issue33648] unused with_c_locale_warning
 option in configure should be removed
Message-ID: <1527257511.03.0.682650639539.issue33648@psf.upfronthosting.co.za>


New submission from Eitan Adler <lists at eitanadler.com>:

There is an option for --with-c-locale-warning which was turned into a run-time option in eb81795d7d3a8c898fa89a376d63fc3bbfb9a081. The configuration should be cleaned up.

----------
messages: 317676
nosy: eitan.adler
priority: normal
severity: normal
status: open
title: unused with_c_locale_warning option in configure should be removed

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33648>
_______________________________________

From report at bugs.python.org  Fri May 25 15:48:42 2018
From: report at bugs.python.org (Yury Selivanov)
Date: Fri, 25 May 2018 19:48:42 +0000
Subject: [New-bugs-announce] [issue33649] asyncio docs overhaul
Message-ID: <1527277722.54.0.682650639539.issue33649@psf.upfronthosting.co.za>


New submission from Yury Selivanov <yselivanov at gmail.com>:

An overhaul of asyncio documentation is long overdue.  Here's the structure for it that I have in mind:

- Introduction
(what is asyncio and async/await)

- A quick tutorial
(show how to use asyncio.run() and basic functions like asyncio.sleep() and teach that asyncio programs are all about async/await and *not* about callbacks or event loops)

- High-level APIs
(Tasks, Streams, Subprocesses, few other functions)

- Low-level APIs
  - Preface (talk a bit about everything: what's an event loop, what is a Future and a Transport)
  - Futures
  - Event loop APIs
  - Transports and Protocols (when to use and when not to use them)

- Tutorials
  - High-level networking server
  - HTTP application
  - Low-level protocol implementation using Transports
  - etc

----------
assignee: docs at python
components: Documentation, asyncio
messages: 317709
nosy: akuchling, asvetlov, docs at python, yselivanov
priority: normal
severity: normal
status: open
title: asyncio docs overhaul
type: enhancement
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33649>
_______________________________________

From report at bugs.python.org  Fri May 25 16:56:35 2018
From: report at bugs.python.org (Yury Selivanov)
Date: Fri, 25 May 2018 20:56:35 +0000
Subject: [New-bugs-announce] [issue33650] Prohibit adding a signal handler
 for SIGCHLD
Message-ID: <1527281795.42.0.682650639539.issue33650@psf.upfronthosting.co.za>


New submission from Yury Selivanov <yselivanov at gmail.com>:

Doing that will break subprocesses.

----------
components: asyncio
messages: 317717
nosy: asvetlov, yselivanov
priority: normal
severity: normal
status: open
title: Prohibit adding a signal handler for SIGCHLD
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33650>
_______________________________________

From report at bugs.python.org  Fri May 25 17:58:55 2018
From: report at bugs.python.org (Wilfredo Sanchez)
Date: Fri, 25 May 2018 21:58:55 +0000
Subject: [New-bugs-announce] [issue33651] Add get() method to sqlite3.Row
 class
Message-ID: <1527285535.19.0.682650639539.issue33651@psf.upfronthosting.co.za>


New submission from Wilfredo Sanchez <wsanchez at wsanchez.net>:

The sqlite3.Row class has mapping-like behavior but does not implement the get() method, so providing default values in code requires a bit of boilerplate that is not necessary with dictionaries.

----------
components: Library (Lib)
messages: 317719
nosy: wsanchez
priority: normal
severity: normal
status: open
title: Add get() method to sqlite3.Row class
type: enhancement
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33651>
_______________________________________

From report at bugs.python.org  Sat May 26 03:10:15 2018
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sat, 26 May 2018 07:10:15 +0000
Subject: [New-bugs-announce] [issue33652] Improve pickling of typing types
Message-ID: <1527318615.23.0.682650639539.issue33652@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:

The following PR makes pickles for typing types more portable. Type variables no longer use _find_name() and can be unpickled in 3.6. Subscripted generics no longer expose internals and can be unpickled in 3.6 and future versions with changed internal implementation.

Before this PR:

>>> import pickle, pickletools, typing                                            
>>> pickletools.dis(pickletools.optimize(pickle.dumps(typing.T, 4)))              
    0: \x80 PROTO      4
    2: \x95 FRAME      30
   11: \x8c SHORT_BINUNICODE 'typing'
   19: \x94 MEMOIZE    (as 0)
   20: \x8c SHORT_BINUNICODE '_find_name'
   32: \x93 STACK_GLOBAL
   33: h    BINGET     0
   35: \x8c SHORT_BINUNICODE 'T'
   38: \x86 TUPLE2
   39: R    REDUCE
   40: .    STOP
highest protocol among opcodes = 4
>>> pickletools.dis(pickletools.optimize(pickle.dumps(typing.Union[int, str], 4)))
    0: \x80 PROTO      4
    2: \x95 FRAME      198
   11: \x8c SHORT_BINUNICODE 'copyreg'
   20: \x8c SHORT_BINUNICODE '_reconstructor'
   36: \x93 STACK_GLOBAL
   37: \x8c SHORT_BINUNICODE 'typing'
   45: \x94 MEMOIZE    (as 0)
   46: \x8c SHORT_BINUNICODE '_GenericAlias'
   61: \x93 STACK_GLOBAL
   62: \x8c SHORT_BINUNICODE 'builtins'
   72: \x94 MEMOIZE    (as 1)
   73: \x8c SHORT_BINUNICODE 'object'
   81: \x93 STACK_GLOBAL
   82: N    NONE
   83: \x87 TUPLE3
   84: R    REDUCE
   85: }    EMPTY_DICT
   86: (    MARK
   87: \x8c     SHORT_BINUNICODE '_inst'
   94: \x88     NEWTRUE
   95: \x8c     SHORT_BINUNICODE '_special'
  105: \x89     NEWFALSE
  106: \x8c     SHORT_BINUNICODE '_name'
  113: N        NONE
  114: \x8c     SHORT_BINUNICODE '__origin__'
  126: h        BINGET     0
  128: \x8c     SHORT_BINUNICODE 'Union'
  135: \x93     STACK_GLOBAL
  136: \x8c     SHORT_BINUNICODE '__args__'
  146: h        BINGET     1
  148: \x8c     SHORT_BINUNICODE 'int'
  153: \x93     STACK_GLOBAL
  154: h        BINGET     1
  156: \x8c     SHORT_BINUNICODE 'str'
  161: \x93     STACK_GLOBAL
  162: \x86     TUPLE2
  163: \x8c     SHORT_BINUNICODE '__parameters__'
  179: )        EMPTY_TUPLE
  180: \x8c     SHORT_BINUNICODE '__slots__'
  191: N        NONE
  192: \x8c     SHORT_BINUNICODE '__module__'
  204: h        BINGET     0
  206: u        SETITEMS   (MARK at 86)
  207: b    BUILD
  208: .    STOP
highest protocol among opcodes = 4

With this PR:

>>> import pickle, pickletools, typing                                                              
>>> pickletools.dis(pickletools.optimize(pickle.dumps(typing.T, 4)))                       
    0: \x80 PROTO      4
    2: \x95 FRAME      13
   11: \x8c SHORT_BINUNICODE 'typing'
   19: \x8c SHORT_BINUNICODE 'T'
   22: \x93 STACK_GLOBAL
   23: .    STOP
highest protocol among opcodes = 4
>>> pickletools.dis(pickletools.optimize(pickle.dumps(typing.Union[int, str], 4)))       
    0: \x80 PROTO      4
    2: \x95 FRAME      66
   11: \x8c SHORT_BINUNICODE '_operator'
   22: \x8c SHORT_BINUNICODE 'getitem'
   31: \x93 STACK_GLOBAL
   32: \x8c SHORT_BINUNICODE 'typing'
   40: \x8c SHORT_BINUNICODE 'Union'
   47: \x93 STACK_GLOBAL
   48: \x8c SHORT_BINUNICODE 'builtins'
   58: \x94 MEMOIZE    (as 0)
   59: \x8c SHORT_BINUNICODE 'int'
   64: \x93 STACK_GLOBAL
   65: h    BINGET     0
   67: \x8c SHORT_BINUNICODE 'str'
   72: \x93 STACK_GLOBAL
   73: \x86 TUPLE2
   74: \x86 TUPLE2
   75: R    REDUCE
   76: .    STOP
highest protocol among opcodes = 4

If there is a chance it would be nice to merge these changes into 3.7. Otherwise either pickles created in 3.7.0 will be incompatible not only with older Python versions, but with future Python versions too, or we will need to add complex code for supporting specific 3.7.0 pickles in future versions.

----------
components: Library (Lib)
messages: 317727
nosy: gvanrossum, levkivskyi, ned.deily, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Improve pickling of typing types
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33652>
_______________________________________

From report at bugs.python.org  Sat May 26 07:06:26 2018
From: report at bugs.python.org (Giampaolo Rodola')
Date: Sat, 26 May 2018 11:06:26 +0000
Subject: [New-bugs-announce] [issue33653] EnvironmentError does not set
 errno unless strerror is set
Message-ID: <1527332786.28.0.682650639539.issue33653@psf.upfronthosting.co.za>


New submission from Giampaolo Rodola' <g.rodola at gmail.com>:

>>> import errno
>>> OSError(errno.EBADF).errno
>>> OSError(errno.EBADF, "yo").errno
9
>>> IOError(errno.EBADF).errno
>>> IOError(errno.EBADF, "yo").errno
9
>>> EnvironmentError(errno.EBADF).errno
>>> 
>>> EnvironmentError(errno.EBADF, "yo").errno
9

This is a bit annoying mainly when mocking fs-related functions in order to raise specific OSError exceptions.

----------
messages: 317737
nosy: giampaolo.rodola
priority: normal
severity: normal
status: open
title: EnvironmentError does not set errno unless strerror is set
versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33653>
_______________________________________

From report at bugs.python.org  Sat May 26 13:08:31 2018
From: report at bugs.python.org (Yury Selivanov)
Date: Sat, 26 May 2018 17:08:31 +0000
Subject: [New-bugs-announce] [issue33654] asyncio: transports don't support
 switching between Protocol and BufferedProtocol
Message-ID: <1527354511.66.0.682650639539.issue33654@psf.upfronthosting.co.za>


New submission from Yury Selivanov <yselivanov at gmail.com>:

1. asyncio.BufferedProtocol is a new Python 3.7 *provisional* API.

2. asyncio.Transport.set_protocol() doesn't support switching between Protocol and BufferedProtocol.

3. Because of (2), another *new* Python 3.7 API "loop.start_tls" doesn't support asyncio.BufferedProtocol at all.

The fix is straightforward and *does not* affect any existing APIs in Python 3.6 and lower.  Since the patch adjusts the APIs a little bit, it would be great to merge this in 3.7.0.

However, since only the provisional APIs are changed, this can be theoretically merged to 3.7.1 too.  It would be nice, though, to get this into 3.7.0.  Ned, your thoughts?

----------
assignee: yselivanov
components: asyncio
messages: 317743
nosy: asvetlov, ned.deily, yselivanov
priority: high
severity: normal
status: open
title: asyncio: transports don't support switching between Protocol and BufferedProtocol
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33654>
_______________________________________

From report at bugs.python.org  Sat May 26 15:52:51 2018
From: report at bugs.python.org (Ned Deily)
Date: Sat, 26 May 2018 19:52:51 +0000
Subject: [New-bugs-announce] [issue33655] test_posix_fallocate fails on
 FreeBSD buildbots with ZFS file systems
Message-ID: <1527364371.11.0.682650639539.issue33655@psf.upfronthosting.co.za>


New submission from Ned Deily <nad at python.org>:

Seen on some of the various FreeBSD buildbots:

======================================================================
ERROR: test_posix_fallocate (test.test_posix.PosixTester)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.7.koobs-freebsd-current/build/Lib/test/test_posix.py", line 342, in test_posix_fallocate
    posix.posix_fallocate(fd, 0, 10)
OSError: [Errno 22] Invalid argument
----------------------------------------------------------------------

As discussed in Issue31106 msg312453 and successors, the failure is due to posix_fallocate not being implemented on ZFS and the difficulty of determining in a platform-independent way what kind of file system a given file resides on.  The test already ignores EINVAL ("Invalid argument") errors when run on Solaris-type platforms due to the common use of ZFS there.  In lieu of a more precise test for fs type, we should also ignore EINVAL errors in this test when run on *BSD platforms.  Also, rather than silently ignoring the error, it may be better to use a SkipTest message.

----------
assignee: ned.deily
components: FreeBSD
messages: 317763
nosy: koobs, ned.deily
priority: normal
severity: normal
stage: patch review
status: open
title: test_posix_fallocate fails on FreeBSD buildbots with ZFS file systems
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33655>
_______________________________________

From report at bugs.python.org  Sat May 26 19:14:50 2018
From: report at bugs.python.org (Terry J. Reedy)
Date: Sat, 26 May 2018 23:14:50 +0000
Subject: [New-bugs-announce] [issue33656] IDLE: Turn on DPI awareness on
 Windows
Message-ID: <1527376490.94.0.682650639539.issue33656@psf.upfronthosting.co.za>


New submission from Terry J. Reedy <tjreedy at udel.edu>:

On IDLE-dev today, Elisha Paine, from Ranelagh School in England, wrote:
'''
I love IDLE (so simple and lightweight compared with other IDEs) and was just wondering if you could add the following code into pyshell.py (as I have done on mine) because, as you will be able to tell, it sets DPI awareness (and on my set-up makes the text a lot clearer)

import ctypes
try: ctypes.windll.shcore.SetProcessDpiAwareness (True)
except: pass
'''
On my Win 10 system with a 2560 x 1440 27" monitor, the text is noticeably sharper, some colors are brighter, and some characters are better formed.  The main effect seems to be from properly lining up vertical and horizontal lines with the pixels.  Lines that are supposed to be 1 or 2 pixels wide are just that, instead of bleeding over onto additional rows or columns.

By comparing IDLE's shell with Windows consoles with the same font and text, both for Command Prompt and Python x.y consoles, I determined that the Windows consoles have DPI awareness on.  The text is longer with it off and some character shapes, such as for 2 and 3, don't match, whereas text length and shape matches perfectly with it on.

At least some other apps also have DPO awareness on.  With the patch, the Open and Save As dialogs opened opened by from IDLE, via the tkinter/tk function, match those opened by Firefox.  Without the patch, not.

I like the improvement and with 3.7.0rc1 delayed, would like to get it in now.  My question for you Windows experts is 1. Is the above exactly the right thing? and 2. Should it be unconditional, or is there a possible downside?  Do consoles always have DPI awareness, or is it conditional on the monitor? 

I will make a PR as soon as I post this and get an issue number.

----------
assignee: terry.reedy
components: IDLE
messages: 317775
nosy: paul.moore, steve.dower, terry.reedy, tim.golden, zach.ware
priority: normal
severity: normal
stage: patch review
status: open
title: IDLE: Turn on DPI awareness on Windows
type: enhancement
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33656>
_______________________________________

From report at bugs.python.org  Sun May 27 04:53:27 2018
From: report at bugs.python.org (Ales Kvapil)
Date: Sun, 27 May 2018 08:53:27 +0000
Subject: [New-bugs-announce] [issue33658] Introduce a method to concatenate
 regex patterns
Message-ID: <1527411207.29.0.682650639539.issue33658@psf.upfronthosting.co.za>


New submission from Ales Kvapil <aleskva at mailinator.com>:

After removing inline flags (deprecated in https://bugs.python.org/issue22493) piping patterns will not work:
pats = [r'(?m)^line.continues$', r'(?s)begin.*?end']
re.compile('|'.join(pats))


Maybe there should be introduced some method to merge patterns (similar to re.escape)?

----------
components: Regular Expressions
messages: 317787
nosy: aleskva, ezio.melotti, mrabarnett
priority: normal
severity: normal
status: open
title: Introduce a method to concatenate regex patterns
type: enhancement
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33658>
_______________________________________

From report at bugs.python.org  Sun May 27 05:48:37 2018
From: report at bugs.python.org (lekma)
Date: Sun, 27 May 2018 09:48:37 +0000
Subject: [New-bugs-announce] [issue33659] leak in pythonrun.c?
Message-ID: <1527414517.04.0.682650639539.issue33659@psf.upfronthosting.co.za>


New submission from lekma <lekmalek at gmail.com>:

Am I wrong in thinking 'filename_obj' should be decrefed before the last two return statements in set_main_loader()? If yes, what am I missing?

----------
components: Interpreter Core
messages: 317789
nosy: lekma
priority: normal
severity: normal
status: open
title: leak in pythonrun.c?
type: behavior
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33659>
_______________________________________

From report at bugs.python.org  Sun May 27 06:58:33 2018
From: report at bugs.python.org (QbLearningPython)
Date: Sun, 27 May 2018 10:58:33 +0000
Subject: [New-bugs-announce] [issue33660] pathlib.Path.resolve() returns
 path with double slash when resolving a relative path in root directory
Message-ID: <1527418713.03.0.682650639539.issue33660@psf.upfronthosting.co.za>


New submission from QbLearningPython <quibizo at hotmail.com>:

I have recently found a weird behaviour while trying to resolve a relative path located on the root directory on a macOs.

I tried to resolve a Path('spam') and the interpreter answered PosixPath('//spam') ?double slash for root? instead of (my) expected PosixPath('/spam').

I think that this is a bug.

I ran the interpreter from root directory (cd /; python). Once running the interpreter, this is what I did:

>>> import pathlib
>>> pathlib.Path.cwd()
PosixPath('/')
# since the interpreter has been launched from root
>>> p = pathlib.Path('spam')
>>> p
PosixPath('spam')
# just for checking
>>> p.resolve()
PosixPath('//spam')
# beware of double slash instead of single slash


I also checked the behaviour of Path.resolve() in a non-root directory (in my case launching the interpreter from /Applications).

>>> import pathlib
>>> pathlib.Path.cwd()
PosixPath('/Applications')
>>> p = pathlib.Path('eggs')
>>> p
PosixPath('eggs')
>>> p.resolve()
PosixPath('/Applications/eggs')
# just one slash as root in this case (as should be)

So it seems that double slashes just appear while resolving relative paths in the root directory.

More examples are:
	
>>> pathlib.Path('spam/egg').resolve()
PosixPath('//spam/egg')
>>> pathlib.Path('./spam').resolve()
PosixPath('//spam')
>>> pathlib.Path('./spam/egg').resolve()
PosixPath('//spam/egg')

but

>>> pathlib.Path('').resolve()
PosixPath('/')
>>> pathlib.Path('.').resolve()
PosixPath('/')

Intriguingly,

>>> pathlib.Path('spam').resolve().resolve()
PosixPath('/spam')
# 'spam'.resolve = '//spam'
# '//spam'.resolve = '/spam'!!!
>>> pathlib.Path('//spam').resolve()
PosixPath('/spam')

I have found the same behaviour in several Python versions:

Python 3.6.5 (default, May 15 2018, 08:20:57)
[GCC 4.2.1 Compatible Apple LLVM 9.1.0 (clang-902.0.39.1)] on darwin

Python 3.4.8 (default, Mar 29 2018, 16:18:25)
[GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.39.2)] on darwin

Python 3.5.5 (default, Mar 29 2018, 16:22:58)
[GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.39.2)] on darwin

Python 3.7.0b4 (default, May 4 2018, 22:01:49)
[Clang 9.1.0 (clang-902.0.39.1)] on darwin


All running on: macOs High Sierra 10.13.4 (17E202)


There is also confirmation of same issue on Ubuntu 16.04 (Python 3.5.2) and Opensuse tumbleweed (Python 3.6.5)


I have searched for some information on this issue but I did not found anything useful.

Python docs (https://docs.python.org/3/library/pathlib.html) talks about "UNC shares" but this is not the case (in using a macOs HFS+ filesystem).

PEP 428 (https://www.python.org/dev/peps/pep-0428/) says:


    Multiple leading slashes are treated differently depending on the path flavour. They are always retained on Windows paths (because of the UNC notation):

    >>> PureWindowsPath('//some/path')
    PureWindowsPath('//some/path/')

    On POSIX, they are collapsed except if there are exactly two leading slashes, which is a special case in the POSIX specification on pathname resolution [8] (this is also necessary for Cygwin compatibility):

    >>> PurePosixPath('///some/path')
    PurePosixPath('/some/path')
    >>> PurePosixPath('//some/path')
    PurePosixPath('//some/path')


I do not think that this is related to the aforementioned issue.

However, I also checked the POSIX specification link (http://pubs.opengroup.org/onlinepubs/009...#tag_04_11) and found:

    A pathname that begins with two successive slashes may be interpreted in an implementation-defined manner, although more than two leading slashes shall be treated as a single slash.


I do not really think that this can cause a double slashes while resolving a relative path on macOs.


So, I think that this issue could be a real bug in pathlib.Path.resolve() method. Specifically on POSIX flavour.

A user of Python Forum (killerrex) and I have traced the bugs to Lib/pathlib.py:319 in the Python 3.6 repository https://github.com/python/cpython/blob/3...pathlib.py.

Specifically, in line 319:
	
    newpath = path + sep + name

For pathlib.Path('spam').resolve() in the root directory, newpath is '//spam' since:

    path is '/'
    sep is '/'
    name is 'spam'

killerrex has suggested two solutions:

1) from line 345 

    base = '' if path.is_absolute() else os.getcwd()
    if base == sep:
        base = ''
    return _resolve(base, str(path)) or sep

2) from line 319:

    if path.endswith(sep):
        newpath = path + name
    else:
        newpath = path + sep + name


Thank you.

----------
components: Library (Lib)
messages: 317790
nosy: QbLearningPython
priority: normal
severity: normal
status: open
title: pathlib.Path.resolve() returns path with double slash when resolving a relative path in root directory
type: behavior
versions: Python 3.4, Python 3.5, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33660>
_______________________________________

From report at bugs.python.org  Sun May 27 10:20:06 2018
From: report at bugs.python.org (Artem Smotrakov)
Date: Sun, 27 May 2018 14:20:06 +0000
Subject: [New-bugs-announce] [issue33661] urllib may leak sensitive HTTP
 headers to a third-party web site
Message-ID: <1527430806.55.0.682650639539.issue33661@psf.upfronthosting.co.za>


New submission from Artem Smotrakov <artem.smotrakov at gmail.com>:

After discussing it on security at python.org, it was decided to disclose it. Here is the original report:




Hello Python Security Team,

Looks like urllib may leak sensitive HTTP headers to third parties when handling redirects.

Let's consider the following environment:
- http://httpleak.gypsyengineer.com/index.php asks a user to authenticate via basic HTTP authentication scheme
- http://httpleak.gypsyengineer.com/redirect.php?url=<url> is an open redirect which returns 301 code, and redirects a client to the specified URL
- http://headers.gypsyengineer.com just prints out all HTTP headers which a web browser sent

Let's then consider the following scenario:
- create an instance of urllib.request.Request to open 'http://httpleak.gypsyengineer.com/redirect.php?url=http://headers.gypsyengineer.com'
- call urllib.request.Request.add_header() method to set Authorization and Cookie headers
- call urllib.request.urlopen() method to open a connection

Here is what happens next:
- urllib sends the HTTP authentication header to httpleak.gypsyengineer.com as expected
- redirect.php returns 301 code which redirects to headers.gypsyengineer.com (note that httpleak.gypsyengineer.com and headers.gypsyengineer.com are different domains)
- urllib processes 301 code and makes a request to http://headers.gypsyengineer.com

The problem is that urllib sends the Authorization and Cookie headers headers to http://headers.gypsyengineer.com as well.

Let's imagine that a user is authenticated on a web site via one of HTTP authentication schemes (basic, digest, NTLM, SPNEGO/Kerberos), 
and the web site has an open redirect like http://httpleak.gypsyengineer.com/redirect.php
If an attacker can trick the user to open http://httpleak.gypsyengineer.com/redirect.php?url=http://attacker.com, 
then urllib is going to send sensitive headers to http://attacker.com where the attacker can gather them. 
As a result, the attacker can imporsonate the user on the original web site.

Here is a simple POC which shows the problem:

import urllib.request
req = urllib.request.Request('http://httpleak.gypsyengineer.com/redirect.php?url=http://headers.gypsyengineer.com')
req.add_header('Authorization', 'Basic YWRtaW46dGVzdA==')
req.add_header('Cookie', 'This is only for httpleak.gypsyengineer.com');
with urllib.request.urlopen(req) as f:
  print(f.read(2048).decode("utf-8"))


Running this code results to loading http://headers.gypsyengineer.com which prints out Authorization and Cookie headers 
which are supposed to be sent only to httpleak.gypsyengineer.com:

Hello, I am <b>headers.gypsyengineer.com</b></br></br>
Here are HTTP headers you just sent me:</br></br>
Accept-Encoding: identity</br>
User-Agent: Python-urllib/3.8</br>
<b>Authorization: Basic YWRtaW46dGVzdA==</br></b>
<b>Cookie: This is only for httpleak.gypsyengineer.com</br></b>
Host: headers.gypsyengineer.com</br>
Cache-Control: max-age=259200</br>
Connection: keep-alive</br>


I could reproduce it with 3.5.2, and latest build of https://github.com/python/cpython

If I am not missing something, it would be better if urllib filtered out sensitive HTTP headers while handling redirects.

Please let me know if I wrote anything dumb and stupid, or if you have any questions :) Thanks!

Artem

----------
components: Library (Lib)
messages: 317793
nosy: alex, artem.smotrakov
priority: normal
severity: normal
status: open
title: urllib may leak sensitive HTTP headers to a third-party web site
type: security
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33661>
_______________________________________

From report at bugs.python.org  Sun May 27 18:34:41 2018
From: report at bugs.python.org (skorpeo)
Date: Sun, 27 May 2018 22:34:41 +0000
Subject: [New-bugs-announce] [issue33662] asyncio Stream Reader Blocks on
 read when data fetched is less than limit
Message-ID: <1527460481.84.0.682650639539.issue33662@psf.upfronthosting.co.za>


New submission from skorpeo <skorpeo11 at gmail.com>:

I humbly submit what I think may be a bug in the asyncio.StreamReader.read() function.  When n=-1 is supplied and the data is less than self._limit the read function creates a future and never wakes it up.  I believe the culprit is https://github.com/python/cpython/blob/9d3627e311211a1b4abcda29c36fe4afe2c46532/Lib/asyncio/streams.py#L632.  To fix the issue a condition is added to break out of the loop if the data read is less than the limit.  I can only attach one file so I am providing the fix here for asyncio streams.py:  

blocks.append(block)           # existing
if len(block) < self._limit:   # new
    break                      # new

I have also attached a test file that shows the blocking behavior which is alleviated with the above fix.  Finally, I am not sure how to handle a situation where the data is exactly equal to the limit and no subsequent data is sent.

----------
components: asyncio
files: pty_test.py
messages: 317810
nosy: asvetlov, skorpeo, yselivanov
priority: normal
severity: normal
status: open
title: asyncio Stream Reader Blocks on read when data fetched is less than limit
type: behavior
versions: Python 3.6
Added file: https://bugs.python.org/file47615/pty_test.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33662>
_______________________________________

From report at bugs.python.org  Sun May 27 19:03:22 2018
From: report at bugs.python.org (Jean-Marc Le Peuvedic)
Date: Sun, 27 May 2018 23:03:22 +0000
Subject: [New-bugs-announce] [issue33663] Web.py wsgiserver3.py raises
 TypeError when CSS file is not found
Message-ID: <1527462202.39.0.682650639539.issue33663@psf.upfronthosting.co.za>


New submission from Jean-Marc Le Peuvedic <lepeuvedic at gmail.com>:

When running the built-in web server of web.py, the following error messages appear when the HTTP client fetches a non existing CSS file:

TypeError('WSGI response header value 469 is not of type str.',)
Traceback (most recent call last):
  File "/home/jm/miniconda3/envs/REST/lib/python3.6/site-packages/web/wsgiserver/wsgiserver3.py", line 1089, in communicate
    req.respond()
  File "/home/jm/miniconda3/envs/REST/lib/python3.6/site-packages/web/wsgiserver/wsgiserver3.py", line 877, in respond
    self.server.gateway(self).respond()
  File "/home/jm/miniconda3/envs/REST/lib/python3.6/site-packages/web/wsgiserver/wsgiserver3.py", line 1982, in respond
    for chunk in response:
  File "/home/jm/miniconda3/envs/REST/lib/python3.6/site-packages/web/httpserver.py", line 267, in __iter__
    self.start_response(self.status, self.headers)
  File "/home/jm/miniconda3/envs/REST/lib/python3.6/site-packages/web/httpserver.py", line 320, in xstart_response
    out = start_response(status, response_headers, *args)
  File "/home/jm/miniconda3/envs/REST/lib/python3.6/site-packages/web/wsgiserver/wsgiserver3.py", line 2029, in start_response
    "WSGI response header value %r is not of type str." % v)
TypeError: WSGI response header value 469 is not of type str.

The faulty header is added by Python library, http/server.py. 
Error added in version 3.4 according to comments.

Lines 467-471 in the attached file:
            body = content.encode('UTF-8', 'replace')
            self.send_header("Content-Type", self.error_content_type)
            self.send_header('Content-Length', int(len(body)))
        self.end_headers()

The value for 'Content-Length' is passed as an 'int', but only a 'str' is acceptable.

In the latest revision of 'server.py', the same code appears line 453.
A possible correction is :

            body = content.encode('UTF-8', 'replace')
            self.send_header("Content-Type", self.error_content_type)
            self.send_header('Content-Length', str(int(len(body))))
        self.end_headers()

----------
components: Library (Lib)
files: server.py
messages: 317813
nosy: jmlp
priority: normal
severity: normal
status: open
title: Web.py wsgiserver3.py raises TypeError when CSS file is not found
type: behavior
versions: Python 3.6
Added file: https://bugs.python.org/file47616/server.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33663>
_______________________________________

From report at bugs.python.org  Sun May 27 20:46:46 2018
From: report at bugs.python.org (Terry J. Reedy)
Date: Mon, 28 May 2018 00:46:46 +0000
Subject: [New-bugs-announce] [issue33664] IDLE:  scroll text by lines,
 not pixels.
Message-ID: <1527468406.58.0.682650639539.issue33664@psf.upfronthosting.co.za>


New submission from Terry J. Reedy <tjreedy at udel.edu>:

When tk and hence, IDLE, opens a text window, it contains an integral number of lines of text in the base font.  Clicking a vertical scrollbar arrow button moves text up or down exactly one line.  Clicking in the trough between the buttons and and the slider moves text up or down one 'page', a page being the number of lines in the window -2.

The text and window can be de-synchronized with the slider and mouse wheel.  (If the window size is changes to a non-integral number of lines, 'synchronized' means a complete line at the top.)  By default, the wheel moves text about 3 lines per 'click'.  Apparently, it is exactly 50 pixels at a time.  This is discussed on #25015.  For that issue, the focus was on '3' (or 50) being too small. 

This issue is about changing 'about' to 'exactly', so that synchronized text remains so when using the wheel or slider.  For both, the solution should be to use Text.xview_scroll(n, units).

For the wheel, we can replace the default tk wheel handler.  While we are at it, we could make the wheel effect be +- 5 lines, which would solve #25015, which I will close.  I don't think a configuration setting is needed.

For the slider, we can replace xview as the scrollbar command with a function that maps command('moveto', fraction) to xview_scroll(n, 'units').  The number n should be the difference between the current top line
  ("text.index("@0,0").split('.')[0]")
and the desired top line
  ("round(fraction * text.index('end').split('.')[0]").
Experiments should show if we need to adjust the rounded value 

We should not need to adjust for the fact that the last <window-size> lines of text cannot become top lines.  A request to scroll 'too far' goes as far as possible without raising an exception.

----------
assignee: terry.reedy
components: IDLE
messages: 317819
nosy: cheryl.sabella, terry.reedy
priority: normal
severity: normal
stage: test needed
status: open
title: IDLE:  scroll text by lines, not pixels.
type: behavior
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33664>
_______________________________________

From report at bugs.python.org  Mon May 28 06:39:05 2018
From: report at bugs.python.org (pez)
Date: Mon, 28 May 2018 10:39:05 +0000
Subject: [New-bugs-announce] [issue33665] tkinter.ttk.Scrollbar.fraction()
 is inaccurate, or at least inconsistent compared to the non ttk version
Message-ID: <1527503945.83.0.682650639539.issue33665@psf.upfronthosting.co.za>


New submission from pez <4coreylopez at gmail.com>:

tkinter.Scrollbar(orient='horizontal').fraction() gives you an accurate representation of how far along the mouse is in the trough. The output is 0.0 when the mouse is on the first pixel of the trough. The output is 1.0 when the mouse is on the last pixel of the trough.

tkinter.ttk.Scrollbar(orient='horizontal').fraction() believes the trough is 16 pixels shorter than it is. The output is 0.0 when the mouse is on the first pixel of the trough. However, the output is greater than 1.0 when the mouse is on the last pixel of the trough. This extra 16 pixels is static, so the behavior is the same regardless of how wide the scrollbar is.

While possibly coincidental, it's interesting that the 16 pixels is also close to the default width of the scrollbar.

This is happening for me on Windows 7.
Did not test vertical orientation.

----------
components: Tkinter
files: ttk_scroll_bug.py
messages: 317836
nosy: pez
priority: normal
severity: normal
status: open
title: tkinter.ttk.Scrollbar.fraction() is inaccurate, or at least inconsistent compared to the non ttk version
type: behavior
versions: Python 3.6
Added file: https://bugs.python.org/file47617/ttk_scroll_bug.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33665>
_______________________________________

From report at bugs.python.org  Mon May 28 09:06:57 2018
From: report at bugs.python.org (=?utf-8?q?Miro_Hron=C4=8Dok?=)
Date: Mon, 28 May 2018 13:06:57 +0000
Subject: [New-bugs-announce] [issue33666] os.errno gone AWOL
Message-ID: <1527512817.93.0.682650639539.issue33666@psf.upfronthosting.co.za>


New submission from Miro Hron?ok <miro at hroncok.cz>:

In 3.7.0b4 I see the following traceback:

>>> import os
>>> os.errno
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: module 'os' has no attribute 'errno'

This was not the case for Python 3.6:

>>> import os
>>> os.errno
<module 'errno' (built-in)>

os.errno might not have been documented, however there are projects out there that use it:

https://github.com/intel/bmap-tools/issues/34
https://bugzilla.redhat.com/show_bug.cgi?id=1583196

So I suggest the removal of os.errno is something worth documenting at https://docs.python.org/3.7/whatsnew/3.7.html

The root of the change is in https://github.com/python/cpython/pull/1269/files#diff-405b29928f2a3ae216e45afe9b5d0c60

----------
assignee: docs at python
components: Documentation, Library (Lib)
messages: 317847
nosy: docs at python, hroncok
priority: normal
severity: normal
status: open
title: os.errno gone AWOL
type: behavior
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33666>
_______________________________________

From report at bugs.python.org  Mon May 28 09:12:45 2018
From: report at bugs.python.org (Pawel)
Date: Mon, 28 May 2018 13:12:45 +0000
Subject: [New-bugs-announce] [issue33667] Mock calls on mutable objects
Message-ID: <1527513165.39.0.682650639539.issue33667@psf.upfronthosting.co.za>


New submission from Pawel <inne.poczta at gmail.com>:

When method of mocked object is called multiple times with mutable object as parameter and attribute of this object has been updated between calls, mock_calls is only aware of the last value. I think it unexpected behaviour, that mock doesn't track value changes.
I've attached example code

----------
components: Tests
files: test.py
messages: 317849
nosy: pawelj
priority: normal
severity: normal
status: open
title: Mock calls on mutable objects
type: behavior
versions: Python 3.5, Python 3.6
Added file: https://bugs.python.org/file47618/test.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33667>
_______________________________________

From report at bugs.python.org  Mon May 28 10:22:23 2018
From: report at bugs.python.org (Oleg Oleynik)
Date: Mon, 28 May 2018 14:22:23 +0000
Subject: [New-bugs-announce] [issue33668] Wrong behavior of help function on
 module
Message-ID: <1527517343.16.0.682650639539.issue33668@psf.upfronthosting.co.za>


New submission from Oleg Oleynik <oleg_oleinik at mail.ru>:

I write file test.py:
def __getattr__(key):
    return None
help(__name__)

when I try to run it I've got error:
Traceback (most recent call last):
  File "test.py", line 5, in <module>
    help(__name__)
  File "C:\Program Files\Python37\lib\_sitebuiltins.py", line 103, in __call__
    return pydoc.help(*args, **kwds)
  File "C:\Program Files\Python37\lib\pydoc.py", line 1894, in __call__
    self.help(request)
  File "C:\Program Files\Python37\lib\pydoc.py", line 1944, in help
    elif request: doc(request, 'Help on %s:', output=self._output)
  File "C:\Program Files\Python37\lib\pydoc.py", line 1674, in doc
    pager(render_doc(thing, title, forceload))
  File "C:\Program Files\Python37\lib\pydoc.py", line 1667, in render_doc
    return title % desc + '\n\n' + renderer.document(object, name)
  File "C:\Program Files\Python37\lib\pydoc.py", line 385, in document
    if inspect.ismodule(object): return self.docmodule(*args)
  File "C:\Program Files\Python37\lib\pydoc.py", line 1136, in docmodule
    for key, value in inspect.getmembers(object, inspect.isclass):
  File "C:\Program Files\Python37\lib\inspect.py", line 330, in getmembers
    for base in object.__bases__:
TypeError: 'NoneType' object is not iterable

If I change definition of __getattr__ function to:
def __getattr__(key):
   raise AttributeError()
then help function runs without errors. But I need to return None value from __getattr__ function.
When I wrote this function in class definition there is also no errors during runtime

----------
components: Library (Lib)
messages: 317852
nosy: oleg_oleinik
priority: normal
severity: normal
status: open
title: Wrong behavior of help function on module
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33668>
_______________________________________

From report at bugs.python.org  Mon May 28 10:57:29 2018
From: report at bugs.python.org (Xiang Zhang)
Date: Mon, 28 May 2018 14:57:29 +0000
Subject: [New-bugs-announce] [issue33669] str.format should raise exception
 when placeholder number doesn't match argument number
Message-ID: <1527519449.91.0.682650639539.issue33669@psf.upfronthosting.co.za>


New submission from Xiang Zhang <angwerzx at 126.com>:

I'm somewhat surprised when I have code like `"hello %s".format(person)` in my project there is no exception raised and "hello %s" returned. Have several tries it seems `str.format` won't check argument number.

>>> '{} {}'.format(1)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
IndexError: tuple index out of range
>>> '{} {}'.format(1, 2, 3)
'1 2'
>>> ''.format(1, 2, 3)
''
>>>

The IndexError is not ideal. I think this behavior could be improved since it's highly possible such code is an oversight bug. The old format string does a good job in this place:

>>> "%s %s" % 1
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: not enough arguments for format string
>>> "%s %s" % (1,2,3)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: not all arguments converted during string formatting
>>> "" % 1
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: not all arguments converted during string formatting

----------
components: Interpreter Core
messages: 317859
nosy: xiang.zhang
priority: normal
severity: normal
status: open
title: str.format should raise exception when placeholder number doesn't match argument number
type: behavior
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33669>
_______________________________________

From report at bugs.python.org  Mon May 28 11:37:25 2018
From: report at bugs.python.org (=?utf-8?q?Andr=C3=A9s_Delfino?=)
Date: Mon, 28 May 2018 15:37:25 +0000
Subject: [New-bugs-announce] [issue33670] Use errorlevel of Sphinx main() in
 Doc\make.bat
Message-ID: <1527521845.19.0.682650639539.issue33670@psf.upfronthosting.co.za>


New submission from Andr?s Delfino <adelfino at gmail.com>:

Doc\make.bat is exiting without exposing Sphinx's main() return value, showing weird things like:

C:\Users\adelfino\Desktop\Personal\repos\cpython\Doc>make venv
Using py -3.6 (found with py.exe)
Merging Misc/NEWS with py -3.6 -m blurb
Running Sphinx v1.7.4

Sphinx error:
Builder name venv not registered or available through entry point

Build succeeded. All output should be in build\venv

PR fixes this.

----------
components: Demos and Tools
messages: 317870
nosy: adelfino
priority: normal
severity: normal
status: open
title: Use errorlevel of Sphinx main() in Doc\make.bat
type: enhancement
versions: Python 2.7, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33670>
_______________________________________

From report at bugs.python.org  Mon May 28 12:17:23 2018
From: report at bugs.python.org (Giampaolo Rodola')
Date: Mon, 28 May 2018 16:17:23 +0000
Subject: [New-bugs-announce] [issue33671] Efficient efficient zero-copy
 syscalls for shutil.copy* functions (Linux, OSX and Win)
Message-ID: <1527524243.69.0.682650639539.issue33671@psf.upfronthosting.co.za>


New submission from Giampaolo Rodola' <g.rodola at gmail.com>:

Patch in attachment uses platform specific zero-copy syscalls on Linux and Solaris (os.sendfile(2)), Windows (CopyFileW) and OSX (fcopyfile(2)) speeding up shutil.copyfile() and other functions using it (copy(), copy2(), copytree(), move()).

Average speedup for a 512MB file copy is +24% on Linux, +50% on OSX and +48% on Windows by copying file on the same partition (SSD disk was used).

Follows some benchmarks.

Setup
=====

Create 128K, 8M, 512M file:

    $ python -c "import os; f = open('f1', 'wb'); f.write(os.urandom(128 * 1024))"
    $ python -c "import os; f = open('f1', 'wb'); f.write(os.urandom(8 * 1024 * 1024))"
    $ python -c "import os; f = open('f1', 'wb'); f.write(os.urandom(512 * 1024 * 1024))"

Benchmark:

    $ time ./python -m timeit -s 'import shutil; p1 = "f1"; p2 = "f2"' 'shutil.copyfile(p1, p2)'

Linux
=====

128K copy (+13%):

    without patch:
        1000 loops, best of 5: 228 usec per loop
        real    0m1.756s
        user    0m0.386s
        sys     0m1.116s

    with patch:
        1000 loops, best of 5: 198 usec per loop
        real    0m1.464s
        user    0m0.281s
        sys     0m0.958s

8MB copy (+24%):

    without patch:
        50 loops, best of 5: 10.1 msec per loop
        real    0m2.703s
        user    0m0.316s
        sys     0m1.847s

    with patch:
        50 loops, best of 5: 7.78 msec per loop
        real    0m2.447s
        user    0m0.086s
        sys     0m1.682s

512MB copy (+26%):

    without patch:
        1 loop, best of 5: 872 msec per loop
        real    0m5.574s
        user    0m0.402s
        sys     0m3.115s

    with patch:
        1 loop, best of 5: 646 msec per loop
        real    0m5.475s
        user    0m0.037s
        sys     0m2.959s

OSX
===

128K copy (+8.5%):

    without patch:
        500 loops, best of 5: 508 usec per loop
        real    0m2.971s
        user    0m0.442s
        sys     0m2.168s

    with patch:
        500 loops, best of 5: 464 usec per loop
        real    0m2.798s
        user    0m0.379s
        sys     0m2.031s

8MB copy (+67%):

    without patch:
        20 loops, best of 5: 32.8 msec per loop
        real    0m3.672s
        user    0m0.357s
        sys     0m1.434s

    with patch:
        20 loops, best of 5: 10.8 msec per loop
        real    0m1.860s
        user    0m0.079s
        sys     0m0.719s

512MB copy (+50%):

    without patch:
        1 loop, best of 5: 953 msec per loop
        real    0m5.930s
        user    0m1.021s
        sys     0m4.835s
    
    with patch:
        1 loop, best of 5: 480 msec per loop
        real    0m3.150s
        user    0m0.067s
        sys     0m2.740s

Windows
=======

128K copy (+69%):

    without patch:
        50 loops, best of 5: 6.45 msec per loop
    with patch:
        50 loops, best of 5: 1.99 msec per loop

8M copy (+64%):

    without patch:
        10 loops, best of 5: 22.6 msec per loop
    with patch:
        50 loops, best of 5: 7.95 msec per loop

512M copy (+48%):

    without patch:
        1 loop, best of 5: 1.21 sec per loop
    with patch:
        1 loop, best of 5: 629 msec per loop

----------
components: Library (Lib)
files: shutil-zero-copy.diff
keywords: needs review, patch
messages: 317878
nosy: giampaolo.rodola
priority: normal
severity: normal
stage: patch review
status: open
title: Efficient efficient zero-copy syscalls for shutil.copy* functions (Linux, OSX and Win)
type: performance
versions: Python 3.8
Added file: https://bugs.python.org/file47621/shutil-zero-copy.diff

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33671>
_______________________________________

From report at bugs.python.org  Mon May 28 12:29:15 2018
From: report at bugs.python.org (Yury Selivanov)
Date: Mon, 28 May 2018 16:29:15 +0000
Subject: [New-bugs-announce] [issue33672] Fix Task.__repr__ crash when
 trying to format Cython's bogus coroutines
Message-ID: <1527524955.7.0.682650639539.issue33672@psf.upfronthosting.co.za>


Change by Yury Selivanov <yselivanov at gmail.com>:


----------
components: asyncio
nosy: asvetlov, yselivanov
priority: normal
severity: normal
status: open
title: Fix Task.__repr__ crash when trying to format Cython's bogus coroutines
type: behavior
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33672>
_______________________________________

From report at bugs.python.org  Mon May 28 13:36:40 2018
From: report at bugs.python.org (=?utf-8?q?Andr=C3=A9s_Delfino?=)
Date: Mon, 28 May 2018 17:36:40 +0000
Subject: [New-bugs-announce] [issue33673] Install python-docs-theme even if
 Sphinx is already installed
Message-ID: <1527529000.2.0.682650639539.issue33673@psf.upfronthosting.co.za>


New submission from Andr?s Delfino <adelfino at gmail.com>:

python-docs-theme is being installed only if Sphinx is not installed too. PR fixes this.

----------
components: Demos and Tools
messages: 317892
nosy: adelfino
priority: normal
severity: normal
status: open
title: Install python-docs-theme even if Sphinx is already installed
type: enhancement
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33673>
_______________________________________

From report at bugs.python.org  Mon May 28 16:37:18 2018
From: report at bugs.python.org (STINNER Victor)
Date: Mon, 28 May 2018 20:37:18 +0000
Subject: [New-bugs-announce] [issue33674] asyncio: race condition in
 SSLProtocol
Message-ID: <1527539838.3.0.682650639539.issue33674@psf.upfronthosting.co.za>


New submission from STINNER Victor <vstinner at redhat.com>:

While debugging bpo-32458 (functional test on START TLS), I found a race condition in SSLProtocol of asyncio/sslproto.py.

Sometimes, _sslpipe.feed_ssldata() is called before _sslpipe.shutdown().

* SSLProtocol.connection_made() -> SSLProtocol._start_handshake(): self._loop.call_soon(self._process_write_backlog)
* SSLProtoco.data_received(): direct call to self._sslpipe.feed_ssldata(data)
* Later, self._process_write_backlog() calls self._sslpipe.do_handshake()

The first write is delayed by call_soon(), whereas the first read is a direct call to the SSL pipe.

----------
components: asyncio
messages: 317916
nosy: asvetlov, vstinner, yselivanov
priority: normal
severity: normal
status: open
title: asyncio: race condition in SSLProtocol
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33674>
_______________________________________

From report at bugs.python.org  Mon May 28 18:41:18 2018
From: report at bugs.python.org (Pablo Galindo Salgado)
Date: Mon, 28 May 2018 22:41:18 +0000
Subject: [New-bugs-announce] [issue33675] Buildbot AMD64 Windows10 3.6 fails
 to compile
Message-ID: <1527547278.22.0.682650639539.issue33675@psf.upfronthosting.co.za>


New submission from Pablo Galindo Salgado <pablogsal at gmail.com>:

It seems that since c489a767af46f33e73d96a5746e46a7365814db2 the AMD64 Windows10 buildbot for Python3.6 fails to compile. Error log:

Using "C:\Program Files (x86)\MSBuild\14.0\bin\\msbuild.exe" (found in the registry)
Cannot locate MSBuild.exe on PATH or as MSBUILD variable 
Deleting .pyc/.pyo files ...
Deleting test leftovers ...
Could Not Find D:\buildarea\3.6.bolen-windows10\build\Lib\*.pyc
The system cannot find the file specified.
Could Not Find D:\buildarea\3.6.bolen-windows10\build\PCbuild\python*.zip
Using "D:\buildarea\3.6.bolen-windows10\build\PCbuild\\..\externals\pythonx86\tools\python.exe" (found in externals directory)
Fetching external libraries...
bzip2-1.0.6 already exists, skipping.
openssl-1.0.2o already exists, skipping.
sqlite-3.21.0.0 already exists, skipping.
tcl-core-8.6.6.0 already exists, skipping.
tk-8.6.6.0 already exists, skipping.
tix-8.4.3.6 already exists, skipping.
xz-5.2.2 already exists, skipping.
Fetching external binaries...
nasm-2.11.06 already exists, skipping.
Finished.
Using "C:\Program Files (x86)\MSBuild\14.0\bin\\msbuild.exe" (found in the registry)
Cannot locate MSBuild.exe on PATH or as MSBUILD variable 
program finished with exit code 2
elapsedTime=1.953000

----------
components: Build
messages: 317937
nosy: pablogsal
priority: normal
severity: normal
status: open
title: Buildbot AMD64 Windows10 3.6 fails to compile
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33675>
_______________________________________

From report at bugs.python.org  Mon May 28 19:03:48 2018
From: report at bugs.python.org (STINNER Victor)
Date: Mon, 28 May 2018 23:03:48 +0000
Subject: [New-bugs-announce] [issue33676] test_multiprocessing_fork:
 dangling threads warning
Message-ID: <1527548628.93.0.682650639539.issue33676@psf.upfronthosting.co.za>


New submission from STINNER Victor <vstinner at redhat.com>:

AMD64 FreeBSD 10.x Shared 3.7:

http://buildbot.python.org/all/#/builders/124/builds/327

== CPU count: 2
Run tests in parallel using 4 child processes
(...)
0:02:09 load avg: 3.70 [138/415/1] test_multiprocessing_fork failed (env changed) (108 sec) -- running: test_concurrent_futures (124 sec)
(...)
test_starmap (test.test_multiprocessing_fork.WithThreadsTestPool) ... ok
test_starmap_async (test.test_multiprocessing_fork.WithThreadsTestPool) ... ok
test_terminate (test.test_multiprocessing_fork.WithThreadsTestPool) ... ok
test_traceback (test.test_multiprocessing_fork.WithThreadsTestPool) ... ok
test_wrapped_exception (test.test_multiprocessing_fork.WithThreadsTestPool) ... ok

Warning -- Dangling threads: {<DummyProcess(Thread-158, started daemon 34498362368)>}

----------
components: Tests
messages: 317948
nosy: vstinner
priority: normal
severity: normal
status: open
title: test_multiprocessing_fork: dangling threads warning
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33676>
_______________________________________

From report at bugs.python.org  Tue May 29 03:06:44 2018
From: report at bugs.python.org (Serhiy Storchaka)
Date: Tue, 29 May 2018 07:06:44 +0000
Subject: [New-bugs-announce] [issue33677] Fix signatures of tp_clear handlers
Message-ID: <1527577604.06.0.682650639539.issue33677@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:

The tp_clear handler should be a pointer to the function that returns int.

    typedef int (*inquiry)(PyObject *);

Two handlers in the stdlib (for AST and deque objects) return void instead. The following PR fixes this.

----------
components: Extension Modules
messages: 317986
nosy: rhettinger, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Fix signatures of tp_clear handlers
type: behavior
versions: Python 2.7, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33677>
_______________________________________

From report at bugs.python.org  Tue May 29 05:43:27 2018
From: report at bugs.python.org (Sebastien Bourdeauducq)
Date: Tue, 29 May 2018 09:43:27 +0000
Subject: [New-bugs-announce] [issue33678]
 selector_events.BaseSelectorEventLoop.sock_connect should preserve socket
 type
Message-ID: <1527587007.35.0.682650639539.issue33678@psf.upfronthosting.co.za>


New submission from Sebastien Bourdeauducq <sb at m-labs.hk>:

The current behavior causes an exception to be raised when trying to create a datagram socket and _ipaddr_info returns None (since asyncio then calls loop.getaddrinfo with SOCK_STREAM and IPPROTO_UDP).

Preserving socket type is made difficult by https://bugs.python.org/issue21327; the proposed patch includes a dirty workaround. I do not know yet if it works on Windows.

--- selector_events.py  2018-05-11 10:01:42.011164837 +0800
+++ selector_events.py.new      2018-05-29 17:41:03.103155480 +0800
@@ -439,8 +439,9 @@
             raise ValueError("the socket must be non-blocking")
 
         if not hasattr(socket, 'AF_UNIX') or sock.family != socket.AF_UNIX:
+            socktype = sock.type & 0xf  # WA https://bugs.python.org/issue21327
             resolved = base_events._ensure_resolved(
-                address, family=sock.family, proto=sock.proto, loop=self)
+                address, family=sock.family, type=socktype, proto=sock.proto, loop=self)
             if not resolved.done():
                 yield from resolved
             _, _, _, _, address = resolved.result()[0]

----------
components: asyncio
messages: 318001
nosy: asvetlov, sebastien.bourdeauducq, yselivanov
priority: normal
severity: normal
status: open
title: selector_events.BaseSelectorEventLoop.sock_connect should preserve socket type
type: behavior
versions: Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33678>
_______________________________________

From report at bugs.python.org  Tue May 29 07:13:51 2018
From: report at bugs.python.org (Cheryl Sabella)
Date: Tue, 29 May 2018 11:13:51 +0000
Subject: [New-bugs-announce] [issue33679] IDLE: Configurable color on code
 context
Message-ID: <1527592431.06.0.682650639539.issue33679@psf.upfronthosting.co.za>


New submission from Cheryl Sabella <cheryl.sabella at gmail.com>:

Item 9 from #33610:
Reenable user config of context colors

* Add context tag to themes in configuration.
* In codecontext, get theme and context color from configuration.
* Use the colors when creating the context label.
* In timer event, update colors if they have changed.
* Change name of font_timer_event function.
* Modify tests to include colors.

----------
assignee: terry.reedy
components: IDLE
messages: 318009
nosy: cheryl.sabella, terry.reedy
priority: normal
severity: normal
status: open
title: IDLE: Configurable color on code context
type: enhancement
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33679>
_______________________________________

From report at bugs.python.org  Tue May 29 07:41:03 2018
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 29 May 2018 11:41:03 +0000
Subject: [New-bugs-announce] [issue33680] regrtest: re-run failed tests in a
 subprocess
Message-ID: <1527594063.1.0.682650639539.issue33680@psf.upfronthosting.co.za>


New submission from STINNER Victor <vstinner at redhat.com>:

The -w option of regrtest (python -m test) re-run failed tests in verbose in the current process. Problem: if the test does crash, regrtest doesn't re-run following tests, it doesn't write the final tests result (like failing tests, total duration, etc.).

To make regrtest tolerant to such crashes, re-run tests should be run in verbose mode. The only constraint is to disable buffering on stdout and stderr to make sure that stdout and stderr order is kept in pipes. Or maybe stderr should be redirected to stdout to avoid ordering issue?

----------
components: Tests
messages: 318011
nosy: vstinner
priority: normal
severity: normal
status: open
title: regrtest: re-run failed tests in a subprocess
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33680>
_______________________________________

From report at bugs.python.org  Tue May 29 08:02:46 2018
From: report at bugs.python.org (Med Nezar BELLAZRAK)
Date: Tue, 29 May 2018 12:02:46 +0000
Subject: [New-bugs-announce] [issue33681] itertools.groupby() returned
 igroup is only callable once
Message-ID: <1527595366.08.0.682650639539.issue33681@psf.upfronthosting.co.za>


New submission from Med Nezar BELLAZRAK <bellazrak.n at gmail.com>:

Hello,

So i discovered a small unusual behavior (tracking it down was time-consuming) when using itertools.groupby(), i have checked the documentation and it states that:

"The returned group is itself an iterator that shares the underlying iterable with groupby(). Because the source is shared, when the groupby() object is advanced, the previous group is no longer visible. So, if that data is needed later, it should be stored as a list"

I do agree with this statement, though i believe a call to igroup successively in the same iteration should yield the same result while now it returns the correct igroup in the first call while the following call returns an empty list.

Here's a small code snippet that illustrates this behavior:

Code:

import itertools

mylist = [(1,2), (1,3), (2,5)]

for key, igroup in itertools.groupby(mylist, lambda x: x[0]):
    print(list(igroup)) # prints the expected igroup
    print(list(igroup)) # prints an empty list
    print(list(igroup)) # prints an empty list

Output:

[(1, 2), (1, 3)]
[]
[]
[(2, 5)]
[]
[]

Thanks in advance for anyone who works on this issue

----------
components: Extension Modules
messages: 318014
nosy: Med Nezar BELLAZRAK
priority: normal
severity: normal
status: open
title: itertools.groupby() returned igroup is only callable once
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33681>
_______________________________________

From report at bugs.python.org  Tue May 29 08:47:12 2018
From: report at bugs.python.org (=?utf-8?q?St=C3=A9phane_Wirtel?=)
Date: Tue, 29 May 2018 12:47:12 +0000
Subject: [New-bugs-announce] [issue33682] Optimize the bytecode for float(0)
 ?
Message-ID: <1527598032.12.0.682650639539.issue33682@psf.upfronthosting.co.za>


New submission from St?phane Wirtel <stephane at wirtel.be>:

Hi,

Maybe already discussed with Victor but I think there is no optimization when we have this simple case for float(X) and int(X)

Example:

>>> import dis
>>> dis.dis("x = float(0)")
  1           0 LOAD_NAME                0 (float)
              2 LOAD_CONST               0 (0)
              4 CALL_FUNCTION            1
              6 STORE_NAME               1 (x)
              8 LOAD_CONST               1 (None)
             10 RETURN_VALUE
>>> dis.dis("x = 1 + 1")
  1           0 LOAD_CONST               0 (2)
              2 STORE_NAME               0 (x)
              4 LOAD_CONST               1 (None)
              6 RETURN_VALUE
>>> dis.dis("x = int(0)")
  1           0 LOAD_NAME                0 (int)
              2 LOAD_CONST               0 (0)
              4 CALL_FUNCTION            1
              6 STORE_NAME               1 (x)
              8 LOAD_CONST               1 (None)
             10 RETURN_VALUE
>>> 


There is an optim for x = 1 + 1

----------
messages: 318016
nosy: matrixise, vstinner
priority: normal
severity: normal
status: open
title: Optimize the bytecode for float(0) ?
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33682>
_______________________________________

From report at bugs.python.org  Tue May 29 10:07:11 2018
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 29 May 2018 14:07:11 +0000
Subject: [New-bugs-announce] [issue33683] asyncio: sendfile tests ignore
 SO_SNDBUF on Windows
Message-ID: <1527602831.07.0.682650639539.issue33683@psf.upfronthosting.co.za>


New submission from STINNER Victor <vstinner at redhat.com>:

Follow-up of bpo-33353: On my PR 7200, test_sendfile_close_peer_in_the_middle_of_receiving() failed on Windows (AppVeyor) :-(

For the SelectorEventLoop, it seems like sock.setsockopt(socket.SOL_SOCKET, socket.SO_SNDBUF, 1024) doesn't work as intented: sock.send(<16384 bytes>) returns 16384, as the 1024 limit is not taken in account :-(

Same issue for the ProactorEventLoop...

I'm not sure if it's a bug in asyncio, a bug (limitation) of Windows which ignores SO_SNDBUF, or something else...

----------
components: Tests, Windows, asyncio
messages: 318031
nosy: asvetlov, paul.moore, steve.dower, tim.golden, vstinner, yselivanov, zach.ware
priority: normal
severity: normal
status: open
title: asyncio: sendfile tests ignore SO_SNDBUF on Windows
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33683>
_______________________________________

From report at bugs.python.org  Tue May 29 10:29:39 2018
From: report at bugs.python.org (zhou.ronghua)
Date: Tue, 29 May 2018 14:29:39 +0000
Subject: [New-bugs-announce] [issue33684] parse failed for mutibytes
 characters, encode will show in \xxx
Message-ID: <1527604179.28.0.682650639539.issue33684@psf.upfronthosting.co.za>


New submission from zhou.ronghua <zhou.ronghua at zte.com.cn>:

when type this command in windows(xp or win7, all the same):
python -m json.tool xxx.txt xxx.json
if xxx.txt contains Chinese(or other multibytes characters):
if xxx.txt is encoded in ansi, xxx.json will encode Chinese as \xxx, very bad to see what they are;
if xxx.txt is encoded in utf8(without bom for most of the time), because with no bom, json.tool will think it is encoded in ansi, and decode fail.

as now, utf8 is widely use, set default to utf8 for most of the time when auto detect encoding failed

----------
messages: 318039
nosy: zhou.ronghua
priority: normal
severity: normal
status: open
title: parse failed for mutibytes characters, encode will show in \xxx

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33684>
_______________________________________

From report at bugs.python.org  Tue May 29 12:28:20 2018
From: report at bugs.python.org (Marcelo Alves)
Date: Tue, 29 May 2018 16:28:20 +0000
Subject: [New-bugs-announce] [issue33685] Instances bound methods with
 different memory addresses but sharing same id
Message-ID: <1527611300.59.0.682650639539.issue33685@psf.upfronthosting.co.za>


New submission from Marcelo Alves <marcelo.franco.alves at gmail.com>:

Different instances should have different bound method ids, since they have different memory addresses, isn?t it? Example:

I have a class and two instances:

    class MyClass:
       def something():
          pass
    
    a = MyClass()
    b = MyClass()

If we print `a.something` and `b.something`, we can see that they have different memory addresses:

# a.something
<bound method MyClass.something of <__main__.MyClass object at 0x103438588>>

# b.something
<bound method MyClass.something of <__main__.MyClass object at 0x10342add8>>

This clear indicates that they aren?t the same, and we can confirm that comparing both using the `is` operator:

>>> a.something is b.something
False

But the identity of both indicates that they are the same, according with the doc of `is` and `id`:

>>> id(a.something)
4350192008

>>> id(b.something)
4350192008

The documentation of `is` says:

The operators?is?and?is?not?test for object identity:?x?is?y?is true if and only if?x?and?y?are the same object.?x?is?not?y?yields the inverse truth value.?[6]

ref: https://docs.python.org/2/reference/expressions.html#is

And the documentation of `id` says:

Return the ?identity? of an object. This is an integer (or long integer) which is guaranteed to be unique and constant for this object during its lifetime. Two objects with non-overlapping lifetimes may have the same?id()?value.

CPython implementation detail:?This is the address of the object in memory.

Thanks!

----------
components: Interpreter Core
messages: 318057
nosy: celicoo
priority: normal
severity: normal
status: open
title: Instances bound methods with different memory addresses but sharing same id
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33685>
_______________________________________

From report at bugs.python.org  Tue May 29 16:46:06 2018
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 29 May 2018 20:46:06 +0000
Subject: [New-bugs-announce] [issue33686] test_concurrent_futures:
 test_pending_calls_race() failed on x86 Windows7 3.6
Message-ID: <1527626766.22.0.682650639539.issue33686@psf.upfronthosting.co.za>


New submission from STINNER Victor <vstinner at redhat.com>:

x86 Windows7 3.6:

http://buildbot.python.org/all/#/builders/90/builds/354

0:23:45 [111/407/1] test_concurrent_futures failed

======================================================================
FAIL: test_pending_calls_race (test.test_concurrent_futures.ThreadPoolWaitTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\cygwin\home\db3l\buildarea\3.6.bolen-windows7\build\lib\test\test_concurrent_futures.py", line 96, in tearDown
    self.assertLess(dt, 60, "synchronization issue: test lasted too long")
AssertionError: 213.2027723789215 not less than 60 : synchronization issue: test lasted too long

----------
components: Tests, Windows
messages: 318090
nosy: paul.moore, steve.dower, tim.golden, vstinner, zach.ware
priority: normal
severity: normal
status: open
title: test_concurrent_futures: test_pending_calls_race() failed on x86 Windows7 3.6
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33686>
_______________________________________

From report at bugs.python.org  Tue May 29 18:02:07 2018
From: report at bugs.python.org (Poul-Henning Kamp)
Date: Tue, 29 May 2018 22:02:07 +0000
Subject: [New-bugs-announce] [issue33687] uu.py calls os.path.chmod which
 doesn't exist
Message-ID: <1527631327.84.0.682650639539.issue33687@psf.upfronthosting.co.za>


New submission from Poul-Henning Kamp <phk at FreeBSD.org>:

Library file uu.py on at least 2.7 and 3.6 contains:

            try:
                os.path.chmod(out_file, mode)
            except AttributeError:
                pass

As far as I can tell, os.path.chmod does not exist, so this always raises AttributeError and becomes a no-op.

Is suspect the proper fix is to remove the ".path" ?

----------
components: Library (Lib)
messages: 318119
nosy: bsdphk
priority: normal
severity: normal
status: open
title: uu.py calls os.path.chmod which doesn't exist
type: behavior
versions: Python 2.7, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33687>
_______________________________________

From report at bugs.python.org  Tue May 29 18:45:03 2018
From: report at bugs.python.org (Yury Selivanov)
Date: Tue, 29 May 2018 22:45:03 +0000
Subject: [New-bugs-announce] [issue33688] asyncio child watchers aren't fork
 friendly
Message-ID: <1527633903.57.0.682650639539.issue33688@psf.upfronthosting.co.za>


New submission from Yury Selivanov <yselivanov at gmail.com>:

_UnixDefaultEventLoopPolicy doesn't detect fork and tries to re-use the inherited child watcher. [1] is my idea how to fix this.

Andrew, AFAIK subprocesses was something that you worked on at some point. Do you have time to look into this?  Need tests and I'm not sure if I got [1] quite right.

This probably won't make it to 3.7.0, but 3.7.1 seems like a good target.

[1] https://github.com/1st1/cpython/commit/b4e6b26c565d9417a6f57a794a4669d7a7eeae88

----------
components: asyncio
messages: 318133
nosy: asvetlov, yselivanov
priority: normal
severity: normal
status: open
title: asyncio child watchers aren't fork friendly
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33688>
_______________________________________

From report at bugs.python.org  Tue May 29 19:03:14 2018
From: report at bugs.python.org (Malcolm Smith)
Date: Tue, 29 May 2018 23:03:14 +0000
Subject: [New-bugs-announce] [issue33689] Blank lines in .pth file cause a
 duplicate sys.path entry
Message-ID: <1527634994.48.0.682650639539.issue33689@psf.upfronthosting.co.za>


New submission from Malcolm Smith <malcolm.smith at gmail.com>:

The `site` module documentation says that in .pth files, "Blank lines and lines beginning with # are skipped.". However, the implementation does not actually skip blank lines. It then joins the empty string to the site-packages directory, resulting in the site-packages directory being added to sys.path a second time.

Example:

$ python -c 'import sys; print(sys.path)'
['', '/home/smith/.virtualenvs/foo/lib/python36.zip', '/home/smith/.virtualenvs/foo/lib/python3.6', '/home/smith/.virtualenvs/foo/lib/python3.6/lib-dynload', '/usr/lib/python3.6', '/home/smith/.virtualenvs/foo/lib/python3.6/site-packages']
$ echo > /home/smith/.virtualenvs/foo/lib/python3.6/site-packages/test.pth
$ python -c 'import sys; print(sys.path)'
['', '/home/smith/.virtualenvs/foo/lib/python36.zip', '/home/smith/.virtualenvs/foo/lib/python3.6', '/home/smith/.virtualenvs/foo/lib/python3.6/lib-dynload', '/usr/lib/python3.6', '/home/smith/.virtualenvs/foo/lib/python3.6/site-packages', '/home/smith/.virtualenvs/foo/lib/python3.6/site-packages']

A patch fixing this is attached to issue 29326, but was ignored when that issue turned out to be caused by something else.

----------
components: Library (Lib)
messages: 318137
nosy: Malcolm Smith
priority: normal
severity: normal
status: open
title: Blank lines in .pth file cause a duplicate sys.path entry
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33689>
_______________________________________

From report at bugs.python.org  Wed May 30 00:57:25 2018
From: report at bugs.python.org (Abhilash Raj)
Date: Wed, 30 May 2018 04:57:25 +0000
Subject: [New-bugs-announce] [issue33690] urlib.parse.urlencode with empty
 list and doseq=True drops the parameter
Message-ID: <1527656245.94.0.682650639539.issue33690@psf.upfronthosting.co.za>


New submission from Abhilash Raj <raj.abhilash1 at gmail.com>:

When using `urllib.parse.urlencode` it is impossible to pass an empty list when doseq=True.

So, I see the following behavior when using urlencode:

In [3]: urlencode({'key': ['value']}, doseq=True)
Out[3]: 'key=value'

In [4]: urlencode({'key': []}, doseq=True)
Out[4]: ''

So, in our source code, we do urlecode(params, doseq=True), which can have keys with empty list as values and it gets dropped completely. To get it to work properly, I instead pass around an empty string as a value when the length of list is 0, which conveys the server that the value of the key is empty.

I wonder if it would make sense for urlencode to do that by default given doseq=True is meant to encode lists properly.

----------
messages: 318153
nosy: maxking
priority: normal
severity: normal
status: open
title: urlib.parse.urlencode with empty list and doseq=True drops the parameter
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33690>
_______________________________________

From report at bugs.python.org  Wed May 30 01:34:34 2018
From: report at bugs.python.org (Serhiy Storchaka)
Date: Wed, 30 May 2018 05:34:34 +0000
Subject: [New-bugs-announce] [issue33691] Refactor docstring handling code
 in the compiler
Message-ID: <1527658474.71.0.682650639539.issue33691@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:

Similar non-trivial code is used in several places in the compiler for handling docstrings. The proposed PR adds a helper _PyAST_GetDocString() for reusing a code.

----------
components: Interpreter Core
messages: 318155
nosy: benjamin.peterson, brett.cannon, inada.naoki, ncoghlan, serhiy.storchaka, yselivanov
priority: normal
severity: normal
status: open
title: Refactor docstring handling code in the compiler
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33691>
_______________________________________

From report at bugs.python.org  Wed May 30 05:36:41 2018
From: report at bugs.python.org (Valentin Zhao)
Date: Wed, 30 May 2018 09:36:41 +0000
Subject: [New-bugs-announce] [issue33692] Chinese characters issue with
 input() function
Message-ID: <1527673001.64.0.682650639539.issue33692@psf.upfronthosting.co.za>


New submission from Valentin Zhao <jmsqzzl at gmail.com>:

DIRECTLY run(i.e., python xxx/xxx/input_test.py) the file below and input other Chinese characters, then press backspace to delete them. You will find that you cannot delete them all, and even the cursor in the terminal misplaces and overlaps with the characters.

This issue is only reproduced under Mac OS X, everything is fine with Windows.

----------
components: 2to3 (2.x to 3.x conversion tool)
files: input_test.py
messages: 318160
nosy: Valentin Zhao
priority: normal
severity: normal
status: open
title: Chinese characters issue with input() function
type: behavior
versions: Python 3.6
Added file: https://bugs.python.org/file47623/input_test.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33692>
_______________________________________

From report at bugs.python.org  Wed May 30 06:59:34 2018
From: report at bugs.python.org (Greg Walters)
Date: Wed, 30 May 2018 10:59:34 +0000
Subject: [New-bugs-announce] [issue33693] test test_webbrowser failed
Message-ID: <1527677974.5.0.682650639539.issue33693@psf.upfronthosting.co.za>


New submission from Greg Walters <greg.gregwa at gmail.com>:

Python 3.70b4
Linux Mint 18.3
Dell Latitude-E5500
While running make test got the following error...
======================================================================
ERROR: test_get (test.test_webbrowser.ImportTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/greg/Downloads/Python-3.7.0b4/Lib/test/test_webbrowser.py", line 290, in test_get
    webbrowser.get('fakebrowser')
  File "/home/greg/Downloads/Python-3.7.0b4/Lib/webbrowser.py", line 42, in get
    register_standard_browsers()
  File "/home/greg/Downloads/Python-3.7.0b4/Lib/webbrowser.py", line 567, in register_standard_browsers
    cmd = _synthesize(cmdline, -1)
  File "/home/greg/Downloads/Python-3.7.0b4/Lib/webbrowser.py", line 116, in _synthesize
    register(browser, None, controller, update_tryorder)
TypeError: register() takes from 2 to 3 positional arguments but 4 were given

======================================================================
ERROR: test_register (test.test_webbrowser.ImportTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/greg/Downloads/Python-3.7.0b4/Lib/test/test_webbrowser.py", line 277, in test_register
    webbrowser.register('Example1', ExampleBrowser)
  File "/home/greg/Downloads/Python-3.7.0b4/Lib/webbrowser.py", line 26, in register
    register_standard_browsers()
  File "/home/greg/Downloads/Python-3.7.0b4/Lib/webbrowser.py", line 567, in register_standard_browsers
    cmd = _synthesize(cmdline, -1)
  File "/home/greg/Downloads/Python-3.7.0b4/Lib/webbrowser.py", line 116, in _synthesize
    register(browser, None, controller, update_tryorder)
TypeError: register() takes from 2 to 3 positional arguments but 4 were given

----------------------------------------------------------------------
Ran 31 tests in 0.073s

FAILED (errors=2)
test test_webbrowser failed
1 test failed again:
    test_webbrowser

Total duration: 10 min 50 sec
Tests result: FAILURE
Makefile:1066: recipe for target 'test' failed
make: *** [test] Error 2


Did not try to go any further.

----------
components: Build
messages: 318163
nosy: Greg Walters
priority: normal
severity: normal
status: open
title: test test_webbrowser failed
type: compile error
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33693>
_______________________________________

From report at bugs.python.org  Wed May 30 07:19:38 2018
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 30 May 2018 11:19:38 +0000
Subject: [New-bugs-announce] [issue33694] test_asyncio:
 test_start_tls_server_1() fails on Python on x86 Windows7 3.7 and 3.x
Message-ID: <1527679178.26.0.682650639539.issue33694@psf.upfronthosting.co.za>


New submission from STINNER Victor <vstinner at redhat.com>:

test_asyncio.test_start_tls_server_1() got many fixes recently: see bpo-32458 and bpo-33674... but it still fails on Python on x86 Windows7 3.x at revision bb9474f1fb2fc7c7ed9f826b78262d6a12b5f9e8 which contains all these fixes.

The test fails even when test_asyncio is re-run alone (not when other tests run in parallel).

http://buildbot.python.org/all/#/builders/58/builds/930

======================================================================
ERROR: test_start_tls_server_1 (test.test_asyncio.test_sslproto.ProactorStartTLSTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\test_asyncio\test_sslproto.py", line 467, in test_start_tls_server_1
    self.loop.run_until_complete(run_main())
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\asyncio\base_events.py", line 566, in run_until_complete
    raise RuntimeError('Event loop stopped before Future completed.')
RuntimeError: Event loop stopped before Future completed.

======================================================================
FAIL: test_start_tls_server_1 (test.test_asyncio.test_sslproto.ProactorStartTLSTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\test_asyncio\functional.py", line 42, in tearDown
    self.fail('unexpected calls to loop.call_exception_handler()')
AssertionError: unexpected calls to loop.call_exception_handler()


The test fails also on x86 Windows7 3.7:


======================================================================
ERROR: test_start_tls_server_1 (test.test_asyncio.test_sslproto.ProactorStartTLSTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\cygwin\home\db3l\buildarea\3.7.bolen-windows7\build\lib\test\test_asyncio\test_sslproto.py", line 467, in test_start_tls_server_1
    self.loop.run_until_complete(run_main())
  File "D:\cygwin\home\db3l\buildarea\3.7.bolen-windows7\build\lib\asyncio\base_events.py", line 566, in run_until_complete
    raise RuntimeError('Event loop stopped before Future completed.')
RuntimeError: Event loop stopped before Future completed.

======================================================================
FAIL: test_start_tls_server_1 (test.test_asyncio.test_sslproto.ProactorStartTLSTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\cygwin\home\db3l\buildarea\3.7.bolen-windows7\build\lib\test\test_asyncio\functional.py", line 42, in tearDown
    self.fail('unexpected calls to loop.call_exception_handler()')
AssertionError: unexpected calls to loop.call_exception_handler()


Moreover, 3.7 got an additional failure:

======================================================================
ERROR: test_pipe_handle (test.test_asyncio.test_windows_utils.PipeTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\cygwin\home\db3l\buildarea\3.7.bolen-windows7\build\lib\test\test_asyncio\test_windows_utils.py", line 73, in test_pipe_handle
    raise RuntimeError('expected ERROR_INVALID_HANDLE')
RuntimeError: expected ERROR_INVALID_HANDLE


The Windows7 buildbot is known to be slow. For example, test_value() of test_multiprocessing_spawn failed with a timeout (15 min) on x86 Windows7 3.x:

0:48:12 [276/416/1] test_multiprocessing_spawn crashed (Exit code 1)
Timeout (0:15:00)!
Thread 0x000008e4 (most recent call first):
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\multiprocessing\popen_spawn_win32.py", line 80 in wait
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\multiprocessing\process.py", line 140 in join
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\_test_multiprocessing.py", line 1925 in test_value



Maybe something is wrong with my karma, I don't know. Or test_asyncio just hate me.

----------
components: Tests, asyncio
messages: 318164
nosy: asvetlov, vstinner, yselivanov
priority: normal
severity: normal
status: open
title: test_asyncio: test_start_tls_server_1() fails on Python on x86 Windows7 3.7 and 3.x
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33694>
_______________________________________

From report at bugs.python.org  Wed May 30 08:22:35 2018
From: report at bugs.python.org (Giampaolo Rodola')
Date: Wed, 30 May 2018 12:22:35 +0000
Subject: [New-bugs-announce] [issue33695] Have shutil.copytree(),
 copy() and copystat() use cached scandir() stat()s
Message-ID: <1527682955.69.0.682650639539.issue33695@psf.upfronthosting.co.za>


New submission from Giampaolo Rodola' <g.rodola at gmail.com>:

Patch in attachment makes shutil.copytree() use os.scandir() and (differently from #33414) DirEntry instances are passed around so that cached stat()s are used also from within copy2() and copystat() functions. The number of times the filesystem gets accessed via os.stat() is therefore reduced quite consistently. A similar improvement can be done for rmtree() (but that's for another ticket). Patch and benchmark script are in attachment.

Linux (+13.5% speedup)
======================

--- without patch:

    ./python  bench.py 
    Priming the system's cache...
    7956 files and dirs, repeat 1/3... min = 0.551s
    7956 files and dirs, repeat 2/3... min = 0.548s
    7956 files and dirs, repeat 3/3... min = 0.548s
    best result = 0.548s

--- with patch:

    $ ./python  bench.py 
    Priming the system's cache...
    7956 files and dirs, repeat 1/3... min = 0.481s
    7956 files and dirs, repeat 2/3... min = 0.479s
    7956 files and dirs, repeat 3/3... min = 0.474s
    best result = 0.474s

Windows (+17% speedup)
======================

--- without patch:

    ./python  bench.py 
    Priming the system's cache...
    7956 files and dirs, repeat 1/3... min = 9.015s
    7956 files and dirs, repeat 2/3... min = 8.747s
    7956 files and dirs, repeat 3/3... min = 8.614s
    best result = 8.614s

--- with patch:

    $ ./python  bench.py 
    Priming the system's cache...
    7956 files and dirs, repeat 1/3... min = 7.827s
    7956 files and dirs, repeat 2/3... min = 7.369s
    7956 files and dirs, repeat 3/3... min = 7.153s
    best result = 7.153s

Windows SMB share (+30%)
========================

--- without patch:

    C:\Users\user\Desktop\cpython>PCbuild\win32\python.exe bench.py
    Priming the system's cache...
    7956 files and dirs, repeat 1/3... min = 46.853s
    7956 files and dirs, repeat 2/3... min = 46.330s
    7956 files and dirs, repeat 3/3... min = 44.720s
    best result = 44.720s

--- with patch:

    C:\Users\user\Desktop\cpython>PCbuild\win32\python.exe bench.py
    Priming the system's cache...
    7956 files and dirs, repeat 1/3... min = 31.729s
    7956 files and dirs, repeat 2/3... min = 30.936s
    7956 files and dirs, repeat 3/3... min = 30.936s
    best result = 30.936s

Number of stat() syscalls (-38%)
================================

--- without patch:

    $ strace ./python bench.py  2>&1 | grep "stat(" | wc -l
    324808
    
--- with patch:

    $ strace ./python bench.py  2>&1 | grep "stat(" | wc -l
    198768

----------
components: Library (Lib)
files: bench.py
messages: 318175
nosy: giampaolo.rodola
priority: normal
severity: normal
stage: patch review
status: open
title: Have shutil.copytree(), copy() and copystat() use cached scandir() stat()s
type: performance
versions: Python 3.8
Added file: https://bugs.python.org/file47624/bench.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33695>
_______________________________________

From report at bugs.python.org  Wed May 30 09:20:02 2018
From: report at bugs.python.org (=?utf-8?q?Andr=C3=A9s_Delfino?=)
Date: Wed, 30 May 2018 13:20:02 +0000
Subject: [New-bugs-announce] [issue33696] Install python-docs-theme even if
 SPHINXBUILD is defined
Message-ID: <1527686402.52.0.682650639539.issue33696@psf.upfronthosting.co.za>


New submission from Andr?s Delfino <adelfino at gmail.com>:

On bug 33673, I fixed python-docs-theme only getting installed if SPHINXBUILD was not defined and Sphinx was not installed, which was an improvement, but not the correct fix for python-docs-theme installation.

Now, python-docs-theme is getting installed only if SPHINXBUILD is not defined.

The attached PR fixes this, and also removes two variable assignments that are no-ops: both variables are already defined by the user as environment variables or by the script at that point.

----------
components: Windows
messages: 318183
nosy: adelfino, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Install python-docs-theme even if SPHINXBUILD is defined
type: enhancement
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33696>
_______________________________________

From report at bugs.python.org  Wed May 30 10:30:59 2018
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 30 May 2018 14:30:59 +0000
Subject: [New-bugs-announce] [issue33697] AppVeyor:
Message-ID: <1527690659.62.0.682650639539.issue33697@psf.upfronthosting.co.za>


New submission from STINNER Victor <vstinner at redhat.com>:

On my PR 7244, test_zipfile.test_write_filtered_python_package() failed on AppVeyor.

https://ci.appveyor.com/project/python/cpython/build/3.6build16559/job/3lqiymop38oorsn0

======================================================================
ERROR: test_write_filtered_python_package (test.test_zipfile.PyZipFileTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\projects\cpython\lib\test\test_zipfile.py", line 851, in test_write_filtered_python_package
    zipfp.writepy(packagedir)
  File "C:\projects\cpython\lib\zipfile.py", line 1849, in writepy
    filterfunc=filterfunc)  # Recursive call
  File "C:\projects\cpython\lib\zipfile.py", line 1849, in writepy
    filterfunc=filterfunc)  # Recursive call
  File "C:\projects\cpython\lib\zipfile.py", line 1856, in writepy
    basename)
  File "C:\projects\cpython\lib\zipfile.py", line 1935, in _get_codename
    if _compile(file_py):
  File "C:\projects\cpython\lib\zipfile.py", line 1898, in _compile
    py_compile.compile(file, doraise=True, optimize=optimize)
  File "C:\projects\cpython\lib\py_compile.py", line 143, in compile
    importlib._bootstrap_external._write_atomic(cfile, bytecode, mode)
  File "<frozen importlib._bootstrap_external>", line 119, in _write_atomic
PermissionError: [WinError 5] Access is denied: 'C:\\projects\\cpython\\lib\\test\\test_importlib\\source\\__pycache__\\test_path_hook.cpython-36.pyc.44848936' -> 'C:\\projects\\cpython\\lib\\test\\test_importlib\\source\\__pycache__\\test_path_hook.cpython-36.pyc'

----------
components: Tests, Windows
messages: 318188
nosy: paul.moore, steve.dower, tim.golden, vstinner, zach.ware
priority: normal
severity: normal
status: open
title: AppVeyor:
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33697>
_______________________________________

From report at bugs.python.org  Wed May 30 13:12:41 2018
From: report at bugs.python.org (Vladimir Chebotarev)
Date: Wed, 30 May 2018 17:12:41 +0000
Subject: [New-bugs-announce] [issue33698] `._pth` does not allow to populate
 `sys.path` empty entry
Message-ID: <1527700361.05.0.682650639539.issue33698@psf.upfronthosting.co.za>


New submission from Vladimir Chebotarev <vladimir.chebotarev at gmail.com>:

Moving from bpo-29326
====
Hi Steve.

I'll try to explain what is my motivation.

I need a reliable way to run Python (not matter embedded or not) in isolated mode, but still having current directory in `sys.path` (empty entry).

Ironically I could misuse normal mode to simulate isolated mode with specified %PYTHONPATH% (which is not much handy, by the way), but hiding Windows Registry keys would look pretty hard (it is impossible to make a bat/cmd wrapper). Possibility to search %PYTHONPATH% in HKLM entries makes it even worse.

Empty entry in `sys.path` is a extremely popular thing, and many software depend on it. It looks like a bug that isolated mode cannot support such entry.

# How would I fix it.

Possibly we should allow another keyword for adding an empty entry (like we have an exclusion for `import site`).

Ideas are appreciated.

----------
assignee: docs at python
components: Documentation, Windows
messages: 318202
nosy: docs at python, excitoon, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
pull_requests: 6878
severity: normal
status: open
title: `._pth` does not allow to populate `sys.path` empty entry
type: behavior
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33698>
_______________________________________

From report at bugs.python.org  Wed May 30 15:39:42 2018
From: report at bugs.python.org (=?utf-8?q?Andr=C3=A9s_Delfino?=)
Date: Wed, 30 May 2018 19:39:42 +0000
Subject: [New-bugs-announce] [issue33699] Don't describe try's else clause
 in a footnote
Message-ID: <1527709182.68.0.682650639539.issue33699@psf.upfronthosting.co.za>


New submission from Andr?s Delfino <adelfino at gmail.com>:

This behavior has not changed in 11 years so I think it's safe to describe the "else" clause along the rest of the "try" clause.

Attached PR fixes this.

----------
assignee: docs at python
components: Documentation
messages: 318205
nosy: adelfino, docs at python
priority: normal
severity: normal
status: open
title: Don't describe try's else clause in a footnote
type: enhancement
versions: Python 2.7, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33699>
_______________________________________

From report at bugs.python.org  Wed May 30 17:44:23 2018
From: report at bugs.python.org (Julien Palard)
Date: Wed, 30 May 2018 21:44:23 +0000
Subject: [New-bugs-announce] [issue33700] [doc] Old version picker don't
 understand language tags in URL
Message-ID: <1527716663.86.0.682650639539.issue33700@psf.upfronthosting.co.za>


New submission from Julien Palard <julien+python at palard.fr>:

In the old version picker used back in 3.5 [1], the function patch_url was patching using the following regex:

    var url_re = /\.org\/(\d|py3k|dev|((release\/)?\d\.\d[\w\d\.]*))\//,

which does not expects a language tag.

This means, for example, that being on https://docs.python.org/fr/3.5/library/__main__.html and switching to 3.6 will fail (but the reverse will succeed as the implementation of the switcher in 3.6 understands the language tag).

I see two potential fixes:

- Fix in the 3.5 branch which is not allowed as the branch is in "security only" mode.
- Add a rule, nginx-side, to fix the misconstructed URLS (looking like /3.6/fr/3.5/library/...) redirectin g them to the right place.

Other ideas and feedback welcome.


[1]: https://docs.python.org/fr/3.5/_static/version_switch.js

----------
assignee: mdk
components: Documentation
messages: 318210
nosy: mdk
priority: low
severity: normal
stage: needs patch
status: open
title: [doc] Old version picker don't understand language tags in URL
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33700>
_______________________________________

From report at bugs.python.org  Wed May 30 18:34:01 2018
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 30 May 2018 22:34:01 +0000
Subject: [New-bugs-announce] [issue33701] test_datetime crashed (SIGSEGV) on
 Travis CI
Message-ID: <1527719641.19.0.682650639539.issue33701@psf.upfronthosting.co.za>


New submission from STINNER Victor <vstinner at redhat.com>:

https://travis-ci.org/python/cpython/jobs/385458840

0:00:11 load avg: 22.29 [ 23/415/1] test_datetime crashed (Exit code -11)
Fatal Python error: Segmentation fault
Current thread 0x00002b52f7326400 (most recent call first):
  File "/home/travis/build/python/cpython/Lib/test/datetimetester.py", line 4630 in test_check_arg_types
  File "/home/travis/build/python/cpython/Lib/unittest/case.py", line 615 in run
  File "/home/travis/build/python/cpython/Lib/unittest/case.py", line 663 in __call__
  File "/home/travis/build/python/cpython/Lib/unittest/suite.py", line 122 in run
  File "/home/travis/build/python/cpython/Lib/unittest/suite.py", line 84 in __call__
  File "/home/travis/build/python/cpython/Lib/unittest/suite.py", line 122 in run
  File "/home/travis/build/python/cpython/Lib/unittest/suite.py", line 84 in __call__
  File "/home/travis/build/python/cpython/Lib/unittest/runner.py", line 176 in run
  (...)

Lib/test/datetimetester.py:4630:

    def test_check_arg_types(self):
        class Number:
            def __init__(self, value):
                self.value = value
            def __int__(self):
                return self.value

        for xx in [decimal.Decimal(10),    <~~~ HERE
                   decimal.Decimal('10.9'),
                   Number(10)]:
            self.assertEqual(datetime(10, 10, 10, 10, 10, 10, 10),
                             datetime(xx, xx, xx, xx, xx, xx, xx))


Hum, this bug looks like bpo-33627.

Is it a bug in the _decimal module?

----------
components: XML
messages: 318216
nosy: facundobatista, mark.dickinson, rhettinger, skrah, vstinner
priority: normal
severity: normal
status: open
title: test_datetime crashed (SIGSEGV) on Travis CI
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33701>
_______________________________________

From report at bugs.python.org  Wed May 30 19:51:42 2018
From: report at bugs.python.org (=?utf-8?q?Andr=C3=A9s_Delfino?=)
Date: Wed, 30 May 2018 23:51:42 +0000
Subject: [New-bugs-announce] [issue33702] Add some missings links in
 production lists and a little polish
Message-ID: <1527724302.5.0.682650639539.issue33702@psf.upfronthosting.co.za>


New submission from Andr?s Delfino <adelfino at gmail.com>:

There are some missing links in production lists (try, with, from __future__ import), and function and coroutines definitions production lists have too long lines.

PR fixes this and does some polish on whitespace.

----------
assignee: docs at python
components: Documentation
messages: 318231
nosy: adelfino, docs at python
priority: normal
severity: normal
status: open
title: Add some missings links in production lists and a little polish
type: enhancement
versions: Python 2.7, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33702>
_______________________________________

From report at bugs.python.org  Thu May 31 00:18:48 2018
From: report at bugs.python.org (Nitin Bhojwani)
Date: Thu, 31 May 2018 04:18:48 +0000
Subject: [New-bugs-announce] [issue33703] Object deletion and re-creation
 points to same attribute if attribute is a dictionary.
Message-ID: <1527740328.11.0.682650639539.issue33703@psf.upfronthosting.co.za>


New submission from Nitin Bhojwani <nitinbhojwani1993 at gmail.com>:

I observed a strange behaviour while working on a module.
This bug is present in multiple python version so seems it is related to core-interpreter.

Behaviour can be reproduced by:
1. Create an object and in __init__ of class, we define a dictionary attribute to that object.
2. After that we change that attribute, like assign any key-value(s) to that attribute.
3. Delete the object.
4. recreate the object.
5. If you check, this new object's same attribute(dictionary) is same old attribute which was pointed by the older object.

This is a behavioural bug, as once delete is performed on an object the whole object and it's (attributes') references must be removed.

I can demonstrate this using a test from unittest, in which I put instance creation in setUp phase, instance deletion in tearDown phase and run same test multiple times. The result is different each time.

The zip file contains ngraph.py which defines a class and test_util.py which runs test against it.
Unzip and execute as:

python test_util.py #2.7.10
python3 test_util.py #3.6.3

----------
components: Interpreter Core
files: test-bug.zip
messages: 318241
nosy: Nitin Bhojwani
priority: normal
severity: normal
status: open
title: Object deletion and re-creation points to same attribute if attribute is a dictionary.
type: behavior
versions: Python 3.6
Added file: https://bugs.python.org/file47627/test-bug.zip

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33703>
_______________________________________

From report at bugs.python.org  Thu May 31 00:28:43 2018
From: report at bugs.python.org (Saba Kauser)
Date: Thu, 31 May 2018 04:28:43 +0000
Subject: [New-bugs-announce] [issue33704] python 3.7 and python 3.8 stable
 release
Message-ID: <1527740923.47.0.682650639539.issue33704@psf.upfronthosting.co.za>


New submission from Saba Kauser <skauseribmdb at gmail.com>:

Hi,
I am planning to make new release of python ibm_db driver and I wanted to know the stable versions of python beyond python 3.6.

I can see python 3.8 marked as "in development" and I do read that python 3.7.0b4 is the final planned beta release preview. Would this mean that I cannot consider python 3.7 and python 3.8 for my next release and I would build only when these are marked stable.

Thank you!
Saba.

----------
components: Build
messages: 318242
nosy: sabakauser
priority: normal
severity: normal
status: open
title: python 3.7 and python 3.8 stable release
type: behavior
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33704>
_______________________________________

From report at bugs.python.org  Thu May 31 00:53:09 2018
From: report at bugs.python.org (Steven D'Aprano)
Date: Thu, 31 May 2018 04:53:09 +0000
Subject: [New-bugs-announce] [issue33705] Unicode is normalised after
 keywords are checked for
Message-ID: <1527742389.0.0.682650639539.issue33705@psf.upfronthosting.co.za>


New submission from Steven D'Aprano <steve+python at pearwood.info>:

There is a loophole in the Unicode normalisation which allows the creation of names matching keywords.

class Spam:
    locals()['if'] = 1

Spam.??    # U+1D422 U+1D41F
# returns 1

Those two characters are 'MATHEMATICAL BOLD SMALL I' and 'MATHEMATICAL BOLD SMALL F'. They ought to be normalised to "if", which is a keyword.

Of course Spam.if is a syntax error, and I believe Spam.?? ought to be as well.

Another example:

py> globals()['for'] = 2
py> ?or
2



I also asked about this here:

https://mail.python.org/pipermail/python-dev/2018-May/153619.html

----------
components: Interpreter Core, Unicode
messages: 318250
nosy: ezio.melotti, steven.daprano, vstinner
priority: normal
severity: normal
status: open
title: Unicode is normalised after keywords are checked for
type: behavior

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33705>
_______________________________________

From report at bugs.python.org  Thu May 31 02:31:10 2018
From: report at bugs.python.org (Christoph Gohlke)
Date: Thu, 31 May 2018 06:31:10 +0000
Subject: [New-bugs-announce] [issue33706] Segfault in command line
 processing due to buffer over-read
Message-ID: <1527748270.57.0.682650639539.issue33706@psf.upfronthosting.co.za>


New submission from Christoph Gohlke <cgohlke at uci.edu>:

When testing Python 3.7.0b5 x64 (and betas before) on Windows 10, I occasionally get segfaults when passing a program as string on the command line. The shortest command to reproduce this on my system is `python.exe -c 1` with heap detection turned on but that might not be reproducible on other systems.

It turns out there is a buffer over-read when copying the `PyOS_optarg` string in `main.c`. The attached patch fixes the segfault on my system.

----------
files: master...cgohlke patch-1.diff
keywords: patch
messages: 318260
nosy: cgohlke
priority: normal
severity: normal
status: open
title: Segfault in command line processing due to buffer over-read
versions: Python 3.7
Added file: https://bugs.python.org/file47628/master...cgohlke patch-1.diff

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33706>
_______________________________________

From report at bugs.python.org  Thu May 31 04:46:40 2018
From: report at bugs.python.org (=?utf-8?b?0JzQsNGA0Log0JrQvtGA0LXQvdCx0LXRgNCz?=)
Date: Thu, 31 May 2018 08:46:40 +0000
Subject: [New-bugs-announce] [issue33707] Doc:
Message-ID: <1527756400.46.0.682650639539.issue33707@psf.upfronthosting.co.za>


New submission from ???? ????????? <socketpair at gmail.com>:

https://docs.python.org/3/library/asyncio-sync.html#asyncio.Event :

Class implementing event objects. An event manages a flag that can be set to true with the set() method and reset to false with the clear() method. The ----->  wait()  <----- method blocks until the flag is true. The flag is initially false.

this link points to wrong place.

----------
assignee: docs at python
components: Documentation, asyncio
messages: 318268
nosy: asvetlov, docs at python, socketpair, yselivanov
priority: normal
severity: normal
status: open
title: Doc:
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33707>
_______________________________________

From report at bugs.python.org  Thu May 31 04:48:14 2018
From: report at bugs.python.org (=?utf-8?b?0JzQsNGA0Log0JrQvtGA0LXQvdCx0LXRgNCz?=)
Date: Thu, 31 May 2018 08:48:14 +0000
Subject: [New-bugs-announce] [issue33708] Doc: Asyncio's Event documentation
 typo.
Message-ID: <1527756494.64.0.682650639539.issue33708@psf.upfronthosting.co.za>


New submission from ???? ????????? <socketpair at gmail.com>:

https://docs.python.org/3/library/asyncio-sync.html#asyncio.Event :

Class implementing event objects. An event manages a flag that can be set to true with the set() method and reset to false with the clear() method. The ----->  wait()  <----- method blocks until the flag is true. The flag is initially false.

this link points to wrong place.

----------
assignee: docs at python
components: Documentation, asyncio
messages: 318270
nosy: asvetlov, docs at python, socketpair, yselivanov
priority: normal
severity: normal
status: open
title: Doc: Asyncio's Event documentation typo.
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33708>
_______________________________________

From report at bugs.python.org  Thu May 31 04:54:10 2018
From: report at bugs.python.org (Ivan Pozdeev)
Date: Thu, 31 May 2018 08:54:10 +0000
Subject: [New-bugs-announce] [issue33709] test.support.FS_NONASCII returns
 incorrect result in Windows with non-US locale
Message-ID: <1527756850.06.0.682650639539.issue33709@psf.upfronthosting.co.za>


New submission from Ivan Pozdeev <ivan_pozdeev at mail.ru>:

This causes test_ntpath and test_posixpath to fail in subj.

Sample failure:

======================================================================
FAIL: test_expandvars_nonascii (__main__.NtCommonTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "c:\Users\Sasha\Documents\cpython\lib\test\test_genericpath.py", line 230
, in test_expandvars_nonascii
    check(u'$spam bar', u'%s bar' % unonascii)
  File "c:\Users\Sasha\Documents\cpython\lib\test\test_genericpath.py", line 214
, in check
    self.assertEqual(expandvars(value), expected)
AssertionError: u'? bar' != u'\xe6 bar'
- ? bar
? ^
+ \xe6 bar
? ^


Cause:

if sys.getfilesystemencoding()=='mbcs', encoding Unicode characters that are missing in the current locale succeeds but produces '?'.

So, test.support.FS_NONASCII's test fails to detect if a character is present in the current locale.

----------
components: Tests
messages: 318271
nosy: Ivan.Pozdeev
priority: normal
severity: normal
status: open
title: test.support.FS_NONASCII returns incorrect result in Windows with non-US locale
type: behavior
versions: Python 2.7, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33709>
_______________________________________

From report at bugs.python.org  Thu May 31 07:00:56 2018
From: report at bugs.python.org (Serhiy Storchaka)
Date: Thu, 31 May 2018 11:00:56 +0000
Subject: [New-bugs-announce] [issue33710] Deprecate gettext.lgettext()
Message-ID: <1527764456.73.0.682650639539.issue33710@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:

Using gettext.lgettext() is the one of two right ways of doing translation in Python 2. In Python 2, gettext.gettext() returns a raw 8-bit string as it was written in the translation file. Since different translation files can use different encodings, and the locale encoding can be different from them, gettext.gettext() usually is not appropriate. gettext.lgettext() re-encodes all translated messages from file encodings to the specified one (or to the locale encoding by default). It works properly for str-based messages. Other right way is using gettext.ugettext() which returns a Unicode string.

In Python 3 gettext.gettext() was removed, and gettext.ugettext() was renamed to gettext.gettext(). This is the single right way. gettext.lgettext() still returns messages encoded to bytes, but since virtually all messages are Unicode strings in Python 3, it is virtually useless. At least I don't know any proper use case for it. In addition, gettext.lgettext() was half-broken up to recent times (see issue29755).

Seems gettext.lgettext() was not removed in Python 3.0 just due to an oversight. I suggest to deprecate it in 3.8 and remove it in future versions.

----------
components: Library (Lib)
messages: 318283
nosy: barry, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Deprecate gettext.lgettext()
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33710>
_______________________________________

From report at bugs.python.org  Thu May 31 07:13:44 2018
From: report at bugs.python.org (Ivan Pozdeev)
Date: Thu, 31 May 2018 11:13:44 +0000
Subject: [New-bugs-announce] [issue33711] Could not find externals/db-* in
 msi.py on license generation
Message-ID: <1527765224.13.0.682650639539.issue33711@psf.upfronthosting.co.za>


New submission from Ivan Pozdeev <ivan_pozdeev at mail.ru>:

c:\Users\Sasha\Documents\cpython\Tools\msi>python msi.py
Traceback (most recent call last):
  File "msi.py", line 1372, in <module>
    add_files(db)
  File "msi.py", line 956, in add_files
    generate_license()
  File "msi.py", line 928, in generate_license
    raise ValueError, "Could not find "+srcdir+"/externals/"+pat
ValueError: Could not find c:\Users\Sasha\Documents\cpython/externals/db-*

The dir in externals is actually called bsddb-<version>.


I've no idea how they can official releases

----------
components: Build
messages: 318284
nosy: Ivan.Pozdeev
priority: normal
pull_requests: 6907
severity: normal
status: open
title: Could not find externals/db-* in msi.py on license generation
type: behavior
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33711>
_______________________________________

From report at bugs.python.org  Thu May 31 08:08:48 2018
From: report at bugs.python.org (Serhiy Storchaka)
Date: Thu, 31 May 2018 12:08:48 +0000
Subject: [New-bugs-announce] [issue33712] OrderedDict can set an exception
 in tp_clear
Message-ID: <1527768528.87.0.682650639539.issue33712@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:

The tp_clear handler of OrderedDict can set an exception when fail to allocate a nodes cache for an empty dict. An exception in tp_clear is not expected and caused a crash in the garbage collector. In the master branch it will cause just writing a traceback to stderr (see issue33622), but in any case it would be better to handle the failure locally in OrderedDict. Perhaps the cache for an empty dict is not needed.

----------
components: Interpreter Core
messages: 318287
nosy: eric.snow, inada.naoki, serhiy.storchaka
priority: normal
severity: normal
status: open
title: OrderedDict can set an exception in tp_clear
type: crash
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33712>
_______________________________________

From report at bugs.python.org  Thu May 31 08:15:10 2018
From: report at bugs.python.org (Serhiy Storchaka)
Date: Thu, 31 May 2018 12:15:10 +0000
Subject: [New-bugs-announce] [issue33713] memoryview can set an exception in
 tp_clear
Message-ID: <1527768910.92.0.682650639539.issue33713@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:

The tp_clear handler of memoryview can set an exception when fail to release the buffer. An exception in tp_clear is not expected and caused a crash in the garbage collector. In the master branch it will cause just writing a traceback to stderr (see issue33622), but in any case it would be better to handle the failure locally in memoryview. I don't know what is the best solution: silencing an error, writing a traceback with more detailed information, or resurrecting the buffer object.

----------
components: Interpreter Core
messages: 318288
nosy: serhiy.storchaka, skrah
priority: normal
severity: normal
status: open
title: memoryview can set an exception in tp_clear
type: crash
versions: Python 2.7, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33713>
_______________________________________

From report at bugs.python.org  Thu May 31 08:21:45 2018
From: report at bugs.python.org (Serhiy Storchaka)
Date: Thu, 31 May 2018 12:21:45 +0000
Subject: [New-bugs-announce] [issue33714] module can set an exception in
 tp_clear
Message-ID: <1527769305.39.0.682650639539.issue33714@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:

The tp_clear handler of the module object calls a custom clear function if the PyModuleDef.m_clear field is set. This function can set an exception which will be leaked to the garbage collector. An exception in tp_clear is not expected and caused a crash in the garbage collector. In the master branch it will cause just writing a traceback to stderr (see issue33622), but in any case it would be better to handle the failure locally in the module's tp_clear.

----------
components: Interpreter Core
messages: 318289
nosy: serhiy.storchaka
priority: normal
severity: normal
status: open
title: module can set an exception in tp_clear
type: crash
versions: Python 2.7, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33714>
_______________________________________

From report at bugs.python.org  Thu May 31 08:36:45 2018
From: report at bugs.python.org (STINNER Victor)
Date: Thu, 31 May 2018 12:36:45 +0000
Subject: [New-bugs-announce] [issue33715]
 test_multiprocessing_spawn.test_wait_result() failed on x86 Windows7 3.x
Message-ID: <1527770205.15.0.682650639539.issue33715@psf.upfronthosting.co.za>


New submission from STINNER Victor <vstinner at redhat.com>:

http://buildbot.python.org/all/#/builders/58/builds/932

======================================================================
FAIL: test_wait_result (test.test_multiprocessing_spawn.WithManagerTestCondition)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\_test_multiprocessing.py", line 1485, in test_wait_result
    self.assertTrue(c.wait(10))
AssertionError: False is not true

Note: this buildbot is known to be slow.

See also bpo-30317 and bpo-30356. (And maybe also bpo-31687.)

----------
components: Tests, Windows
messages: 318292
nosy: davin, paul.moore, pitrou, steve.dower, tim.golden, vstinner, zach.ware
priority: normal
severity: normal
status: open
title: test_multiprocessing_spawn.test_wait_result() failed on x86 Windows7 3.x
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33715>
_______________________________________

From report at bugs.python.org  Thu May 31 08:41:13 2018
From: report at bugs.python.org (STINNER Victor)
Date: Thu, 31 May 2018 12:41:13 +0000
Subject: [New-bugs-announce] [issue33716]
 test_concurrent_futures.test_crash() failed on x86 Windows7 3.7
Message-ID: <1527770473.78.0.682650639539.issue33716@psf.upfronthosting.co.za>


New submission from STINNER Victor <vstinner at redhat.com>:

x86 Windows7 3.7:
http://buildbot.python.org/all/#/builders/111/builds/299

test_crash (test.test_concurrent_futures.ProcessPoolSpawnExecutorDeadlockTest) ... 26.57s ok
...
test_crash (test.test_concurrent_futures.ProcessPoolSpawnExecutorDeadlockTest) ... 90.96s FAIL
...

======================================================================
FAIL: test_crash (test.test_concurrent_futures.ProcessPoolSpawnExecutorDeadlockTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\cygwin\home\db3l\buildarea\3.7.bolen-windows7\build\lib\test\test_concurrent_futures.py", line 131, in tearDown
    self.assertLess(dt, 60, "synchronization issue: test lasted too long")
AssertionError: 90.95560574531555 not less than 60 : synchronization issue: test lasted too long


This buildbot is known to be slow.

See also bpo-33715.

----------
components: Tests, Windows
messages: 318294
nosy: paul.moore, steve.dower, tim.golden, vstinner, zach.ware
priority: normal
severity: normal
status: open
title: test_concurrent_futures.test_crash() failed on x86 Windows7 3.7
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33716>
_______________________________________

From report at bugs.python.org  Thu May 31 09:42:39 2018
From: report at bugs.python.org (STINNER Victor)
Date: Thu, 31 May 2018 13:42:39 +0000
Subject: [New-bugs-announce] [issue33717] Enhance test.pythinfo: meta-ticket
 for multiple changes
Message-ID: <1527774159.9.0.682650639539.issue33717@psf.upfronthosting.co.za>


New submission from STINNER Victor <vstinner at redhat.com>:

Lib/test/pythoninfo.py is a tool to dump informations to help to debug test failures.

I create this issue to have a bpo number for future enhancements. For example, I now would like to get the version of the C compiler.

----------
components: Tests
messages: 318309
nosy: vstinner
priority: normal
severity: normal
status: open
title: Enhance test.pythinfo: meta-ticket for multiple changes
type: enhancement
versions: Python 2.7, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33717>
_______________________________________

From report at bugs.python.org  Thu May 31 10:05:39 2018
From: report at bugs.python.org (STINNER Victor)
Date: Thu, 31 May 2018 14:05:39 +0000
Subject: [New-bugs-announce] [issue33718] Enhance regrtest: meta-ticket for
 multiple changes
Message-ID: <1527775539.21.0.682650639539.issue33718@psf.upfronthosting.co.za>


New submission from STINNER Victor <vstinner at redhat.com>:

Lib/test/libregrtest/ (python3 -m test) is the test runner used to run the Python test suite.

I create this issue to have a bpo number for future enhancements.

Example of recent enhancements:

* https://github.com/python/cpython/pull/7105
* https://github.com/python/cpython/pull/7159

This issue should also help me to make sure that I backport enhancements from master to 2.7, 3.6 and 3.7 branches.

----------
messages: 318314
nosy: vstinner
priority: normal
severity: normal
status: open
title: Enhance regrtest: meta-ticket for multiple changes
type: enhancement
versions: Python 2.7, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33718>
_______________________________________

From report at bugs.python.org  Thu May 31 10:53:06 2018
From: report at bugs.python.org (STINNER Victor)
Date: Thu, 31 May 2018 14:53:06 +0000
Subject: [New-bugs-announce] [issue33719] Test failures on Python 3.7 beta 5
 and Windows 10
Message-ID: <1527778386.67.0.682650639539.issue33719@psf.upfronthosting.co.za>


New submission from STINNER Victor <vstinner at redhat.com>:

I ran the test suite on an installed Python 3.7 beta5 on Windows 10. The following tests fail:

* test_marshal.test_loads_2x_code(): Windows fatal exception: stack overflow
* test_pkg: failures when run with -X utf8
* test.test_tools.test_sundry.TestSundryScripts.test_sundry(): "optparse.BadOptionError: no such option: -m"

== test_marshal ==

C:\Users\vstinner\AppData\Local\Programs\Python\Python37>python.exe -m test test_marshal -v
== CPython 3.7.0b5 (v3.7.0b5:abb8802389, May 31 2018, 01:54:01) [MSC v.1913 64 bit (AMD64)]
== Windows-10-10.0.16299-SP0 little-endian
== cwd: C:\Users\vstinner\AppData\Local\Temp\test_python_3836
== CPU count: 2
== encodings: locale=cp1252, FS=utf-8
Run tests sequentially
0:00:00 [1/1] test_marshal
(...)
test_loads_2x_code (test.test_marshal.BugsTestCase) ... Windows fatal exception: stack overflow

Current thread 0x000003a0 (most recent call first):
  File "C:\Users\vstinner\AppData\Local\Programs\Python\Python37\lib\unittest\case.py", line 178 in handle
  File "C:\Users\vstinner\AppData\Local\Programs\Python\Python37\lib\unittest\case.py", line 743 in assertRaises
  File "C:\Users\vstinner\AppData\Local\Programs\Python\Python37\lib\test\test_marshal.py", line 215 in test_loads_2x_code
  (...)

== test_pkg ==

C:\Users\vstinner\AppData\Local\Programs\Python\Python37>python.exe -X utf8 -m test -v test_pkg
== CPython 3.7.0b5 (v3.7.0b5:abb8802389, May 31 2018, 01:54:01) [MSC v.1913 64 bit (AMD64)]
== Windows-10-10.0.16299-SP0 little-endian
== cwd: C:\Users\vstinner\AppData\Local\Temp\test_python_6984
== CPU count: 2
== encodings: locale=UTF-8, FS=utf-8
Run tests sequentially
0:00:00 [1/1] test_pkg
test_1 (test.test_pkg.TestPkg) ... ok
test_2 (test.test_pkg.TestPkg) ... ok
test_3 (test.test_pkg.TestPkg) ... ok
test_4 (test.test_pkg.TestPkg) ... ERROR
test_5 (test.test_pkg.TestPkg) ... ok
test_6 (test.test_pkg.TestPkg) ... ok
test_7 (test.test_pkg.TestPkg) ... FAIL
test_8 (test.test_pkg.TestPkg) ... ok

======================================================================
ERROR: test_4 (test.test_pkg.TestPkg)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\vstinner\AppData\Local\Programs\Python\Python37\lib\test\test_pkg.py", line 180, in test_4
    self.run_code(s)
  File "C:\Users\vstinner\AppData\Local\Programs\Python\Python37\lib\test\test_pkg.py", line 69, in run_code
    exec(textwrap.dedent(code), globals(), {"self": self})
  File "<string>", line 2, in <module>
  File "C:\Users\vstinner\AppData\Local\Temp\tmpeeb0h5ra\t4.py", line 1, in <module>
RuntimeError: Shouldnt load t4.py

======================================================================
FAIL: test_7 (test.test_pkg.TestPkg)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\vstinner\AppData\Local\Programs\Python\Python37\lib\test\test_pkg.py", line 260, in test_7
    '__name__', '__package__', '__path__', '__spec__'])
AssertionError: Lists differ: ['__c[34 chars]__loader__', '__name__', '__package__', '__spec__'] != ['__c[34 chars]__loader__', '__name__', '__package__', '__path__', '__spec__']

First differing element 6:
'__spec__'
'__path__'

Second list contains 1 additional elements.
First extra element 7:
'__spec__'

  ['__cached__',
   '__doc__',
   '__file__',
   '__loader__',
   '__name__',
   '__package__',
+  '__path__',
   '__spec__']

----------------------------------------------------------------------
Ran 8 tests in 0.156s

FAILED (failures=1, errors=1)
test test_pkg failed
test_pkg failed

1 test failed:
    test_pkg

Total duration: 187 ms
Tests result: FAILURE

== test_tools ==


C:\Users\vstinner\AppData\Local\Programs\Python\Python37>python.exe -m test -v test_tools -m test_sundry
== CPython 3.7.0b5 (v3.7.0b5:abb8802389, May 31 2018, 01:54:01) [MSC v.1913 64 bit (AMD64)]
== Windows-10-10.0.16299-SP0 little-endian
== cwd: C:\Users\vstinner\AppData\Local\Temp\test_python_7472
== CPU count: 2
== encodings: locale=cp1252, FS=utf-8
Run tests sequentially
0:00:00 [1/1] test_tools
test_analyze_dxp_import (test.test_tools.test_sundry.TestSundryScripts) ... ok
test_sundry (test.test_tools.test_sundry.TestSundryScripts) ... Usage: 2to3 [options] file|dir ...

__main__.py: error: no such option: -m
ERROR
test_sundry_windows (test.test_tools.test_sundry.TestSundryScripts) ... ok

======================================================================
ERROR: test_sundry (test.test_tools.test_sundry.TestSundryScripts)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\vstinner\AppData\Local\Programs\Python\Python37\lib\optparse.py", line 1387, in parse_args
    stop = self._process_args(largs, rargs, values)
  File "C:\Users\vstinner\AppData\Local\Programs\Python\Python37\lib\optparse.py", line 1431, in _process_args
    self._process_short_opts(rargs, values)
  File "C:\Users\vstinner\AppData\Local\Programs\Python\Python37\lib\optparse.py", line 1513, in _process_short_opts
    raise BadOptionError(opt)
optparse.BadOptionError: no such option: -m

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\vstinner\AppData\Local\Programs\Python\Python37\lib\test\test_tools\test_sundry.py", line 36, in test_sundry
    import_tool(name)
  File "C:\Users\vstinner\AppData\Local\Programs\Python\Python37\lib\test\test_tools\__init__.py", line 21, in import_tool
    return importlib.import_module(toolname)
  File "C:\Users\vstinner\AppData\Local\Programs\Python\Python37\lib\importlib\__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "C:\Users\vstinner\AppData\Local\Programs\Python\Python37\Tools\scripts\2to3.py", line 5, in <module>
    sys.exit(main("lib2to3.fixes"))
  File "C:\Users\vstinner\AppData\Local\Programs\Python\Python37\lib\lib2to3\main.py", line 179, in main
    options, args = parser.parse_args(args)
  File "C:\Users\vstinner\AppData\Local\Programs\Python\Python37\lib\optparse.py", line 1389, in parse_args
    self.error(str(err))
  File "C:\Users\vstinner\AppData\Local\Programs\Python\Python37\lib\optparse.py", line 1569, in error
    self.exit(2, "%s: error: %s\n" % (self.get_prog_name(), msg))
  File "C:\Users\vstinner\AppData\Local\Programs\Python\Python37\lib\optparse.py", line 1559, in exit
    sys.exit(status)
SystemExit: 2

----------------------------------------------------------------------
Ran 3 tests in 0.031s

FAILED (errors=1)
test test_tools failed
test_tools failed

1 test failed:
    test_tools

Total duration: 78 ms
Tests result: FAILURE

----------
components: Tests
messages: 318322
nosy: vstinner
priority: normal
severity: normal
status: open
title: Test failures on Python 3.7 beta 5 and Windows 10
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33719>
_______________________________________

From report at bugs.python.org  Thu May 31 11:13:35 2018
From: report at bugs.python.org (STINNER Victor)
Date: Thu, 31 May 2018 15:13:35 +0000
Subject: [New-bugs-announce] [issue33720] test_marshal: crash in Python
 3.7b5 on Windows 10
Message-ID: <1527779615.11.0.682650639539.issue33720@psf.upfronthosting.co.za>


New submission from STINNER Victor <vstinner at redhat.com>:

Follow-up of bpo-33719.

C:\Users\vstinner\AppData\Local\Programs\Python\Python37>python.exe -m test test_marshal -v
== CPython 3.7.0b5 (v3.7.0b5:abb8802389, May 31 2018, 01:54:01) [MSC v.1913 64 bit (AMD64)]
== Windows-10-10.0.16299-SP0 little-endian
== cwd: C:\Users\vstinner\AppData\Local\Temp\test_python_3836
== CPU count: 2
== encodings: locale=cp1252, FS=utf-8
Run tests sequentially
0:00:00 [1/1] test_marshal
(...)
test_loads_2x_code (test.test_marshal.BugsTestCase) ... Windows fatal exception: stack overflow

Current thread 0x000003a0 (most recent call first):
  File "C:\Users\vstinner\AppData\Local\Programs\Python\Python37\lib\unittest\case.py", line 178 in handle
  File "C:\Users\vstinner\AppData\Local\Programs\Python\Python37\lib\unittest\case.py", line 743 in assertRaises
  File "C:\Users\vstinner\AppData\Local\Programs\Python\Python37\lib\test\test_marshal.py", line 215 in test_loads_2x_code
  (...)

Crashes in test_marshal is on old topic:

* bpo-1050
* bpo-2286
* bpo-25264
* bpo-22734
* bpo-27019

Current stack size: 2 million bytes (1.9 MiB)

PCbuild/python.vcxproj:      <StackReserveSize>2000000</StackReserveSize>
PCbuild/pythonw.vcxproj:      <StackReserveSize>2000000</StackReserveSize>

----------
components: Tests, Windows
messages: 318323
nosy: paul.moore, steve.dower, tim.golden, vstinner, zach.ware
priority: normal
severity: normal
status: open
title: test_marshal: crash in Python 3.7b5 on Windows 10
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33720>
_______________________________________

From report at bugs.python.org  Thu May 31 13:19:25 2018
From: report at bugs.python.org (pacujo)
Date: Thu, 31 May 2018 17:19:25 +0000
Subject: [New-bugs-announce] [issue33721] os.path.exists() ought to return
 False if pathname contains NUL
Message-ID: <1527787165.92.0.682650639539.issue33721@psf.upfronthosting.co.za>


New submission from pacujo <marko at pacujo.net>:

os.path.exists() returns True or False for all imaginable string arguments except for one that contains NUL ("\0") (Linux). This behavior is not documented in the library. Moreover, it can easily lead to accidents if  an externally supplied pathname were to contain a NUL because most test suites would not try to cover such a pathname.

I propose os.path.exists() should return False even in this case.

----------
components: Library (Lib)
messages: 318334
nosy: pacujo
priority: normal
severity: normal
status: open
title: os.path.exists() ought to return False if pathname contains NUL
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33721>
_______________________________________

From report at bugs.python.org  Thu May 31 14:35:34 2018
From: report at bugs.python.org (Jay Crotts)
Date: Thu, 31 May 2018 18:35:34 +0000
Subject: [New-bugs-announce] [issue33722] Document builtins in mock_open
Message-ID: <1527791734.24.0.682650639539.issue33722@psf.upfronthosting.co.za>


New submission from Jay Crotts <crotts.jay at gmail.com>:

The examples on using mock_open only include instances where objects are mocked in the REPL, so '__main__'.open is replaced. Commonly objects are mocked for use in other test modules, so builtins.open would be used instead.

A note about this in the documentation would be useful not familiar with python internals. I'm happy to do a PR for it.

----------
assignee: docs at python
components: Documentation
messages: 318337
nosy: docs at python, jcrotts
priority: normal
severity: normal
status: open
title: Document builtins in mock_open
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33722>
_______________________________________

From report at bugs.python.org  Thu May 31 17:45:16 2018
From: report at bugs.python.org (STINNER Victor)
Date: Thu, 31 May 2018 21:45:16 +0000
Subject: [New-bugs-announce] [issue33723] test_time.test_thread_time()
 failed on AMD64 Debian root 3.x
Message-ID: <1527803116.79.0.682650639539.issue33723@psf.upfronthosting.co.za>


New submission from STINNER Victor <vstinner at redhat.com>:

AMD64 Debian root 3.x:

http://buildbot.python.org/all/#/builders/27/builds/1067

======================================================================
FAIL: test_thread_time (test.test_time.TimeTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/test_time.py", line 536, in test_thread_time
    self.assertGreaterEqual(stop - start, 0.020)  # machine busy?
AssertionError: 0.019943113999999984 not greater than or equal to 0.02

----------
components: Tests
messages: 318344
nosy: pitrou, vstinner
priority: normal
severity: normal
status: open
title: test_time.test_thread_time() failed on AMD64 Debian root 3.x
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33723>
_______________________________________

From report at bugs.python.org  Thu May 31 17:51:41 2018
From: report at bugs.python.org (STINNER Victor)
Date: Thu, 31 May 2018 21:51:41 +0000
Subject: [New-bugs-announce] [issue33724] test__xxsubinterpreters failed on
 ARMv7 Ubuntu 3.x
Message-ID: <1527803501.98.0.682650639539.issue33724@psf.upfronthosting.co.za>


New submission from STINNER Victor <vstinner at redhat.com>:

Follow-up of bpo-33615.

ARMv7 Ubuntu 3.x:

http://buildbot.python.org/all/#/builders/106/builds/1118

Re-running test 'test__xxsubinterpreters' in verbose mode
test_bad_id (test.test__xxsubinterpreters.ChannelIDTests) ... ok
test_bad_kwargs (test.test__xxsubinterpreters.ChannelIDTests) ... ok
test_coerce_id (test.test__xxsubinterpreters.ChannelIDTests) ... ok
test_default_kwargs (test.test__xxsubinterpreters.ChannelIDTests) ... ok
test_does_not_exist (test.test__xxsubinterpreters.ChannelIDTests) ... ok
test_equality (test.test__xxsubinterpreters.ChannelIDTests) ... ok
test_repr (test.test__xxsubinterpreters.ChannelIDTests) ... ok
test_str (test.test__xxsubinterpreters.ChannelIDTests) ... FAIL
test_with_kwargs (test.test__xxsubinterpreters.ChannelIDTests) ... ok
test_by_unassociated_interp (test.test__xxsubinterpreters.ChannelReleaseTests) ... ERROR
test_close_if_unassociated (test.test__xxsubinterpreters.ChannelReleaseTests) ... ERROR
test_multiple_times (test.test__xxsubinterpreters.ChannelReleaseTests) ... ok
test_multiple_users (test.test__xxsubinterpreters.ChannelReleaseTests) ... ERROR
test_never_used (test.test__xxsubinterpreters.ChannelReleaseTests) ... ok
test_no_kwargs (test.test__xxsubinterpreters.ChannelReleaseTests) ... ok
test_partially (test.test__xxsubinterpreters.ChannelReleaseTests) ... ok
test_single_user (test.test__xxsubinterpreters.ChannelReleaseTests) ... ok
test_used_multiple_times_by_single_user (test.test__xxsubinterpreters.ChannelReleaseTests) ... ok
test_with_unused_items (test.test__xxsubinterpreters.ChannelReleaseTests) ... ok
test_close_both_with_unused_items_forced (test.test__xxsubinterpreters.ChannelTests) ... ok
test_close_both_with_unused_items_unforced (test.test__xxsubinterpreters.ChannelTests) ... ok
test_close_by_unassociated_interp (test.test__xxsubinterpreters.ChannelTests) ... ERROR
test_close_defaults_with_unused_items (test.test__xxsubinterpreters.ChannelTests) ... ok
test_close_empty (test.test__xxsubinterpreters.ChannelTests) ... ok
test_close_multiple_times (test.test__xxsubinterpreters.ChannelTests) ... ok
test_close_multiple_users (test.test__xxsubinterpreters.ChannelTests) ... ERROR
test_close_never_used (test.test__xxsubinterpreters.ChannelTests) ... ok
test_close_recv_with_unused_items_forced (test.test__xxsubinterpreters.ChannelTests) ... ok
test_close_recv_with_unused_items_unforced (test.test__xxsubinterpreters.ChannelTests) ... ok
test_close_send_with_unused_items_forced (test.test__xxsubinterpreters.ChannelTests) ... ok
test_close_send_with_unused_items_unforced (test.test__xxsubinterpreters.ChannelTests) ... ok
test_close_single_user (test.test__xxsubinterpreters.ChannelTests) ... ok
test_close_used_multiple_times_by_single_user (test.test__xxsubinterpreters.ChannelTests) ... ok
test_create_cid (test.test__xxsubinterpreters.ChannelTests) ... ok
test_ids_global (test.test__xxsubinterpreters.ChannelTests) ... FAIL
test_recv_empty (test.test__xxsubinterpreters.ChannelTests) ... ok
test_recv_not_found (test.test__xxsubinterpreters.ChannelTests) ... ok
test_run_string_arg_resolved (test.test__xxsubinterpreters.ChannelTests) ... skipped 'bpo-33615: triggering crashes so temporarily disabled'
test_run_string_arg_unresolved (test.test__xxsubinterpreters.ChannelTests) ... ok
test_send_not_found (test.test__xxsubinterpreters.ChannelTests) ... ok
test_send_recv_different_interpreters (test.test__xxsubinterpreters.ChannelTests) ... ERROR
Exception in thread Thread-9:
Traceback (most recent call last):
  File "/ssd/buildbot/buildarea/3.x.gps-ubuntu-exynos5-armv7l/build/Lib/threading.py", line 917, in _bootstrap_inner
    self.run()
  File "/ssd/buildbot/buildarea/3.x.gps-ubuntu-exynos5-armv7l/build/Lib/threading.py", line 865, in run
    self._target(*self._args, **self._kwargs)
  File "/ssd/buildbot/buildarea/3.x.gps-ubuntu-exynos5-armv7l/build/Lib/test/test__xxsubinterpreters.py", line 1280, in f
    """))
  File "/ssd/buildbot/buildarea/3.x.gps-ubuntu-exynos5-armv7l/build/Lib/test/test__xxsubinterpreters.py", line 42, in _run_output
    interpreters.run_string(interp, script, shared)
_xxsubinterpreters.RunFailedError: <class '_xxsubinterpreters.ChannelNotFoundError'>: channel 1103095676 not found

test_send_recv_different_interpreters_and_threads (test.test__xxsubinterpreters.ChannelTests) ... FAIL
test_send_recv_different_threads (test.test__xxsubinterpreters.ChannelTests) ... ok
test_send_recv_main (test.test__xxsubinterpreters.ChannelTests) ... ok
test_send_recv_same_interpreter (test.test__xxsubinterpreters.ChannelTests) ... ok
test_sequential_ids (test.test__xxsubinterpreters.ChannelTests) ... ok
test_after_destroy_all (test.test__xxsubinterpreters.CreateTests) ... ok
test_after_destroy_some (test.test__xxsubinterpreters.CreateTests) ... ok
test_in_main (test.test__xxsubinterpreters.CreateTests) ... ok
test_in_subinterpreter (test.test__xxsubinterpreters.CreateTests) ... FAIL
test_in_thread (test.test__xxsubinterpreters.CreateTests) ... ok
test_in_threaded_subinterpreter (test.test__xxsubinterpreters.CreateTests) ... FAIL
test_unique_id (test.test__xxsubinterpreters.CreateTests) ... skipped 'enable this test when working on pystate.c'
test_all (test.test__xxsubinterpreters.DestroyTests) ... ok
test_already_destroyed (test.test__xxsubinterpreters.DestroyTests) ... ok
test_bad_id (test.test__xxsubinterpreters.DestroyTests) ... ok
test_does_not_exist (test.test__xxsubinterpreters.DestroyTests) ... ok
test_from_current (test.test__xxsubinterpreters.DestroyTests) ... ok
test_from_other_thread (test.test__xxsubinterpreters.DestroyTests) ... ok
test_from_sibling (test.test__xxsubinterpreters.DestroyTests) ... ERROR
test_main (test.test__xxsubinterpreters.DestroyTests) ... ok
test_one (test.test__xxsubinterpreters.DestroyTests) ... ok
test_still_running (test.test__xxsubinterpreters.DestroyTests) ... ok
test_close (test.test__xxsubinterpreters.ExhaustiveChannelTests) ... skipped 'these tests take several hours to run'
test_force_close (test.test__xxsubinterpreters.ExhaustiveChannelTests) ... skipped 'these tests take several hours to run'
test_main (test.test__xxsubinterpreters.GetCurrentTests) ... ok
test_subinterpreter (test.test__xxsubinterpreters.GetCurrentTests) ... FAIL
test_from_main (test.test__xxsubinterpreters.GetMainTests) ... ok
test_from_subinterpreter (test.test__xxsubinterpreters.GetMainTests) ... FAIL
test_bad_id (test.test__xxsubinterpreters.InterpreterIDTests) ... ok
test_coerce_id (test.test__xxsubinterpreters.InterpreterIDTests) ... ok
test_does_not_exist (test.test__xxsubinterpreters.InterpreterIDTests) ... ok
test_equality (test.test__xxsubinterpreters.InterpreterIDTests) ... ok
test_repr (test.test__xxsubinterpreters.InterpreterIDTests) ... ok
test_str (test.test__xxsubinterpreters.InterpreterIDTests) ... FAIL
test_with_int (test.test__xxsubinterpreters.InterpreterIDTests) ... ok
test_already_destroyed (test.test__xxsubinterpreters.IsRunningTests) ... ok
test_bad_id (test.test__xxsubinterpreters.IsRunningTests) ... ok
test_does_not_exist (test.test__xxsubinterpreters.IsRunningTests) ... ok
test_from_subinterpreter (test.test__xxsubinterpreters.IsRunningTests) ... ERROR
test_main (test.test__xxsubinterpreters.IsRunningTests) ... ok
test_subinterpreter (test.test__xxsubinterpreters.IsRunningTests) ... ok
test_default_shareables (test.test__xxsubinterpreters.IsShareableTests) ... ok
test_not_shareable (test.test__xxsubinterpreters.IsShareableTests) ... ok
test_after_creating (test.test__xxsubinterpreters.ListAllTests) ... ok
test_after_destroying (test.test__xxsubinterpreters.ListAllTests) ... ok
test_initial (test.test__xxsubinterpreters.ListAllTests) ... ok
test_SystemExit (test.test__xxsubinterpreters.RunStringTests) ... ok
test_already_running (test.test__xxsubinterpreters.RunStringTests) ... ok
test_bad_id (test.test__xxsubinterpreters.RunStringTests) ... ok
test_bad_script (test.test__xxsubinterpreters.RunStringTests) ... ok
test_bytes_for_script (test.test__xxsubinterpreters.RunStringTests) ... ok
test_create_thread (test.test__xxsubinterpreters.RunStringTests) ... ok
test_does_not_exist (test.test__xxsubinterpreters.RunStringTests) ... ok
test_error_id (test.test__xxsubinterpreters.RunStringTests) ... ok
test_execution_namespace_is_main (test.test__xxsubinterpreters.RunStringTests) ... ok
test_failure (test.test__xxsubinterpreters.RunStringTests) ... ok
test_fork (test.test__xxsubinterpreters.RunStringTests) ... ok
test_in_thread (test.test__xxsubinterpreters.RunStringTests) ... ok
test_invalid_syntax (test.test__xxsubinterpreters.RunStringTests) ... ok
test_main_reused (test.test__xxsubinterpreters.RunStringTests) ... ok
test_shared_overwrites (test.test__xxsubinterpreters.RunStringTests) ... ok
test_shared_overwrites_default_vars (test.test__xxsubinterpreters.RunStringTests) ... ok
test_still_running_at_exit (test.test__xxsubinterpreters.RunStringTests) ... skipped 'blocking forever'
test_success (test.test__xxsubinterpreters.RunStringTests) ... ok
test_sys_exit (test.test__xxsubinterpreters.RunStringTests) ... ok
test_with_shared (test.test__xxsubinterpreters.RunStringTests) ... ok
test_bytes (test.test__xxsubinterpreters.ShareableTypeTests) ... ok
test_int (test.test__xxsubinterpreters.ShareableTypeTests) ... ok
test_singletons (test.test__xxsubinterpreters.ShareableTypeTests) ... ok
test_types (test.test__xxsubinterpreters.ShareableTypeTests) ... ok

======================================================================
ERROR: test_by_unassociated_interp (test.test__xxsubinterpreters.ChannelReleaseTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/ssd/buildbot/buildarea/3.x.gps-ubuntu-exynos5-armv7l/build/Lib/test/test__xxsubinterpreters.py", line 1640, in test_by_unassociated_interp
    """))
_xxsubinterpreters.RunFailedError: <class 'ValueError'>: 'id' must be a non-negative int

======================================================================
ERROR: test_close_if_unassociated (test.test__xxsubinterpreters.ChannelReleaseTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/ssd/buildbot/buildarea/3.x.gps-ubuntu-exynos5-armv7l/build/Lib/test/test__xxsubinterpreters.py", line 1656, in test_close_if_unassociated
    """))
_xxsubinterpreters.RunFailedError: <class 'ValueError'>: 'id' must be a non-negative int

======================================================================
ERROR: test_multiple_users (test.test__xxsubinterpreters.ChannelReleaseTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/ssd/buildbot/buildarea/3.x.gps-ubuntu-exynos5-armv7l/build/Lib/test/test__xxsubinterpreters.py", line 1582, in test_multiple_users
    """))
_xxsubinterpreters.RunFailedError: <class 'ValueError'>: 'id' must be a non-negative int

======================================================================
ERROR: test_close_by_unassociated_interp (test.test__xxsubinterpreters.ChannelTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/ssd/buildbot/buildarea/3.x.gps-ubuntu-exynos5-armv7l/build/Lib/test/test__xxsubinterpreters.py", line 1501, in test_close_by_unassociated_interp
    """))
_xxsubinterpreters.RunFailedError: <class 'ValueError'>: 'id' must be a non-negative int

======================================================================
ERROR: test_close_multiple_users (test.test__xxsubinterpreters.ChannelTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/ssd/buildbot/buildarea/3.x.gps-ubuntu-exynos5-armv7l/build/Lib/test/test__xxsubinterpreters.py", line 1358, in test_close_multiple_users
    """))
_xxsubinterpreters.RunFailedError: <class 'ValueError'>: 'id' must be a non-negative int

======================================================================
ERROR: test_send_recv_different_interpreters (test.test__xxsubinterpreters.ChannelTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/ssd/buildbot/buildarea/3.x.gps-ubuntu-exynos5-armv7l/build/Lib/test/test__xxsubinterpreters.py", line 1237, in test_send_recv_different_interpreters
    """))
  File "/ssd/buildbot/buildarea/3.x.gps-ubuntu-exynos5-armv7l/build/Lib/test/test__xxsubinterpreters.py", line 42, in _run_output
    interpreters.run_string(interp, script, shared)
_xxsubinterpreters.RunFailedError: <class 'ValueError'>: 'id' must be a non-negative int

======================================================================
ERROR: test_from_sibling (test.test__xxsubinterpreters.DestroyTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/ssd/buildbot/buildarea/3.x.gps-ubuntu-exynos5-armv7l/build/Lib/test/test__xxsubinterpreters.py", line 734, in test_from_sibling
    interpreters.run_string(id1, script)
_xxsubinterpreters.RunFailedError: <class 'RuntimeError'>: unrecognized interpreter ID -1098177076

======================================================================
ERROR: test_from_subinterpreter (test.test__xxsubinterpreters.IsRunningTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/ssd/buildbot/buildarea/3.x.gps-ubuntu-exynos5-armv7l/build/Lib/test/test__xxsubinterpreters.py", line 490, in test_from_subinterpreter
    """))
  File "/ssd/buildbot/buildarea/3.x.gps-ubuntu-exynos5-armv7l/build/Lib/test/test__xxsubinterpreters.py", line 42, in _run_output
    interpreters.run_string(interp, script, shared)
_xxsubinterpreters.RunFailedError: <class 'RuntimeError'>: unrecognized interpreter ID -1098177076

======================================================================
FAIL: test_str (test.test__xxsubinterpreters.ChannelIDTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/ssd/buildbot/buildarea/3.x.gps-ubuntu-exynos5-armv7l/build/Lib/test/test__xxsubinterpreters.py", line 1142, in test_str
    self.assertEqual(str(cid), '10')
AssertionError: '0' != '10'
- 0
+ 10
? +


======================================================================
FAIL: test_ids_global (test.test__xxsubinterpreters.ChannelTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/ssd/buildbot/buildarea/3.x.gps-ubuntu-exynos5-armv7l/build/Lib/test/test__xxsubinterpreters.py", line 1206, in test_ids_global
    self.assertEqual(cid2, int(cid1) + 1)
AssertionError: 1113969968 != 1112900745

======================================================================
FAIL: test_send_recv_different_interpreters_and_threads (test.test__xxsubinterpreters.ChannelTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/ssd/buildbot/buildarea/3.x.gps-ubuntu-exynos5-armv7l/build/Lib/test/test__xxsubinterpreters.py", line 1288, in test_send_recv_different_interpreters_and_threads
    self.assertEqual(obj, b'eggs')
AssertionError: b'spam' != b'eggs'

======================================================================
FAIL: test_in_subinterpreter (test.test__xxsubinterpreters.CreateTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/ssd/buildbot/buildarea/3.x.gps-ubuntu-exynos5-armv7l/build/Lib/test/test__xxsubinterpreters.py", line 613, in test_in_subinterpreter
    self.assertEqual(set(interpreters.list_all()), {main, id1, id2})
AssertionError: Items in the first set but not the second:
InterpreterID(24)
Items in the second set but not the first:
1113009960

======================================================================
FAIL: test_in_threaded_subinterpreter (test.test__xxsubinterpreters.CreateTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/ssd/buildbot/buildarea/3.x.gps-ubuntu-exynos5-armv7l/build/Lib/test/test__xxsubinterpreters.py", line 632, in test_in_threaded_subinterpreter
    self.assertEqual(set(interpreters.list_all()), {main, id1, id2})
AssertionError: Items in the first set but not the second:
InterpreterID(27)
Items in the second set but not the first:
1113006640

======================================================================
FAIL: test_subinterpreter (test.test__xxsubinterpreters.GetCurrentTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/ssd/buildbot/buildarea/3.x.gps-ubuntu-exynos5-armv7l/build/Lib/test/test__xxsubinterpreters.py", line 443, in test_subinterpreter
    self.assertEqual(cur, expected)
AssertionError: 1113007880 != InterpreterID(40)

======================================================================
FAIL: test_from_subinterpreter (test.test__xxsubinterpreters.GetMainTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/ssd/buildbot/buildarea/3.x.gps-ubuntu-exynos5-armv7l/build/Lib/test/test__xxsubinterpreters.py", line 465, in test_from_subinterpreter
    self.assertEqual(main, expected)
AssertionError: 1113006680 != InterpreterID(0)

======================================================================
FAIL: test_str (test.test__xxsubinterpreters.InterpreterIDTests)
test test__xxsubinterpreters failed
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/ssd/buildbot/buildarea/3.x.gps-ubuntu-exynos5-armv7l/build/Lib/test/test__xxsubinterpreters.py", line 548, in test_str
    self.assertEqual(str(id), '10')
AssertionError: '0' != '10'
- 0
+ 10
? +


----------------------------------------------------------------------
Ran 111 tests in 4.863s

FAILED (failures=8, errors=8, skipped=5)
1 test failed again:
    test__xxsubinterpreters

----------
components: Tests
messages: 318345
nosy: eric.snow, vstinner
priority: normal
severity: normal
status: open
title: test__xxsubinterpreters failed on ARMv7 Ubuntu 3.x
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33724>
_______________________________________

From report at bugs.python.org  Thu May 31 20:53:06 2018
From: report at bugs.python.org (Kapil Thangavelu)
Date: Fri, 01 Jun 2018 00:53:06 +0000
Subject: [New-bugs-announce] [issue33725] High Sierra hang when using
 multi-processing
Message-ID: <1527814386.62.0.682650639539.issue33725@psf.upfronthosting.co.za>


New submission from Kapil Thangavelu <kapilt at gmail.com>:

This issue seems to be reported a few times on various githubs projects. I've also reproduced using a brew install of python 2.7.15. I haven't been able to reproduce with python 3.6. Note this requires a framework build of python.

Background on the underlying issue cause due to a change in high Sierra 
http://sealiesoftware.com/blog/archive/2017/6/5/Objective-C_and_fork_in_macOS_1013.html
A ruby perspective on the same issue exhibiting for some apps
https://blog.phusion.nl/2017/10/13/why-ruby-app-servers-break-on-macos-high-sierra-and-what-can-be-done-about-it/


The work around seems to be setting an environment variable OBJC_DISABLE_INITIALIZE_FORK_SAFETY prior to executing python.

Other reports

https://bugs.python.org/issue30837
https://github.com/ansible/ansible/issues/32499
https://github.com/imWildCat/scylla/issues/22
https://github.com/elastic/beats-tester/pull/73
https://github.com/jhaals/ansible-vault/issues/60

----------
components: macOS
messages: 318352
nosy: kapilt, ned.deily, ronaldoussoren
priority: normal
severity: normal
status: open
title: High Sierra hang when using multi-processing
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33725>
_______________________________________

From report at bugs.python.org  Thu May 31 23:03:25 2018
From: report at bugs.python.org (=?utf-8?q?Andr=C3=A9s_Delfino?=)
Date: Fri, 01 Jun 2018 03:03:25 +0000
Subject: [New-bugs-announce] [issue33726] Add short descriptions to PEP
 references in seealso
Message-ID: <1527822205.22.0.682650639539.issue33726@psf.upfronthosting.co.za>


New submission from Andr?s Delfino <adelfino at gmail.com>:

There are a couple of PEP references with no description in Simple/Compound Statements.

Attached PR fixes this.

----------
assignee: docs at python
components: Documentation
messages: 318359
nosy: adelfino, docs at python
priority: normal
severity: normal
status: open
title: Add short descriptions to PEP references in seealso
type: enhancement
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33726>
_______________________________________