From report at bugs.python.org  Sun Jan  1 01:35:59 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sun, 01 Jan 2017 06:35:59 +0000
Subject: [issue29071] IDLE doesn't highlight f-strings properly
In-Reply-To: <1482714978.44.0.603392455768.issue29071@psf.upfronthosting.co.za>
Message-ID: <1483252559.7.0.53821672138.issue29071@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

The regular expression can be simplified using local flags:

-    stringprefix = (r"(\br|R|u|U|f|F|fr|Fr|fR|FR|rf|rF|Rf|RF"
-                    "|b|B|br|Br|bR|BR|rb|rB|Rb|RB)?")
+    stringprefix = r"(?i:\br|u|f|fr|rf|b|br|rb)?"

----------
nosy: +serhiy.storchaka

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29071>
_______________________________________

From report at bugs.python.org  Sun Jan  1 02:34:49 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sun, 01 Jan 2017 07:34:49 +0000
Subject: [issue29094] Regression in zipfile writing in 2.7.13
In-Reply-To: <1482931591.96.0.8795611166.issue29094@psf.upfronthosting.co.za>
Message-ID: <1483256089.37.0.662630069046.issue29094@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

The specification of ZIP files is not pretty clear, but the unzip utility first try to interpret offsets relatively to the start of the archive that can be different from the start of the file. Go's standard archive/zip package interprets offsets as absolute file positions and doesn't support concatenated ZIP files.

In any case there is a regression. Proposed patch partially reverts issue26293. Offsets in created ZIP file are now absolute when create with modes 'w' and 'x'. But they are relative when create with mode 'a'. This should fix a regression in pex, but keep the case of issue26293.

I'm going to push the patch before tagging 3.5.3 rc1.

----------
keywords: +patch
nosy: +benjamin.peterson, larry, ned.deily
priority: normal -> release blocker
stage:  -> patch review
type:  -> behavior
versions: +Python 3.5, Python 3.6, Python 3.7
Added file: http://bugs.python.org/file46107/zipfile_write_absolute_offsets.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29094>
_______________________________________

From report at bugs.python.org  Sun Jan  1 02:41:51 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sun, 01 Jan 2017 07:41:51 +0000
Subject: [issue28964] AST literal_eval exceptions provide no information about
 line number
In-Reply-To: <1481665664.26.0.256194535172.issue28964@psf.upfronthosting.co.za>
Message-ID: <1483256511.96.0.291134793874.issue28964@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

Usually literal_eval() is used with short one-line input. For what large documents do you use it?

----------
nosy: +serhiy.storchaka

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28964>
_______________________________________

From report at bugs.python.org  Sun Jan  1 03:27:56 2017
From: report at bugs.python.org (Dmitry Shachnev)
Date: Sun, 01 Jan 2017 08:27:56 +0000
Subject: [issue28401] Don't support the PEP384 stable ABI in pydebug builds
In-Reply-To: <1476034292.17.0.316209799179.issue28401@psf.upfronthosting.co.za>
Message-ID: <1483259276.12.0.459827890698.issue28401@psf.upfronthosting.co.za>


Changes by Dmitry Shachnev <Mitya57 at gmail.com>:


----------
nosy: +mitya57

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28401>
_______________________________________

From report at bugs.python.org  Sun Jan  1 05:35:30 2017
From: report at bugs.python.org (Chi Hsuan Yen)
Date: Sun, 01 Jan 2017 10:35:30 +0000
Subject: [issue29057] Compiler failure on Mac OS X - sys/random.h
In-Reply-To: <1482514649.72.0.460227366509.issue29057@psf.upfronthosting.co.za>
Message-ID: <1483266930.38.0.869896888027.issue29057@psf.upfronthosting.co.za>


Changes by Chi Hsuan Yen <yan12125 at gmail.com>:


----------
nosy: +Chi Hsuan Yen

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29057>
_______________________________________

From report at bugs.python.org  Sun Jan  1 10:10:54 2017
From: report at bugs.python.org (Benjamin Peterson)
Date: Sun, 01 Jan 2017 15:10:54 +0000
Subject: [issue29094] Regression in zipfile writing in 2.7.13
In-Reply-To: <1482931591.96.0.8795611166.issue29094@psf.upfronthosting.co.za>
Message-ID: <1483283454.88.0.411265772547.issue29094@psf.upfronthosting.co.za>


Benjamin Peterson added the comment:

Is there a reason to ever not use relative offsets? It seems that's strictly more general the absolute.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29094>
_______________________________________

From report at bugs.python.org  Sun Jan  1 10:54:13 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sun, 01 Jan 2017 15:54:13 +0000
Subject: [issue29094] Regression in zipfile writing in 2.7.13
In-Reply-To: <1482931591.96.0.8795611166.issue29094@psf.upfronthosting.co.za>
Message-ID: <1483286053.49.0.106773417273.issue29094@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

Actually all offsets are relative to some point, just this point not always at the start of the file. If concatenate the ZIP file to other file, the unzip utility and the zipfile module are able to infer the starting point and correct offsets. Thus there is no matter what is the starting point of offsets. But Go's standard archive/zip package works only if offsets are relative to the start of the file. I don't know how common such interpretation however.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29094>
_______________________________________

From report at bugs.python.org  Sun Jan  1 12:08:29 2017
From: report at bugs.python.org (Roundup Robot)
Date: Sun, 01 Jan 2017 17:08:29 +0000
Subject: [issue29094] Regression in zipfile writing in 2.7.13
In-Reply-To: <1482931591.96.0.8795611166.issue29094@psf.upfronthosting.co.za>
Message-ID: <20170101170826.9908.91611.2BFF97F2@psf.io>


Roundup Robot added the comment:

New changeset f5aa1c9c2b7e by Serhiy Storchaka in branch '3.5':
Issue #29094: Offsets in a ZIP file created with extern file object and modes
https://hg.python.org/cpython/rev/f5aa1c9c2b7e

New changeset 342bc734f523 by Serhiy Storchaka in branch '2.7':
Issue #29094: Offsets in a ZIP file created with extern file object and modes
https://hg.python.org/cpython/rev/342bc734f523

New changeset f36f9bce997d by Serhiy Storchaka in branch '3.6':
Issue #29094: Offsets in a ZIP file created with extern file object and modes
https://hg.python.org/cpython/rev/f36f9bce997d

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29094>
_______________________________________

From report at bugs.python.org  Sun Jan  1 12:28:53 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sun, 01 Jan 2017 17:28:53 +0000
Subject: [issue29125] Shell injection via TIX_LIBRARY when using tkinter.tix
In-Reply-To: <1483210830.52.0.920210756798.issue29125@psf.upfronthosting.co.za>
Message-ID: <1483291733.72.0.102959134902.issue29125@psf.upfronthosting.co.za>


Changes by Serhiy Storchaka <storchaka+cpython at gmail.com>:


----------
nosy: +serhiy.storchaka
versions: +Python 3.5, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29125>
_______________________________________

From report at bugs.python.org  Sun Jan  1 13:15:16 2017
From: report at bugs.python.org (ppperry)
Date: Sun, 01 Jan 2017 18:15:16 +0000
Subject: [issue16396] Importing ctypes.wintypes on Linux gives a ValueError
 instead of an ImportError
In-Reply-To: <1351956172.0.0.64927799046.issue16396@psf.upfronthosting.co.za>
Message-ID: <1483294516.83.0.347669298362.issue16396@psf.upfronthosting.co.za>


Changes by ppperry <mapreader at olum.org>:


----------
title: Importing ctypes.wintypes on Linux gives a traceback -> Importing ctypes.wintypes on Linux gives a ValueError instead of an ImportError

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue16396>
_______________________________________

From report at bugs.python.org  Sun Jan  1 14:05:45 2017
From: report at bugs.python.org (Zoe Mbikayi)
Date: Sun, 01 Jan 2017 19:05:45 +0000
Subject: [issue29128] No way to instsall win32com
Message-ID: <1483297545.45.0.26168848935.issue29128@psf.upfronthosting.co.za>


Changes by Zoe Mbikayi <zoetshimanga at gmail.com>:


----------
components: Library (Lib)
nosy: Zoe Mbikayi
priority: normal
severity: normal
status: open
title: No way to instsall win32com
type: resource usage
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29128>
_______________________________________

From report at bugs.python.org  Sun Jan  1 14:08:31 2017
From: report at bugs.python.org (Zoe Mbikayi)
Date: Sun, 01 Jan 2017 19:08:31 +0000
Subject: [issue29128] No way to instsall win32com on python 3.6
Message-ID: <1483297711.1.0.344949445361.issue29128@psf.upfronthosting.co.za>


Changes by Zoe Mbikayi <zoetshimanga at gmail.com>:


----------
title: No way to instsall win32com -> No way to instsall win32com on python 3.6
type: resource usage -> behavior

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29128>
_______________________________________

From report at bugs.python.org  Sun Jan  1 14:18:19 2017
From: report at bugs.python.org (Pam McA'Nulty)
Date: Sun, 01 Jan 2017 19:18:19 +0000
Subject: [issue29057] Compiler failure on Mac OS X - sys/random.h
In-Reply-To: <1482514649.72.0.460227366509.issue29057@psf.upfronthosting.co.za>
Message-ID: <1483298299.91.0.238685858709.issue29057@psf.upfronthosting.co.za>


Pam McA'Nulty added the comment:

I've dug into things some more.

In my mac environment, sys/random.h is pretty empty and doesn't actually seem to provide anything that Python/random.c uses.  The compile error is a type error is because 'u_int' never gets typedef'd in the rest of the compile includes.

I think the bottom line is that sys/random.h on mac os is neither useful, nor required.

Below is the error again and I've attached my sys/random.h
```
In file included from Python/random.c:16:
/usr/include/sys/random.h:37:32: error: unknown type name 'u_int'
void read_random(void* buffer, u_int numBytes);
                               ^
/usr/include/sys/random.h:38:33: error: unknown type name 'u_int'
void read_frandom(void* buffer, u_int numBytes);
                                ^
/usr/include/sys/random.h:39:33: error: unknown type name 'u_int'
int  write_random(void* buffer, u_int numBytes);
```

----------
Added file: http://bugs.python.org/file46108/random.h

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29057>
_______________________________________

From report at bugs.python.org  Sun Jan  1 15:11:07 2017
From: report at bugs.python.org (Chi Hsuan Yen)
Date: Sun, 01 Jan 2017 20:11:07 +0000
Subject: [issue29057] Compiler failure on Mac OS X - sys/random.h
In-Reply-To: <1482514649.72.0.460227366509.issue29057@psf.upfronthosting.co.za>
Message-ID: <1483301467.07.0.823664434335.issue29057@psf.upfronthosting.co.za>


Chi Hsuan Yen added the comment:

Here's a how-to for reproducing this bug on Sierra:

1. Install Xcode 7.3.1 (All 7.x should be fine)
2. ./configure --enable-universalsdk=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk
3. make

On Benjamin's question:

> why does configure define HAVE_SYS_RANDOM_H if including sys/random.h causes an error?

autoconf defines several "default includes". For example in line 592 of configure:

#ifdef HAVE_SYS_TYPES_H
# include <sys/types.h>
#endif

And in line 5559:

# On IRIX 5.3, sys/types and inttypes.h are conflicting.
for ac_header in sys/types.h sys/stat.h stdlib.h string.h memory.h strings.h \
		  inttypes.h stdint.h unistd.h
do :
  as_ac_Header=`$as_echo "ac_cv_header_$ac_header" | $as_tr_sh`
ac_fn_c_check_header_compile "$LINENO" "$ac_header" "$as_ac_Header" "$ac_includes_default
"
if eval test \"x\$"$as_ac_Header"\" = x"yes"; then :
  cat >>confdefs.h <<_ACEOF
#define `$as_echo "HAVE_$ac_header" | $as_tr_cpp` 1
_ACEOF

fi

done

The check for sys/random.h starts from line 7758, so sys/types.h is already (accidentally) included when checking it.

So, here's the fix.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29057>
_______________________________________

From report at bugs.python.org  Sun Jan  1 15:11:21 2017
From: report at bugs.python.org (Chi Hsuan Yen)
Date: Sun, 01 Jan 2017 20:11:21 +0000
Subject: [issue29057] Compiler failure on Mac OS X - sys/random.h
In-Reply-To: <1482514649.72.0.460227366509.issue29057@psf.upfronthosting.co.za>
Message-ID: <1483301481.57.0.202155981242.issue29057@psf.upfronthosting.co.za>


Changes by Chi Hsuan Yen <yan12125 at gmail.com>:


Added file: http://bugs.python.org/file46109/issue29057.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29057>
_______________________________________

From report at bugs.python.org  Sun Jan  1 15:12:02 2017
From: report at bugs.python.org (Brett Cannon)
Date: Sun, 01 Jan 2017 20:12:02 +0000
Subject: [issue29128] No way to instsall win32com on python 3.6
Message-ID: <1483301522.24.0.353841386115.issue29128@psf.upfronthosting.co.za>


New submission from Brett Cannon:

Can you provide a bit more detail, e.g. what error are you seeing?

----------
nosy: +brett.cannon

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29128>
_______________________________________

From report at bugs.python.org  Sun Jan  1 16:22:46 2017
From: report at bugs.python.org (Zoe Mbikayi)
Date: Sun, 01 Jan 2017 21:22:46 +0000
Subject: [issue29128] No way to instsall win32com on python 3.6
In-Reply-To: <1483301522.24.0.353841386115.issue29128@psf.upfronthosting.co.za>
Message-ID: <1483305766.77.0.512094471372.issue29128@psf.upfronthosting.co.za>


Zoe Mbikayi added the comment:

i'm using the command pip install pypiwin32
but the installation process stops and I have this error: syntaxtError: Missing parentheses in call to 'print'

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29128>
_______________________________________

From report at bugs.python.org  Sun Jan  1 17:11:46 2017
From: report at bugs.python.org (Steve Merritt)
Date: Sun, 01 Jan 2017 22:11:46 +0000
Subject: [issue28964] AST literal_eval exceptions provide no information about
 line number
In-Reply-To: <1483256511.96.0.291134793874.issue28964@psf.upfronthosting.co.za>
Message-ID: <CAG8dkNxO_Sp91EW5ivRPz7Q-SDcEZ5B0JegaMHfERg6NNUjRJw@mail.gmail.com>


Steve Merritt added the comment:

We're using Python dictionaries to express sizeable (read: 100-600 lines)
chunks of configuration data. We previously used JSON, but we now use
Jinja2 to template chunks of this configuration data, and JSON's inability
to handle trailing commas creates a problem here. So we've been using
ast.literal_eval, but the lack of line numbers makes debugging problems
with the configurations a painful process.

On Sun, Jan 1, 2017 at 2:41 AM Serhiy Storchaka <report at bugs.python.org>
wrote:

>
> Serhiy Storchaka added the comment:
>
> Usually literal_eval() is used with short one-line input. For what large
> documents do you use it?
>
> ----------
> nosy: +serhiy.storchaka
>
> _______________________________________
> Python tracker <report at bugs.python.org>
> <http://bugs.python.org/issue28964>
> _______________________________________
>

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28964>
_______________________________________

From report at bugs.python.org  Sun Jan  1 17:20:02 2017
From: report at bugs.python.org (Pam McA'Nulty)
Date: Sun, 01 Jan 2017 22:20:02 +0000
Subject: [issue29057] Compiler failure on Mac OS X - sys/random.h
In-Reply-To: <1482514649.72.0.460227366509.issue29057@psf.upfronthosting.co.za>
Message-ID: <1483309202.84.0.867643071503.issue29057@psf.upfronthosting.co.za>


Pam McA'Nulty added the comment:

I can conform that Chi Hsuan Yen's patch works for me.

I'm concerned that the build includes sys/random.h despite it not actually being needed to build python on mac os x (my worry is possible "shenanigans" by Apple in future versions of their headers) 

But if this patch makes sense to those who actually speak "./configure", I'm agnostic enough.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29057>
_______________________________________

From report at bugs.python.org  Sun Jan  1 17:28:06 2017
From: report at bugs.python.org (Larry Hastings)
Date: Sun, 01 Jan 2017 22:28:06 +0000
Subject: [issue29094] Regression in zipfile writing in 2.7.13
In-Reply-To: <1482931591.96.0.8795611166.issue29094@psf.upfronthosting.co.za>
Message-ID: <1483309686.95.0.53268749494.issue29094@psf.upfronthosting.co.za>


Larry Hastings added the comment:

If this is fixed, can we close this issue?  This release blocker is one of two issues blocking 3.5.3 rc1.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29094>
_______________________________________

From report at bugs.python.org  Sun Jan  1 17:28:28 2017
From: report at bugs.python.org (Larry Hastings)
Date: Sun, 01 Jan 2017 22:28:28 +0000
Subject: [issue29057] Compiler failure on Mac OS X - sys/random.h
In-Reply-To: <1482514649.72.0.460227366509.issue29057@psf.upfronthosting.co.za>
Message-ID: <1483309708.31.0.922827946121.issue29057@psf.upfronthosting.co.za>


Larry Hastings added the comment:

This is currently blocking the release of 3.5.3 rc1.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29057>
_______________________________________

From report at bugs.python.org  Sun Jan  1 17:40:58 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sun, 01 Jan 2017 22:40:58 +0000
Subject: [issue29094] Regression in zipfile writing in 2.7.13
In-Reply-To: <1482931591.96.0.8795611166.issue29094@psf.upfronthosting.co.za>
Message-ID: <1483310458.21.0.685421028259.issue29094@psf.upfronthosting.co.za>


Changes by Serhiy Storchaka <storchaka+cpython at gmail.com>:


----------
priority: release blocker -> normal
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29094>
_______________________________________

From report at bugs.python.org  Sun Jan  1 18:03:59 2017
From: report at bugs.python.org (Michael Mrozek)
Date: Sun, 01 Jan 2017 23:03:59 +0000
Subject: [issue29129] Copy/paste error in "8.13.14.1.1. Using auto"
Message-ID: <1483311839.39.0.725935533948.issue29129@psf.upfronthosting.co.za>


New submission from Michael Mrozek:

The "8.13.14.1.1 Using auto" section ( https://docs.python.org/3/library/enum.html#using-auto ) looks like it was copied from the subsequent "8.13.14.1.2 Using object" section, and a reference to "object" wasn't changed to "auto" in the first line.

----------
assignee: docs at python
components: Documentation
messages: 284434
nosy: docs at python, mrozekma
priority: normal
severity: normal
status: open
title: Copy/paste error in "8.13.14.1.1. Using auto"
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29129>
_______________________________________

From report at bugs.python.org  Sun Jan  1 18:09:50 2017
From: report at bugs.python.org (Berker Peksag)
Date: Sun, 01 Jan 2017 23:09:50 +0000
Subject: [issue29006] 2.7.13 _sqlite more prone to "database table is locked"
In-Reply-To: <1482095939.25.0.891516594682.issue29006@psf.upfronthosting.co.za>
Message-ID: <1483312190.3.0.898828626865.issue29006@psf.upfronthosting.co.za>


Berker Peksag added the comment:

Thanks for the report. I can also reproduce it with 3.5+. We may want to revert 030e100f048a for the next 2.7 and 3.5 releases if we can't come up with a solution.

----------
components: +Library (Lib)
stage:  -> needs patch
type:  -> behavior
versions: +Python 3.5, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29006>
_______________________________________

From report at bugs.python.org  Sun Jan  1 18:18:17 2017
From: report at bugs.python.org (Larry Hastings)
Date: Sun, 01 Jan 2017 23:18:17 +0000
Subject: [issue29006] 2.7.13 _sqlite more prone to "database table is locked"
In-Reply-To: <1482095939.25.0.891516594682.issue29006@psf.upfronthosting.co.za>
Message-ID: <1483312696.99.0.34960743079.issue29006@psf.upfronthosting.co.za>


Changes by Larry Hastings <larry at hastings.org>:


----------
nosy: +larry

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29006>
_______________________________________

From report at bugs.python.org  Sun Jan  1 18:43:26 2017
From: report at bugs.python.org (Roundup Robot)
Date: Sun, 01 Jan 2017 23:43:26 +0000
Subject: [issue29094] Regression in zipfile writing in 2.7.13
In-Reply-To: <1482931591.96.0.8795611166.issue29094@psf.upfronthosting.co.za>
Message-ID: <20170101234323.10164.90166.8062787B@psf.io>


Roundup Robot added the comment:

New changeset a80c14ace927 by Serhiy Storchaka in branch 'default':
Issue #29094: Offsets in a ZIP file created with extern file object and modes
https://hg.python.org/cpython/rev/a80c14ace927

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29094>
_______________________________________

From report at bugs.python.org  Sun Jan  1 19:28:18 2017
From: report at bugs.python.org (John Hagen)
Date: Mon, 02 Jan 2017 00:28:18 +0000
Subject: [issue29130] Exit code 120 returned from Python unit test testing
 SystemExit
Message-ID: <1483316898.11.0.0292789374137.issue29130@psf.upfronthosting.co.za>


New submission from John Hagen:

I recently tried to port one of my packages to Python 3.6 and unit tests that worked in Python 2.7, 3.3-3.5 began failing in 3.6.

I originally thought it was a problem with coverage, but it turns out it was not. The full thread is: https://bitbucket.org/ned/coveragepy/issues/545/coverage-fails-on-python-36-travis-build

The highlight is this unit test causes Python to exit with status code 120 (which fails a Travis build):

class ParseArgumentsTestCase(unittest.TestCase):
    def test_no_arguments(self):  # type: () -> None
        with self.assertRaises(SystemExit):
            # Suppress argparse stderr.
            class NullWriter:
                def write(self, s):  # type: (str) -> None
                    pass

            sys.stderr = NullWriter()
            parse_arguments()

Ned found this corresponding note in the Python 3.6 release notes:

Changed in version 3.6: If an error occurs in the cleanup after the Python interpreter has caught SystemExit (such as an error flushing buffered data in the standard streams), the exit status is changed to 120.

If this is indeed, correct behavior and Python 3.6 is catching something incorrect (I agree this is not necessarily the most elegant unit test) then I am happy to close this issue. Just wanted to be sure I at least reported it in case this was a real issue.

----------
components: IO
messages: 284437
nosy: John Hagen
priority: normal
severity: normal
status: open
title: Exit code 120 returned from Python unit test testing SystemExit
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29130>
_______________________________________

From report at bugs.python.org  Sun Jan  1 19:40:26 2017
From: report at bugs.python.org (R. David Murray)
Date: Mon, 02 Jan 2017 00:40:26 +0000
Subject: [issue29130] Exit code 120 returned from Python unit test testing
 SystemExit
In-Reply-To: <1483316898.11.0.0292789374137.issue29130@psf.upfronthosting.co.za>
Message-ID: <1483317626.37.0.834834786473.issue29130@psf.upfronthosting.co.za>


R. David Murray added the comment:

My guess would be that the problem that your NullWriter doesn't have a flush method.  But I'm not familiar with this change, so I'm just guessing.

----------
nosy: +r.david.murray

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29130>
_______________________________________

From report at bugs.python.org  Sun Jan  1 21:22:10 2017
From: report at bugs.python.org (Roundup Robot)
Date: Mon, 02 Jan 2017 02:22:10 +0000
Subject: [issue29071] IDLE doesn't highlight f-strings properly
In-Reply-To: <1482714978.44.0.603392455768.issue29071@psf.upfronthosting.co.za>
Message-ID: <20170102022206.21253.38756.F8590C81@psf.io>


Roundup Robot added the comment:

New changeset 62d3c0336df6 by Terry Jan Reedy in branch '3.6':
Issue #29071: Use local flags for IDLE colorizer string prefix matcher.
https://hg.python.org/cpython/rev/62d3c0336df6

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29071>
_______________________________________

From report at bugs.python.org  Sun Jan  1 21:24:52 2017
From: report at bugs.python.org (Terry J. Reedy)
Date: Mon, 02 Jan 2017 02:24:52 +0000
Subject: [issue29071] IDLE doesn't highlight f-strings properly
In-Reply-To: <1482714978.44.0.603392455768.issue29071@psf.upfronthosting.co.za>
Message-ID: <1483323892.43.0.697017408531.issue29071@psf.upfronthosting.co.za>


Terry J. Reedy added the comment:

Thank you for the new feature (I presume) and its application.  I disliked both writing and reading all the near duplication.  I am posting this to python-list as an example of what the new feature is good for.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29071>
_______________________________________

From report at bugs.python.org  Sun Jan  1 21:41:47 2017
From: report at bugs.python.org (Todd Rovito)
Date: Mon, 02 Jan 2017 02:41:47 +0000
Subject: [issue23670] Modifications to support iOS as a cross-compilation
 target
In-Reply-To: <1426400747.11.0.0527548590348.issue23670@psf.upfronthosting.co.za>
Message-ID: <1483324907.66.0.433311552495.issue23670@psf.upfronthosting.co.za>


Todd Rovito added the comment:

Russell, this is excellent work!!!!  I am truly amazed that within a couple of hours I had Python built and running for the iOS simulator this is a feat I didn't think was possible.  Perspective on me I am very familiar with Linux, C, and Python but know very little about iOS and XCode keep that in mind when reviewing my comments.

MacOS: 10.12.2
Xcode: 8.2.1 (8C1002)
iOS: 10.2 (14C89)
Python source code: 3.5.1

I applied your last patch file name 20160217.diff.  The first issue I ran into was clock_settime() is unavailable in iOS so I went into timemodule.c line 164 and commented out the line:
ret = clock_settime((clocked_t)clk_id, &tp);
then added this line
ret = 0;

Next I think a more serious problem was with Python/random.c line 92 "Python/random.c:92:19: error: implicit declaration of function 'getentropy' is invalid in C99 [-Werror,-Wimplicit-function-declaration]"

So I did the same thing as above commented out the lines with the getentropy function and added a line of res = 0.  Who needs entropy anyway?  I wonder if this problem is related to issue 29057 (http://bugs.python.org/issue29057).

Next in your README file you list a file called main.c when the actual code is main.m.

Next main.m was not compiling with Xcode so I replaced with this line after looking at your Python-iOS-template project on GitHub, as I am not qualified to make these types of changes:

-line 31 was not compiling so I replaced:
wpython_home = Py_DecodeLocale([python_home UTF8String], NULL);

-line 54 had a similar error so I replaced:
python_argv[0] = Py_DecodeLocale(main_script, NULL);

-line 56 had a similar error so I replaced with this line:
python_argv[i] = Py_DecodeLocale(argv[i], NULL);

-line 89 comment it out to remove the assertion failure, I am not sure what
this line is supposed to do but no assertion failure seems like a better result.

I didn't want to blindly attach a diff as I don't understand the Apple lingo but I wanted to provide feedback in the hope that it helps get this patch committed so iOS becomes a target for CPython in the near future.  Thanks!

----------
nosy: +Todd.Rovito

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23670>
_______________________________________

From report at bugs.python.org  Sun Jan  1 21:43:14 2017
From: report at bugs.python.org (Roundup Robot)
Date: Mon, 02 Jan 2017 02:43:14 +0000
Subject: [issue29024] Add Kivy entry to Graphic User Interface FAQ
Message-ID: <20170102024311.28640.90420.FAF2A31E@psf.io>


New submission from Roundup Robot:

New changeset d41aa32f7f3c by Berker Peksag in branch '3.5':
Issue #29024: Add Kivy entry to GUI FAQ
https://hg.python.org/cpython/rev/d41aa32f7f3c

New changeset ee25895d9d65 by Berker Peksag in branch '3.6':
Issue #29024: Merge from 3.5
https://hg.python.org/cpython/rev/ee25895d9d65

New changeset 4eb4cf6ac154 by Berker Peksag in branch 'default':
Issue #29024: Merge from 3.6
https://hg.python.org/cpython/rev/4eb4cf6ac154

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29024>
_______________________________________

From report at bugs.python.org  Sun Jan  1 21:43:48 2017
From: report at bugs.python.org (Berker Peksag)
Date: Mon, 02 Jan 2017 02:43:48 +0000
Subject: [issue29024] Add Kivy entry to Graphic User Interface FAQ
In-Reply-To: <20170102024311.28640.90420.FAF2A31E@psf.io>
Message-ID: <1483325028.72.0.0624429435663.issue29024@psf.upfronthosting.co.za>


Berker Peksag added the comment:

Thanks for the patch!

----------
nosy: +berker.peksag
resolution:  -> fixed
stage:  -> resolved
status: open -> closed
type:  -> enhancement
versions: +Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29024>
_______________________________________

From report at bugs.python.org  Sun Jan  1 21:49:06 2017
From: report at bugs.python.org (Roundup Robot)
Date: Mon, 02 Jan 2017 02:49:06 +0000
Subject: [issue29129] Copy/paste error in "8.13.14.1.1. Using auto"
In-Reply-To: <1483311839.39.0.725935533948.issue29129@psf.upfronthosting.co.za>
Message-ID: <20170102024903.124199.60880.6B444FD6@psf.io>


Roundup Robot added the comment:

New changeset 5698d84d0187 by Berker Peksag in branch '3.6':
Issue #29129: Fix typo in "Using auto" section
https://hg.python.org/cpython/rev/5698d84d0187

New changeset 337d78a4a7bf by Berker Peksag in branch 'default':
Issue #29129: Merge from 3.6
https://hg.python.org/cpython/rev/337d78a4a7bf

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29129>
_______________________________________

From report at bugs.python.org  Sun Jan  1 21:49:47 2017
From: report at bugs.python.org (Berker Peksag)
Date: Mon, 02 Jan 2017 02:49:47 +0000
Subject: [issue29129] Copy/paste error in "8.13.14.1.1. Using auto"
In-Reply-To: <1483311839.39.0.725935533948.issue29129@psf.upfronthosting.co.za>
Message-ID: <1483325387.27.0.558788388976.issue29129@psf.upfronthosting.co.za>


Berker Peksag added the comment:

Good catch! Thanks for the report, Michael.

----------
nosy: +berker.peksag
resolution:  -> fixed
stage:  -> resolved
status: open -> closed
type:  -> behavior
versions: +Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29129>
_______________________________________

From report at bugs.python.org  Sun Jan  1 21:49:54 2017
From: report at bugs.python.org (ppperry)
Date: Mon, 02 Jan 2017 02:49:54 +0000
Subject: [issue28952] csv.Sniffer().sniff(0) returns a value without the
 "strict" attribute
In-Reply-To: <1481571534.8.0.412951784743.issue28952@psf.upfronthosting.co.za>
Message-ID: <1483325394.98.0.24867902429.issue28952@psf.upfronthosting.co.za>


Changes by ppperry <mapreader at olum.org>:


----------
title: csv.Sniffer().sniff(0 returns a value without the "strict" attribute -> csv.Sniffer().sniff(0) returns a value without the "strict" attribute

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28952>
_______________________________________

From report at bugs.python.org  Sun Jan  1 21:59:05 2017
From: report at bugs.python.org (Roundup Robot)
Date: Mon, 02 Jan 2017 02:59:05 +0000
Subject: [issue29012] __bases__ is a tuple (possibly empty or a singleton)
In-Reply-To: <1482134407.67.0.814127407079.issue29012@psf.upfronthosting.co.za>
Message-ID: <20170102025902.25578.85467.9B0FB0B8@psf.io>


Roundup Robot added the comment:

New changeset 721df314d45a by Berker Peksag in branch '3.5':
Issue #29012: Remove outdated information about __bases__
https://hg.python.org/cpython/rev/721df314d45a

New changeset 019125fb6d66 by Berker Peksag in branch '3.6':
Issue #29012: Merge from 3.5
https://hg.python.org/cpython/rev/019125fb6d66

New changeset 454426dbff83 by Berker Peksag in branch 'default':
Issue #29012: Merge from 3.6
https://hg.python.org/cpython/rev/454426dbff83

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29012>
_______________________________________

From report at bugs.python.org  Sun Jan  1 21:59:48 2017
From: report at bugs.python.org (Berker Peksag)
Date: Mon, 02 Jan 2017 02:59:48 +0000
Subject: [issue29012] __bases__ is a tuple (possibly empty or a singleton)
In-Reply-To: <1482134407.67.0.814127407079.issue29012@psf.upfronthosting.co.za>
Message-ID: <1483325988.9.0.158606399812.issue29012@psf.upfronthosting.co.za>


Berker Peksag added the comment:

Thanks, Jim.

----------
nosy: +berker.peksag
resolution:  -> fixed
stage:  -> resolved
status: open -> closed
type:  -> behavior
versions:  -Python 3.3, Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29012>
_______________________________________

From report at bugs.python.org  Sun Jan  1 22:12:15 2017
From: report at bugs.python.org (Roundup Robot)
Date: Mon, 02 Jan 2017 03:12:15 +0000
Subject: [issue29013] zipfile: inconsistent doc for ZIP64 file size
In-Reply-To: <1482138273.34.0.375914911911.issue29013@psf.upfronthosting.co.za>
Message-ID: <20170102031213.21509.93331.E2AF9767@psf.io>


Roundup Robot added the comment:

New changeset 4685cd33087b by Berker Peksag in branch '3.5':
Issue #29013: Fix allowZip64 documentation
https://hg.python.org/cpython/rev/4685cd33087b

New changeset 7c5075a14459 by Berker Peksag in branch '3.6':
Issue #29013: Merge from 3.5
https://hg.python.org/cpython/rev/7c5075a14459

New changeset 6ca0f3fcf82f by Berker Peksag in branch 'default':
Issue #29013: Merge from 3.6
https://hg.python.org/cpython/rev/6ca0f3fcf82f

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29013>
_______________________________________

From report at bugs.python.org  Sun Jan  1 22:13:23 2017
From: report at bugs.python.org (Berker Peksag)
Date: Mon, 02 Jan 2017 03:13:23 +0000
Subject: [issue29013] zipfile: inconsistent doc for ZIP64 file size
In-Reply-To: <1482138273.34.0.375914911911.issue29013@psf.upfronthosting.co.za>
Message-ID: <1483326803.86.0.223111238622.issue29013@psf.upfronthosting.co.za>


Berker Peksag added the comment:

Thanks for the report and for the analysis, Monte!

----------
nosy: +berker.peksag
resolution:  -> fixed
stage:  -> resolved
status: open -> closed
type:  -> behavior
versions: +Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29013>
_______________________________________

From report at bugs.python.org  Sun Jan  1 22:35:45 2017
From: report at bugs.python.org (Roundup Robot)
Date: Mon, 02 Jan 2017 03:35:45 +0000
Subject: [issue28985] sqlite3 authorizer codes constants not up to date
In-Reply-To: <1481858693.19.0.888054078986.issue28985@psf.upfronthosting.co.za>
Message-ID: <20170102033541.25753.49006.BBE0183E@psf.io>


Roundup Robot added the comment:

New changeset b9c4139a1309 by Berker Peksag in branch 'default':
Issue #28985: Update authorizer constants in sqlite3 module
https://hg.python.org/cpython/rev/b9c4139a1309

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28985>
_______________________________________

From report at bugs.python.org  Sun Jan  1 22:36:40 2017
From: report at bugs.python.org (Berker Peksag)
Date: Mon, 02 Jan 2017 03:36:40 +0000
Subject: [issue28985] sqlite3 authorizer codes constants not up to date
In-Reply-To: <1481858693.19.0.888054078986.issue28985@psf.upfronthosting.co.za>
Message-ID: <1483328200.43.0.77824579255.issue28985@psf.upfronthosting.co.za>


Berker Peksag added the comment:

Thanks, Dingyuan!

----------
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28985>
_______________________________________

From report at bugs.python.org  Sun Jan  1 22:58:09 2017
From: report at bugs.python.org (Roundup Robot)
Date: Mon, 02 Jan 2017 03:58:09 +0000
Subject: [issue15812] inspect.getframeinfo() cannot show first line
In-Reply-To: <1346252079.7.0.669819400342.issue15812@psf.upfronthosting.co.za>
Message-ID: <20170102035806.123773.22044.E3F6BA39@psf.io>


Roundup Robot added the comment:

New changeset 15454cad5f27 by Berker Peksag in branch '3.5':
Issue #15812: inspect.getframeinfo() now correctly shows the first line of a context
https://hg.python.org/cpython/rev/15454cad5f27

New changeset 410caf255a09 by Berker Peksag in branch '3.6':
Issue #15812: Merge from 3.5
https://hg.python.org/cpython/rev/410caf255a09

New changeset 803c3c21c3bc by Berker Peksag in branch 'default':
Issue #15812: Merge from 3.6
https://hg.python.org/cpython/rev/803c3c21c3bc

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue15812>
_______________________________________

From report at bugs.python.org  Sun Jan  1 22:59:36 2017
From: report at bugs.python.org (Berker Peksag)
Date: Mon, 02 Jan 2017 03:59:36 +0000
Subject: [issue15812] inspect.getframeinfo() cannot show first line
In-Reply-To: <1346252079.7.0.669819400342.issue15812@psf.upfronthosting.co.za>
Message-ID: <1483329576.65.0.64032533649.issue15812@psf.upfronthosting.co.za>


Berker Peksag added the comment:

Thanks for the patch Sam and thanks for the ping Peter!

----------
components: +Library (Lib)
nosy: +berker.peksag
resolution:  -> fixed
stage:  -> resolved
status: open -> closed
type:  -> behavior
versions: +Python 3.5, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue15812>
_______________________________________

From report at bugs.python.org  Sun Jan  1 23:20:50 2017
From: report at bugs.python.org (Steve Dower)
Date: Mon, 02 Jan 2017 04:20:50 +0000
Subject: [issue29124] Freeze fails to compile on windows
In-Reply-To: <1483206665.88.0.0421614971405.issue29124@psf.upfronthosting.co.za>
Message-ID: <1483330850.47.0.39531534339.issue29124@psf.upfronthosting.co.za>


Steve Dower added the comment:

Is platform.architecture() the right way to determine what platform you're targeting? This will tell you the current OS, but typically you want to know the architecture of the Python process (since you often run 32-bit processes on 64-bit Windows). If you just want the Python process, checking "sys.winver.endswith('-32')" is the easiest way in 3.5 and later to see whether it's 32-bit Python.

Otherwise those changes look okay to me.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29124>
_______________________________________

From report at bugs.python.org  Sun Jan  1 23:22:54 2017
From: report at bugs.python.org (Steve Dower)
Date: Mon, 02 Jan 2017 04:22:54 +0000
Subject: [issue29075] Remove Windows Vista support
In-Reply-To: <1482775241.44.0.333291277311.issue29075@psf.upfronthosting.co.za>
Message-ID: <1483330974.09.0.149688583948.issue29075@psf.upfronthosting.co.za>


Steve Dower added the comment:

Only as certain as anyone else in the general public - I don't have any special information besides what has been published.

Since it's been published for the specific intent of helping 3rd parties plan their own deprecation cycles, I think it's okay to assume that it's correct. The chance of there being an XP-like backlash against Vista's end-of-life is very low.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29075>
_______________________________________

From report at bugs.python.org  Sun Jan  1 23:25:37 2017
From: report at bugs.python.org (Roundup Robot)
Date: Mon, 02 Jan 2017 04:25:37 +0000
Subject: [issue24932] Use proper command line parsing in _testembed
In-Reply-To: <1440481888.38.0.825081560944.issue24932@psf.upfronthosting.co.za>
Message-ID: <20170102042534.21253.45830.BFE7F6E4@psf.io>


Roundup Robot added the comment:

New changeset 69b2a6284f3d by Steve Dower in branch 'default':
Issue #24932: Use proper command line parsing in _testembed
https://hg.python.org/cpython/rev/69b2a6284f3d

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24932>
_______________________________________

From report at bugs.python.org  Sun Jan  1 23:27:09 2017
From: report at bugs.python.org (Steve Dower)
Date: Mon, 02 Jan 2017 04:27:09 +0000
Subject: [issue24932] Use proper command line parsing in _testembed
In-Reply-To: <1440481888.38.0.825081560944.issue24932@psf.upfronthosting.co.za>
Message-ID: <1483331229.87.0.00794897586929.issue24932@psf.upfronthosting.co.za>


Steve Dower added the comment:

Patch committed without modification. I'll keep an eye on the buildbots just in case, though I may end up going offline before they get to this changeset.

----------
assignee:  -> steve.dower
stage:  -> commit review

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24932>
_______________________________________

From report at bugs.python.org  Sun Jan  1 23:31:05 2017
From: report at bugs.python.org (Roundup Robot)
Date: Mon, 02 Jan 2017 04:31:05 +0000
Subject: [issue29057] Compiler failure on Mac OS X - sys/random.h
In-Reply-To: <1482514649.72.0.460227366509.issue29057@psf.upfronthosting.co.za>
Message-ID: <20170102043102.10445.38416.B7A2801D@psf.io>


Roundup Robot added the comment:

New changeset 9ab75789c554 by Benjamin Peterson in branch '3.5':
only include sys/random.h if it seems like it might have something useful (#29057)
https://hg.python.org/cpython/rev/9ab75789c554

New changeset 74eb71b91112 by Benjamin Peterson in branch '2.7':
only include sys/random.h if it seems like it might have something useful (#29057)
https://hg.python.org/cpython/rev/74eb71b91112

New changeset 5d0cb1fd79ea by Benjamin Peterson in branch '3.6':
merge 3.5 (#29057)
https://hg.python.org/cpython/rev/5d0cb1fd79ea

New changeset bb0d145b43a3 by Benjamin Peterson in branch 'default':
merge 3.6 (#29057)
https://hg.python.org/cpython/rev/bb0d145b43a3

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29057>
_______________________________________

From report at bugs.python.org  Sun Jan  1 23:40:03 2017
From: report at bugs.python.org (Steve Dower)
Date: Mon, 02 Jan 2017 04:40:03 +0000
Subject: [issue29059] Windows: Python not using ANSI compatible console
In-Reply-To: <1482543191.02.0.407458515739.issue29059@psf.upfronthosting.co.za>
Message-ID: <1483332003.21.0.735382507551.issue29059@psf.upfronthosting.co.za>


Steve Dower added the comment:

> IIRC, ANSI is somewhat incompatible with sending random binary gibberish to the screen, as people accidentally do with TYPE sometimes :) But the random binary gibberish may contain ANSI control sequences... That's why I'm negative on making it a default.

I don't actually know how big a deal this would be. I seem to get identical results from "print(''.join(chr(x) for x in range(32)))" both with and without the VT100 flag set, though of course "print('\033[91m')" behaves differently.

But given "on by default" isn't popular, and "off by default" implies adding new public API that is already available either as a short ctypes snippet or a number of 3rd-party libraries, I think we should consider this rejected.

For future reference, the python-ideas thread starts with https://mail.python.org/pipermail/python-ideas/2016-December/044033.html

----------
resolution:  -> rejected
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29059>
_______________________________________

From report at bugs.python.org  Sun Jan  1 23:42:04 2017
From: report at bugs.python.org (Steve Dower)
Date: Mon, 02 Jan 2017 04:42:04 +0000
Subject: [issue29014] Python 3.5.2 installer doesn't register IDLE .py
 extensions on Windows 10
In-Reply-To: <1482147098.02.0.695368643489.issue29014@psf.upfronthosting.co.za>
Message-ID: <1483332124.3.0.83470095262.issue29014@psf.upfronthosting.co.za>


Changes by Steve Dower <steve.dower at python.org>:


----------
resolution:  -> not a bug
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29014>
_______________________________________

From report at bugs.python.org  Sun Jan  1 23:44:42 2017
From: report at bugs.python.org (Steve Dower)
Date: Mon, 02 Jan 2017 04:44:42 +0000
Subject: [issue29063] Fixed timemodule compile warnings.
In-Reply-To: <1482576109.57.0.29725831282.issue29063@psf.upfronthosting.co.za>
Message-ID: <1483332282.91.0.974609785855.issue29063@psf.upfronthosting.co.za>


Steve Dower added the comment:

Any reason we can't make gmtoff a time_t instead of an int?

If we're going to truncate values to get rid of the warnings, I'd like to also see either proof that it will never exceed the size of an int (which may be a simple comment, but it's not obvious that this is the case from what appears in the patch), or an assertion when it does overflow.

But since we're presumably passing the value back into Python as an int, expanding the destination variable to fit all possible values is best.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29063>
_______________________________________

From report at bugs.python.org  Sun Jan  1 23:45:52 2017
From: report at bugs.python.org (Larry Hastings)
Date: Mon, 02 Jan 2017 04:45:52 +0000
Subject: [issue29057] Compiler failure on Mac OS X - sys/random.h
In-Reply-To: <1482514649.72.0.460227366509.issue29057@psf.upfronthosting.co.za>
Message-ID: <1483332352.45.0.331825359623.issue29057@psf.upfronthosting.co.za>


Larry Hastings added the comment:

Can this be marked closed now?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29057>
_______________________________________

From report at bugs.python.org  Sun Jan  1 23:49:14 2017
From: report at bugs.python.org (Steve Dower)
Date: Mon, 02 Jan 2017 04:49:14 +0000
Subject: [issue28781] On Installation of 3.5 Python get error message
In-Reply-To: <1479918351.94.0.34425443448.issue28781@psf.upfronthosting.co.za>
Message-ID: <1483332554.6.0.337724154837.issue28781@psf.upfronthosting.co.za>


Changes by Steve Dower <steve.dower at python.org>:


----------
resolution:  -> out of date
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28781>
_______________________________________

From report at bugs.python.org  Sun Jan  1 23:50:10 2017
From: report at bugs.python.org (Steve Dower)
Date: Mon, 02 Jan 2017 04:50:10 +0000
Subject: [issue28592] Installation freezes on C Runtime Install
In-Reply-To: <1478129295.09.0.95134744404.issue28592@psf.upfronthosting.co.za>
Message-ID: <1483332610.88.0.232881442423.issue28592@psf.upfronthosting.co.za>


Changes by Steve Dower <steve.dower at python.org>:


----------
resolution:  -> out of date
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28592>
_______________________________________

From report at bugs.python.org  Sun Jan  1 23:54:11 2017
From: report at bugs.python.org (Ned Deily)
Date: Mon, 02 Jan 2017 04:54:11 +0000
Subject: [issue29057] Compiler failure on Mac OS X - sys/random.h
In-Reply-To: <1482514649.72.0.460227366509.issue29057@psf.upfronthosting.co.za>
Message-ID: <1483332851.28.0.183883979304.issue29057@psf.upfronthosting.co.za>


Ned Deily added the comment:

I just did a quick build test of a few 3.5 configurations with various macOS releases and Benjamin's applied change seems to fix the previous build failures.  No idea about OpenBSD.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29057>
_______________________________________

From report at bugs.python.org  Sun Jan  1 23:58:13 2017
From: report at bugs.python.org (Anand Reddy Pandikunta)
Date: Mon, 02 Jan 2017 04:58:13 +0000
Subject: [issue12276] 3.x ignores sys.tracebacklimit=0
In-Reply-To: <1307436501.83.0.59684150491.issue12276@psf.upfronthosting.co.za>
Message-ID: <1483333093.8.0.464627960702.issue12276@psf.upfronthosting.co.za>


Anand Reddy Pandikunta added the comment:

Update patch with better assertions

----------
Added file: http://bugs.python.org/file46110/Limit-traceback-if-sys.tracebacklimit-is-set.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue12276>
_______________________________________

From report at bugs.python.org  Mon Jan  2 00:45:41 2017
From: report at bugs.python.org (mike peremsky)
Date: Mon, 02 Jan 2017 05:45:41 +0000
Subject: [issue29131] Calling printf from the cdll does not print the full
 string
Message-ID: <1483335941.28.0.90423143294.issue29131@psf.upfronthosting.co.za>


New submission from mike peremsky:

I am going throught he Gray Hat Python book and installed Python 3.7 (32-bit) on a windows x64 machine. The following code will only print the first character of the passed string argument. The same code run on Python 2.7 will print the correct string value.


from ctypes import *
 
msvcrt = cdll.msvcrt
 
message_string = "Hello World!\n"
msvcrt.printf("Testing: %s", message_string)
 
Output:
T

----------
components: ctypes
messages: 284464
nosy: mperemsky
priority: normal
severity: normal
status: open
title: Calling printf from the cdll does not print the full string
type: behavior
versions: Python 3.3, Python 3.4, Python 3.5, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29131>
_______________________________________

From report at bugs.python.org  Mon Jan  2 00:48:20 2017
From: report at bugs.python.org (Steve Dower)
Date: Mon, 02 Jan 2017 05:48:20 +0000
Subject: [issue29131] Calling printf from the cdll does not print the full
 string
In-Reply-To: <1483335941.28.0.90423143294.issue29131@psf.upfronthosting.co.za>
Message-ID: <1483336100.25.0.409124131999.issue29131@psf.upfronthosting.co.za>


Steve Dower added the comment:

In Python 3, this passes a wchar_t* string, but printf('%s') expects a char* string.

You may want to start by looking at the tutorial at https://docs.python.org/3/tutorial/index.html to get a feel for what builtins are available. Using ctypes is fairly advanced functionality that very rarely needs to be used.

----------
nosy: +steve.dower

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29131>
_______________________________________

From report at bugs.python.org  Mon Jan  2 00:51:23 2017
From: report at bugs.python.org (Steve Dower)
Date: Mon, 02 Jan 2017 05:51:23 +0000
Subject: [issue29131] Calling printf from the cdll does not print the full
 string
In-Reply-To: <1483335941.28.0.90423143294.issue29131@psf.upfronthosting.co.za>
Message-ID: <1483336283.08.0.805633782303.issue29131@psf.upfronthosting.co.za>


Steve Dower added the comment:

Though if you do really want to use the (very) old msvcrt DLL rather than the proper functionality, calling cdd.msvcrt.wprintf will behave as you expect, or prefixing the strings with b (e.g. b"Testing") will pass it as bytes rather than wchar_t.

But unless all these suggestions make perfect sense to you, starting with the tutorial is probably best :)

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29131>
_______________________________________

From report at bugs.python.org  Mon Jan  2 00:58:13 2017
From: report at bugs.python.org (Larry Hastings)
Date: Mon, 02 Jan 2017 05:58:13 +0000
Subject: [issue29057] Compiler failure on Mac OS X - sys/random.h
In-Reply-To: <1482514649.72.0.460227366509.issue29057@psf.upfronthosting.co.za>
Message-ID: <1483336693.57.0.984746954863.issue29057@psf.upfronthosting.co.za>


Larry Hastings added the comment:

I'm making an executive decision to not hold up the 3.5.3rc1 release for OpenBSD.  Hopefully the OpenBSD folks can make sure it works for them before 3.5.3 final ships in two weeks.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29057>
_______________________________________

From report at bugs.python.org  Mon Jan  2 01:12:48 2017
From: report at bugs.python.org (Steve Dower)
Date: Mon, 02 Jan 2017 06:12:48 +0000
Subject: [issue24932] Use proper command line parsing in _testembed
In-Reply-To: <1440481888.38.0.825081560944.issue24932@psf.upfronthosting.co.za>
Message-ID: <1483337568.51.0.812971066697.issue24932@psf.upfronthosting.co.za>


Steve Dower added the comment:

Buildbots are happy, so I'm calling this done. Jumping straight to getopt is overengineering right now in my opinion, but if there's a need for it with a future test we can consider it then.

----------
resolution:  -> fixed
stage: commit review -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24932>
_______________________________________

From report at bugs.python.org  Mon Jan  2 01:22:28 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Mon, 02 Jan 2017 06:22:28 +0000
Subject: [issue29013] zipfile: inconsistent doc for ZIP64 file size
In-Reply-To: <1482138273.34.0.375914911911.issue29013@psf.upfronthosting.co.za>
Message-ID: <1483338148.94.0.349558659091.issue29013@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

The documentation was correct. The zipfile module supports *reading* ZIP files up to 4 GiB without the ZIP64 extension, but it requires allowZip64=True for *writing* over 2 GiB files to the ZIP file.

The 2 GiB limit is safer because generated ZIP files can be read by implementations that interpret 32-bit sizes as signed. For example Java don't have unsigned integers. And zipfile and zipimport in old Python versions unpack some fields as signed integers.

----------
nosy: +serhiy.storchaka
resolution: fixed -> 
stage: resolved -> 
status: closed -> open

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29013>
_______________________________________

From report at bugs.python.org  Mon Jan  2 01:27:24 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Mon, 02 Jan 2017 06:27:24 +0000
Subject: [issue29012] __bases__ is a tuple (possibly empty or a singleton)
In-Reply-To: <1482134407.67.0.814127407079.issue29012@psf.upfronthosting.co.za>
Message-ID: <1483338444.98.0.225790659044.issue29012@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

I think "possible a singleton" can be removed too. It doesn't add any useful information, any tuple is possible a singleton.

----------
nosy: +serhiy.storchaka

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29012>
_______________________________________

From report at bugs.python.org  Mon Jan  2 01:36:36 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Mon, 02 Jan 2017 06:36:36 +0000
Subject: [issue15812] inspect.getframeinfo() cannot show first line
In-Reply-To: <1346252079.7.0.669819400342.issue15812@psf.upfronthosting.co.za>
Message-ID: <1483338996.52.0.245975515271.issue15812@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

start is bounded to 0 twice.

    start = max(start, 0)
    start = max(0, min(start, len(lines) - context))

The first line can be just removed. Or two above lines can be rewritten as:

    start = min(start, len(lines) - context)
    start = max(start, 0)

----------
nosy: +serhiy.storchaka

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue15812>
_______________________________________

From report at bugs.python.org  Mon Jan  2 02:06:47 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Mon, 02 Jan 2017 07:06:47 +0000
Subject: [issue29071] IDLE doesn't highlight f-strings properly
In-Reply-To: <1483323892.43.0.697017408531.issue29071@psf.upfronthosting.co.za>
Message-ID: <11869363.L3s7SK8dnX@raxxla>


Serhiy Storchaka added the comment:

Thank you for advertising  the new feature. Would you like to improve the 
documentation? It always was the hardest part to me.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29071>
_______________________________________

From report at bugs.python.org  Mon Jan  2 02:49:02 2017
From: report at bugs.python.org (Eryk Sun)
Date: Mon, 02 Jan 2017 07:49:02 +0000
Subject: [issue29124] Freeze fails to compile on windows
In-Reply-To: <1483206665.88.0.0421614971405.issue29124@psf.upfronthosting.co.za>
Message-ID: <1483343342.51.0.110463973625.issue29124@psf.upfronthosting.co.za>


Eryk Sun added the comment:

Using platform.architecture [1] is fine here. `bits` defaults to the size of a pointer in the current process, i.e. `struct.calcsize('P') * 8`. 

On Windows, it's useless for anything except the default parameters. It doesn't call GetBinaryType [2] to distinguish 32-bit and 64-bit executables, so from a 64-bit process it will incorrectly return that a 32-bit executable is 64-bit.

[1]: https://docs.python.org/3/library/platform.html#platform.architecture
[2]: https://msdn.microsoft.com/en-us/library/aa364819

----------
nosy: +eryksun

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29124>
_______________________________________

From report at bugs.python.org  Mon Jan  2 03:09:54 2017
From: report at bugs.python.org (Ned Deily)
Date: Mon, 02 Jan 2017 08:09:54 +0000
Subject: [issue24725] test_socket testFDPassEmpty fails on OS X 10.11+ with
 "Cannot allocate memory"
In-Reply-To: <1437896131.77.0.341264720465.issue24725@psf.upfronthosting.co.za>
Message-ID: <1483344594.39.0.815689433503.issue24725@psf.upfronthosting.co.za>


Ned Deily added the comment:

This issue slipped off the radar (so to speak) after being closed.  For one, the tests also fail on 3.5.x (and probably earlier systems) and fail on macOS 10.12, not just 10.11.  And, two, I'm not sure if anyone opened an issue with Apple about it.  I'm re-opening it and assigning it to me to address those things.

----------
assignee:  -> ned.deily
resolution: fixed -> 
stage: resolved -> needs patch
status: closed -> open
title: test_socket testFDPassEmpty fails on OS X 10.11 DP with "Cannot allocate memory" -> test_socket testFDPassEmpty fails on OS X 10.11+ with "Cannot allocate memory"
versions:  -Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24725>
_______________________________________

From report at bugs.python.org  Mon Jan  2 03:14:50 2017
From: report at bugs.python.org (Monte Davidoff)
Date: Mon, 02 Jan 2017 08:14:50 +0000
Subject: [issue29013] zipfile: inconsistent doc for ZIP64 file size
In-Reply-To: <1482138273.34.0.375914911911.issue29013@psf.upfronthosting.co.za>
Message-ID: <1483344890.13.0.900364558946.issue29013@psf.upfronthosting.co.za>


Monte Davidoff added the comment:

Serhiy, thank you for the correction and the additional information. I tried reading a zip file larger than 4 GiB with allowZip64=False, and it worked, so it looks like allowZip64 only applies to writing. I suggest we fix the inconsistency in the documentation as follows:

(1) In the description of the zipfile.ZipFile class, revert the change back to:

"If allowZip64 is True (the default) zipfile will create ZIP files that use the ZIP64 extensions when the zipfile is larger than 2 GiB."

(2) Change the second paragraph in the zipfile module documentation from:

"It can handle ZIP files that use the ZIP64 extensions (that is ZIP files that are more than 4 GiB in size)."

to:

"It can handle ZIP files that use the ZIP64 extensions (that is ZIP files that are more than 2 GiB in size)."

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29013>
_______________________________________

From report at bugs.python.org  Mon Jan  2 03:38:06 2017
From: report at bugs.python.org (Evan)
Date: Mon, 02 Jan 2017 08:38:06 +0000
Subject: [issue29128] No way to instsall win32com on python 3.6
In-Reply-To: <1483301522.24.0.353841386115.issue29128@psf.upfronthosting.co.za>
Message-ID: <1483346286.0.0.554567460727.issue29128@psf.upfronthosting.co.za>


Evan added the comment:

This package doesn't yet have a wheel for Python 3.6, so it falls back on the source distribution, and the setup.py file doesn't run under Python 3.

You can either wait for the maintainer to upload the new wheel to PyPI, or take one of the recently built installers from https://sourceforge.net/projects/pywin32/files/pywin32/Build%20220/

----------
nosy: +evan_

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29128>
_______________________________________

From report at bugs.python.org  Mon Jan  2 03:49:45 2017
From: report at bugs.python.org (Chi Hsuan Yen)
Date: Mon, 02 Jan 2017 08:49:45 +0000
Subject: [issue29128] No way to instsall win32com on python 3.6
In-Reply-To: <1483301522.24.0.353841386115.issue29128@psf.upfronthosting.co.za>
Message-ID: <1483346985.46.0.24890948736.issue29128@psf.upfronthosting.co.za>


Chi Hsuan Yen added the comment:

https://github.com/pywin32/pypiwin32/pull/2

Anyway this is not a Python issue but pywin32's.

----------
nosy: +Chi Hsuan Yen

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29128>
_______________________________________

From report at bugs.python.org  Mon Jan  2 04:37:02 2017
From: report at bugs.python.org (Marc-Andre Lemburg)
Date: Mon, 02 Jan 2017 09:37:02 +0000
Subject: [issue29006] 2.7.13 _sqlite more prone to "database table is locked"
In-Reply-To: <1482095939.25.0.891516594682.issue29006@psf.upfronthosting.co.za>
Message-ID: <1483349822.89.0.823345141346.issue29006@psf.upfronthosting.co.za>


Marc-Andre Lemburg added the comment:

As general comment, I think you'd have to check which operation modes SQLite supports for the case of a transaction commit/rollback in the light of open cursors.

ODBC defines the following cases and each data source can specify a different behavior (https://msdn.microsoft.com/en-us/library/ms711769%28v=vs.85%29.aspx):

 * All cursors are closed, and access plans for prepared statements on that connection are deleted.

 * All cursors are closed, and access plans for prepared statements on that connection remain intact.

 * All cursors remain open, and access plans for prepared statements on that connection remain intact.

The Python DB-API does not specify any particular behavior (since this depends on the database backend), so I guess pysqlite3 should implement whatever SQLite supports per default in such cases (or make this configurable).

BTW: It is quite common for databases to support the second mode of operation:

 * All cursors are closed, and access plans for prepared statements on that connection remain intact.

since this allows pooling cursors to cache often used access plans.

----------
nosy: +lemburg

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29006>
_______________________________________

From report at bugs.python.org  Mon Jan  2 07:34:52 2017
From: report at bugs.python.org (Terry J. Reedy)
Date: Mon, 02 Jan 2017 12:34:52 +0000
Subject: [issue29071] IDLE doesn't highlight f-strings properly
In-Reply-To: <1482714978.44.0.603392455768.issue29071@psf.upfronthosting.co.za>
Message-ID: <1483360492.57.0.379235684473.issue29071@psf.upfronthosting.co.za>


Terry J. Reedy added the comment:

The doc appears to be carefully accurate, but the second sentence especially is a bit awkward.  I will open a new issue.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29071>
_______________________________________

From report at bugs.python.org  Mon Jan  2 07:56:23 2017
From: report at bugs.python.org (Evan)
Date: Mon, 02 Jan 2017 12:56:23 +0000
Subject: [issue29132] shlex.shlex with punctuation_chars and posix doesn't
 handle punctuation next to quotes
Message-ID: <1483361783.89.0.00336686739256.issue29132@psf.upfronthosting.co.za>


New submission from Evan:

When both punctuation_chars and posix are used, any punctuation directly next to quoted characters is interpreted as a single token.

>>> from shlex import shlex
>>> list(shlex('>"a"', posix=True))
['>', 'a']
>>> list(shlex('>"a"', punctuation_chars=True))
['>', '"a"']
>>> list(shlex('>"a"', posix=True, punctuation_chars=True))
['>a']  # should be ['>', 'a']

----------
components: Library (Lib)
messages: 284480
nosy: evan_, r.david.murray, vinay.sajip
priority: normal
severity: normal
status: open
title: shlex.shlex with punctuation_chars and posix doesn't handle punctuation next to quotes
versions: Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29132>
_______________________________________

From report at bugs.python.org  Mon Jan  2 08:00:12 2017
From: report at bugs.python.org (Evan)
Date: Mon, 02 Jan 2017 13:00:12 +0000
Subject: [issue29133] Minor inaccuracy in shlex.shlex punctuation_chars example
Message-ID: <1483362012.78.0.704413629859.issue29133@psf.upfronthosting.co.za>


New submission from Evan:

https://docs.python.org/3/library/shlex.html#improved-compatibility-with-shells

The code sample here does not match the output - the first line is labelled 'New' and the second line 'Old'.

----------
assignee: docs at python
components: Documentation
messages: 284481
nosy: docs at python, evan_
priority: normal
severity: normal
status: open
title: Minor inaccuracy in shlex.shlex punctuation_chars example
versions: Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29133>
_______________________________________

From report at bugs.python.org  Mon Jan  2 08:01:22 2017
From: report at bugs.python.org (Evan)
Date: Mon, 02 Jan 2017 13:01:22 +0000
Subject: [issue28595] shlex.shlex should not augment wordchars
In-Reply-To: <1478166136.94.0.995190174918.issue28595@psf.upfronthosting.co.za>
Message-ID: <1483362082.53.0.330064551283.issue28595@psf.upfronthosting.co.za>


Evan added the comment:

I've attached a more complete patch. This takes the conservative approach of retaining all functionality of 3.6 and treating this as a new feature for 3.7. I don't think this is suitable for 3.6.1 given this new behavior contradicts what is currently written in the documentation (though this can be discussed further).

I've also run into a couple of other bugs that I've made separate issues for:

* The example in the documentation mixes up 'Old' and 'New' (issue29133)
* The updated example contains a bug (the '>abc' token should actually be two tokens) (issue29132)

These should both be fixed in 3.6.1 regardless of where this goes.

----------
versions: +Python 3.7
Added file: http://bugs.python.org/file46111/shlex.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28595>
_______________________________________

From report at bugs.python.org  Mon Jan  2 08:05:54 2017
From: report at bugs.python.org (Christian Heimes)
Date: Mon, 02 Jan 2017 13:05:54 +0000
Subject: [issue29125] Shell injection via TIX_LIBRARY when using tkinter.tix
In-Reply-To: <1483210830.52.0.920210756798.issue29125@psf.upfronthosting.co.za>
Message-ID: <1483362354.87.0.342199301383.issue29125@psf.upfronthosting.co.za>


Changes by Christian Heimes <lists at cheimes.de>:


----------
nosy: +benjamin.peterson, christian.heimes, larry, ned.deily
priority: normal -> release blocker

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29125>
_______________________________________

From report at bugs.python.org  Mon Jan  2 08:30:01 2017
From: report at bugs.python.org (R. David Murray)
Date: Mon, 02 Jan 2017 13:30:01 +0000
Subject: [issue29131] Calling printf from the cdll does not print the full
 string
In-Reply-To: <1483335941.28.0.90423143294.issue29131@psf.upfronthosting.co.za>
Message-ID: <1483363801.5.0.546171933415.issue29131@psf.upfronthosting.co.za>


Changes by R. David Murray <rdmurray at bitdance.com>:


----------
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29131>
_______________________________________

From report at bugs.python.org  Mon Jan  2 09:22:03 2017
From: report at bugs.python.org (Steve Dower)
Date: Mon, 02 Jan 2017 14:22:03 +0000
Subject: [issue29124] Freeze fails to compile on windows
In-Reply-To: <1483206665.88.0.0421614971405.issue29124@psf.upfronthosting.co.za>
Message-ID: <1483366923.64.0.38899011976.issue29124@psf.upfronthosting.co.za>


Steve Dower added the comment:

Ah, I was thinking of platform.machine().

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29124>
_______________________________________

From report at bugs.python.org  Mon Jan  2 10:38:07 2017
From: report at bugs.python.org (Evan)
Date: Mon, 02 Jan 2017 15:38:07 +0000
Subject: [issue29132] shlex.shlex with punctuation_chars and posix doesn't
 handle punctuation next to quotes
In-Reply-To: <1483361783.89.0.00336686739256.issue29132@psf.upfronthosting.co.za>
Message-ID: <1483371487.43.0.86061633579.issue29132@psf.upfronthosting.co.za>


Evan added the comment:

I've attached a simplistic patch which fixes the problem. I suspect a better fix might be to shuffle the elif branches around so the extra condition isn't necessary. Vinay, what are your thoughts?

----------
keywords: +patch
Added file: http://bugs.python.org/file46112/shlex-posix-quote.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29132>
_______________________________________

From report at bugs.python.org  Mon Jan  2 10:43:20 2017
From: report at bugs.python.org (Larry Hastings)
Date: Mon, 02 Jan 2017 15:43:20 +0000
Subject: [issue29125] Shell injection via TIX_LIBRARY when using tkinter.tix
In-Reply-To: <1483210830.52.0.920210756798.issue29125@psf.upfronthosting.co.za>
Message-ID: <1483371800.28.0.77011807675.issue29125@psf.upfronthosting.co.za>


Larry Hastings added the comment:

This code hasn't changed in years.  So while I believe it's a security bug and should be fixed, I don't know if I agree it's a bad enough security bug to stop Python 3.5.3rc1, which is literally in the middle of the release process.

I'm guessing this is easily fixed (if not os.path.isfile(tixlib): return), so how about we release 3.5.3rc1 with this bug and I'll cherry-pick this fix for 3.5.3 final.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29125>
_______________________________________

From report at bugs.python.org  Mon Jan  2 10:59:30 2017
From: report at bugs.python.org (Christian Heimes)
Date: Mon, 02 Jan 2017 15:59:30 +0000
Subject: [issue29125] Shell injection via TIX_LIBRARY when using tkinter.tix
In-Reply-To: <1483210830.52.0.920210756798.issue29125@psf.upfronthosting.co.za>
Message-ID: <1483372770.14.0.268247080334.issue29125@psf.upfronthosting.co.za>


Christian Heimes added the comment:

yeah, sounds totally fine to me. It's a low risk change and the issue has a small attack surface. I set the priority to release blocker to draw your attention.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29125>
_______________________________________

From report at bugs.python.org  Mon Jan  2 11:01:10 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Mon, 02 Jan 2017 16:01:10 +0000
Subject: [issue29125] Shell injection via TIX_LIBRARY when using tkinter.tix
In-Reply-To: <1483210830.52.0.920210756798.issue29125@psf.upfronthosting.co.za>
Message-ID: <1483372870.17.0.19851900193.issue29125@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

I agreed that this security issue has low severity. Only applications that use Tix are vulnerable, and this is very small number of applications.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29125>
_______________________________________

From report at bugs.python.org  Mon Jan  2 11:13:17 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Mon, 02 Jan 2017 16:13:17 +0000
Subject: [issue29125] Shell injection via TIX_LIBRARY when using tkinter.tix
In-Reply-To: <1483210830.52.0.920210756798.issue29125@psf.upfronthosting.co.za>
Message-ID: <1483373597.65.0.469474866629.issue29125@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

Here is a fix.

----------
keywords: +patch
Added file: http://bugs.python.org/file46113/tix_library_shell_injection.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29125>
_______________________________________

From report at bugs.python.org  Mon Jan  2 11:13:31 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Mon, 02 Jan 2017 16:13:31 +0000
Subject: [issue29125] Shell injection via TIX_LIBRARY when using tkinter.tix
In-Reply-To: <1483210830.52.0.920210756798.issue29125@psf.upfronthosting.co.za>
Message-ID: <1483373611.17.0.855791668573.issue29125@psf.upfronthosting.co.za>


Changes by Serhiy Storchaka <storchaka+cpython at gmail.com>:


----------
stage:  -> patch review

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29125>
_______________________________________

From report at bugs.python.org  Mon Jan  2 11:30:05 2017
From: report at bugs.python.org (Zachary Ware)
Date: Mon, 02 Jan 2017 16:30:05 +0000
Subject: [issue29128] No way to instsall win32com on python 3.6
In-Reply-To: <1483301522.24.0.353841386115.issue29128@psf.upfronthosting.co.za>
Message-ID: <1483374605.89.0.322914331508.issue29128@psf.upfronthosting.co.za>


Changes by Zachary Ware <zachary.ware at gmail.com>:


----------
resolution:  -> third party
stage:  -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29128>
_______________________________________

From report at bugs.python.org  Mon Jan  2 11:54:11 2017
From: report at bugs.python.org (YoSTEALTH)
Date: Mon, 02 Jan 2017 16:54:11 +0000
Subject: [issue29134] contextlib doc bug
Message-ID: <1483376051.72.0.678487921628.issue29134@psf.upfronthosting.co.za>


New submission from YoSTEALTH:

Link: https://docs.python.org/3/library/contextlib.html#contextlib.ContextDecorator

"from contextlib import ContextDecorator

class mycontext(ContextBaseClass, ContextDecorator):"

"ContextBaseClass" is referenced but its no where to be found in source.

----------
messages: 284489
nosy: YoSTEALTH
priority: normal
severity: normal
status: open
title: contextlib doc bug

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29134>
_______________________________________

From report at bugs.python.org  Mon Jan  2 11:57:50 2017
From: report at bugs.python.org (Chris Warrick)
Date: Mon, 02 Jan 2017 16:57:50 +0000
Subject: [issue29134] contextlib doc bug
In-Reply-To: <1483376051.72.0.678487921628.issue29134@psf.upfronthosting.co.za>
Message-ID: <1483376270.38.0.681040854432.issue29134@psf.upfronthosting.co.za>


Changes by Chris Warrick <kwpolska at gmail.com>:


----------
nosy: +Kwpolska

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29134>
_______________________________________

From report at bugs.python.org  Mon Jan  2 12:06:09 2017
From: report at bugs.python.org (Marco Buttu)
Date: Mon, 02 Jan 2017 17:06:09 +0000
Subject: [issue29133] Minor inaccuracy in shlex.shlex punctuation_chars example
In-Reply-To: <1483362012.78.0.704413629859.issue29133@psf.upfronthosting.co.za>
Message-ID: <1483376769.74.0.330747878241.issue29133@psf.upfronthosting.co.za>


Marco Buttu added the comment:

Thank you. The patch fixes it and makes the example under doctest.

----------
keywords: +patch
nosy: +marco.buttu
Added file: http://bugs.python.org/file46114/issue29133.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29133>
_______________________________________

From report at bugs.python.org  Mon Jan  2 12:41:32 2017
From: report at bugs.python.org (SilentGhost)
Date: Mon, 02 Jan 2017 17:41:32 +0000
Subject: [issue29134] contextlib doc bug
In-Reply-To: <1483376051.72.0.678487921628.issue29134@psf.upfronthosting.co.za>
Message-ID: <1483378892.48.0.885759081478.issue29134@psf.upfronthosting.co.za>


SilentGhost added the comment:

ContextBaseClass is meant to represent a user-defined class that is a parent of mycontext.

----------
nosy: +SilentGhost

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29134>
_______________________________________

From report at bugs.python.org  Mon Jan  2 13:36:37 2017
From: report at bugs.python.org (Cornelius Diekmann)
Date: Mon, 02 Jan 2017 18:36:37 +0000
Subject: [issue29070] Integration tests for pty.spawn on Linux and all other
 platforms
In-Reply-To: <1482684225.17.0.467768789806.issue29070@psf.upfronthosting.co.za>
Message-ID: <1483382197.74.0.0151275374088.issue29070@psf.upfronthosting.co.za>


Cornelius Diekmann added the comment:

I did a larger update to my proposed patch. I read the pty man pages and posix standard and designed a test suite which documents the expected behavior of the pty module. The test suite is biased towards my Linux system, but I respect the posix standard. Tested on Linux, OS X, and FreeBSD.

The patch introduces no behavioral changes to the actual pty.py module! It should be easy to merge.

Changes to pty.py: I made _select and _execlp local definitions of the pty module to ease mocking them in the test suite. A comment finally explains why pty.py defines STDIN_FILENO, STDOUT_FILENO, ... itself.

All tests pass on Linux.
Tests fail on macOS 10.6.8 and FreeBSD 11. The tests just hang. The patch suggested in issue26228 solves these issues. With this patch (or even without patch on Linux), the test suite needs about 1 to 2 seconds). I did not include the patch of issue26228 into the patch proposed here since I promised no change in behavior to the pty module. We now have a lot of test cases for the pty module and we are now finally aware that something is broken in the module. It's better to have failing tests than to have no tests at all :-)

I included the change to the documentation I proposed in issue29054.

The test suite does several things:

* SmallPtyTests mainly tests the _copy loop and documents current behavior. As issue issue26228 has shown, this behavior is not what one would expect. When adding the patch for the mentioned issue, one also needs to adapt test__copy_eof_on_all and test__copy_eof_on_master. These tests will make it easier to resolve issue26228 since they provide a clear documentation of the current behavior and contribute the missing test cases.

* PtyTest is left unchanged.

* One has probably noticed that the current test_pty.py suite randomly fails. I introduced _os_read_exactly and _os_read_only to make all new tests deterministic and easy to debug.

* PtyPosixIntegrationTest does a very simple integration test by spawning the programs `true' and `false'. Currently, these tests hang on the platforms where pty.spawn() is broken and testing cannot continue. As stated above, with the patch, everything is fine also on FreeBSD and OS X.

* PtyMockingExecTestBase is a base class where I prepare to replace the actual exec-call made by pty.spawn() to only run local python code. This makes it possible to test pty.spawn() in a portable, platform-independent manner (where platform-independent means posix-compatible).

* PtyWhiteBoxIntegrationPingPong1000Test exchanges a thousand Ping Pong Messages between master and the pty.spawn()ed slave.

* PtyWhiteBoxIntegrationReadSlaveTest tests that we read all data from the slave. This test is inspired by msg283912 and it shows that my naive patch in the beginning of issue29054 would lose data.

* PtyWhiteBoxIntegrationTermiosTest tests "advanced" features of terminals. The pty.spawn() function is the ultimate integration test setup for this.


Waiting for a review :-)

----------
Added file: http://bugs.python.org/file46115/pty_tests.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29070>
_______________________________________

From report at bugs.python.org  Mon Jan  2 13:39:21 2017
From: report at bugs.python.org (Cornelius Diekmann)
Date: Mon, 02 Jan 2017 18:39:21 +0000
Subject: [issue26228] pty.spawn hangs on FreeBSD 9.3, 10.x
In-Reply-To: <1453953861.58.0.104331734212.issue26228@psf.upfronthosting.co.za>
Message-ID: <1483382361.54.0.59351643802.issue26228@psf.upfronthosting.co.za>


Cornelius Diekmann added the comment:

I just tested pty.spawn() on OS X 10.6.8 and FreeBSD 11 and it also hangs. It works on Linux.

Your patch solves the problem. I proposed a test suite for pty.spawn() in issue29070. The test suite currently only exposes the problem, as suggested by Martin.

----------
nosy: +Cornelius Diekmann

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue26228>
_______________________________________

From report at bugs.python.org  Mon Jan  2 13:43:10 2017
From: report at bugs.python.org (Cornelius Diekmann)
Date: Mon, 02 Jan 2017 18:43:10 +0000
Subject: [issue29054] pty.py: pty.spawn hangs after client disconnect over nc
 (netcat)
In-Reply-To: <1482502637.76.0.304881442895.issue29054@psf.upfronthosting.co.za>
Message-ID: <1483382590.48.0.651871157362.issue29054@psf.upfronthosting.co.za>


Cornelius Diekmann added the comment:

Status change: I proposed a generic test suite for pty.spawn() in issue29070. Once we have agreed on the current behavior of pty.spawn() and the test suite is merged, I would like to come back to this issue which requests for a change in behavior of pty.spawn(). Currently, I would like to document that this issue is waiting for issue29070 and this issue doesn't need any attention.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29054>
_______________________________________

From report at bugs.python.org  Mon Jan  2 14:11:12 2017
From: report at bugs.python.org (Marco Buttu)
Date: Mon, 02 Jan 2017 19:11:12 +0000
Subject: [issue29134] contextlib doc bug
In-Reply-To: <1483376051.72.0.678487921628.issue29134@psf.upfronthosting.co.za>
Message-ID: <1483384272.85.0.758416089269.issue29134@psf.upfronthosting.co.za>


Marco Buttu added the comment:

Going a bit OT, in the contextlib doc there are two examples that have both normal code and shell code in the same code block. In this way we loose the prompt syntax highlight, and the code block can also confuse the reader.  In the patch attached the two examples are executed entirely in the shell, and both are under doctest.

----------
keywords: +patch
nosy: +marco.buttu
Added file: http://bugs.python.org/file46116/contextlib.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29134>
_______________________________________

From report at bugs.python.org  Mon Jan  2 14:50:59 2017
From: report at bugs.python.org (YoSTEALTH)
Date: Mon, 02 Jan 2017 19:50:59 +0000
Subject: [issue29134] contextlib doc bug
In-Reply-To: <1483376051.72.0.678487921628.issue29134@psf.upfronthosting.co.za>
Message-ID: <1483386659.55.0.712493080866.issue29134@psf.upfronthosting.co.za>


YoSTEALTH added the comment:

@SilentGhost
You are right, I see it now!

If this is the case maybe "ContextBaseClass" should be changed to "UserDefinedContextClass" (or something...) having "Base" in the class name was confusing, since module "io" has IOBase class that refer to its "base class" figured it was the same concept for contextlib as well.

Also it would be nice if example was improved a bit.


Another Idea:
 - why not have an empty context manager base class? like:

def ContextBase:

    def __enter__(self):
        return self

    def __exit__(self, *errors):
        pass

This is something anyone can use to extend their class to support context manager? I actually have a class like this i have used many times in my projects! Its simple yet useful.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29134>
_______________________________________

From report at bugs.python.org  Mon Jan  2 14:52:38 2017
From: report at bugs.python.org (YoSTEALTH)
Date: Mon, 02 Jan 2017 19:52:38 +0000
Subject: [issue29134] contextlib doc bug
In-Reply-To: <1483376051.72.0.678487921628.issue29134@psf.upfronthosting.co.za>
Message-ID: <1483386758.15.0.478788992191.issue29134@psf.upfronthosting.co.za>


YoSTEALTH added the comment:

typo:
def ContextBase:
to
class ContextBase:

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29134>
_______________________________________

From report at bugs.python.org  Mon Jan  2 15:37:59 2017
From: report at bugs.python.org (William Gianopoulos)
Date: Mon, 02 Jan 2017 20:37:59 +0000
Subject: [issue29135] run_proces logs the command without escaping parmaeters
 such that the coammns logged are not valid
Message-ID: <1483389479.69.0.4543924586.issue29135@psf.upfronthosting.co.za>


New submission from William Gianopoulos:

So, the arguments to run_process are not escaped when logged such that the logged command is un-parsable.
The following call:

self.run_process(['notify-send', '--app-name', 'Mozilla Build System', 'Mozilla Build System', msg])  where msg='Build complete'

ends up logging the following:

/usr/bin/notify-send --app-name Mozilla Build System Mozilla Build System Build complete

Where to be a valid command it needs to be:

/usr/bin/notify-send --app-name 'Mozilla Build System' 'Mozilla Build System' 'Build complete'


So, I think this needs to either not log the command at all or for each parameter replace any occurrence of the character "'" with "\'" and then enclose the entire parameter with "'" to make sure the logged command can actually be properly parsed.

----------
components: Library (Lib)
messages: 284498
nosy: wgianopoulos
priority: normal
severity: normal
status: open
title: run_proces logs the command without escaping parmaeters such that the coammns logged are not valid
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29135>
_______________________________________

From report at bugs.python.org  Mon Jan  2 15:42:40 2017
From: report at bugs.python.org (Ezio Melotti)
Date: Mon, 02 Jan 2017 20:42:40 +0000
Subject: [issue27200] make doctest in CPython has failures
In-Reply-To: <1464988408.0.0.840317371896.issue27200@psf.upfronthosting.co.za>
Message-ID: <1483389760.97.0.576349669843.issue27200@psf.upfronthosting.co.za>


Ezio Melotti added the comment:

I left a few comments on rietveld,
In general, if making the doctest pass entails adding superfluous code (e.g. sorted(), <BLANKLINES>, #doctest: +SKIP (if visible), setups/teardowns, or even whole files) or removing details that might be useful (ids or filenames in reprs), I'd rather skip the snippet altogether.

Keep in mind that most of those snippets are there to document, not for testing, so imho the priority should be keeping them clear and to the point.
The behavior of those snippets should already be covered by unittest, so the only downside of skipping is that, if the behavior changes, the examples will be wrong until someone notices.

If others agree, this should make the patch simpler and easier to review; if not, breaking it down in 3-4 patches that address separate issue might also help reviewers.

----------
nosy: +ezio.melotti

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue27200>
_______________________________________

From report at bugs.python.org  Mon Jan  2 15:42:43 2017
From: report at bugs.python.org (bpoaugust)
Date: Mon, 02 Jan 2017 20:42:43 +0000
Subject: [issue29053] Implement >From_ decoding on input from mbox
In-Reply-To: <1482502608.45.0.127449678115.issue29053@psf.upfronthosting.co.za>
Message-ID: <1483389763.01.0.760520328736.issue29053@psf.upfronthosting.co.za>


bpoaugust added the comment:

The patch can be simplified by just looking for b'\n' in the last 6 chars, and caching from b'\n' if found.

This would mean more file seeking in exchange for less buffer matching.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29053>
_______________________________________

From report at bugs.python.org  Mon Jan  2 15:52:12 2017
From: report at bugs.python.org (bpoaugust)
Date: Mon, 02 Jan 2017 20:52:12 +0000
Subject: [issue29020] collapse_rfc2231_value has inconsistent unquoting
In-Reply-To: <1482193812.46.0.729778317823.issue29020@psf.upfronthosting.co.za>
Message-ID: <1483390332.96.0.434524440252.issue29020@psf.upfronthosting.co.za>


bpoaugust added the comment:

If there are concerns about 3rd party code relying on the current behaviour of the function, then just create a new function without the unquoting, and deprecate the original function.

The existing function does not always unquote, so any code which relies on it is liable to break.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29020>
_______________________________________

From report at bugs.python.org  Mon Jan  2 15:57:20 2017
From: report at bugs.python.org (bpoaugust)
Date: Mon, 02 Jan 2017 20:57:20 +0000
Subject: [issue28945] get_boundary (and get_filename) invokes unquote twice
In-Reply-To: <1481547297.88.0.91240101349.issue28945@psf.upfronthosting.co.za>
Message-ID: <1483390640.02.0.498712338792.issue28945@psf.upfronthosting.co.za>


bpoaugust added the comment:

The patch is incomplete.

There is another unquote call:

336 return unquote(text)

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28945>
_______________________________________

From report at bugs.python.org  Mon Jan  2 16:02:29 2017
From: report at bugs.python.org (R. David Murray)
Date: Mon, 02 Jan 2017 21:02:29 +0000
Subject: [issue29135] run_proces logs the command without escaping parmaeters
 such that the coammns logged are not valid
In-Reply-To: <1483389479.69.0.4543924586.issue29135@psf.upfronthosting.co.za>
Message-ID: <1483390949.26.0.62389123764.issue29135@psf.upfronthosting.co.za>


R. David Murray added the comment:

What is run_process?  I'm not getting any hits from grep on the standard library.

----------
nosy: +r.david.murray

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29135>
_______________________________________

From report at bugs.python.org  Mon Jan  2 16:07:24 2017
From: report at bugs.python.org (Christian Heimes)
Date: Mon, 02 Jan 2017 21:07:24 +0000
Subject: [issue29136] Add OP_NO_TLSv1_3
Message-ID: <1483391244.76.0.29431388133.issue29136@psf.upfronthosting.co.za>


New submission from Christian Heimes:

OpenSSL 1.1.1 is going to provide TLS 1.3. The preferred protocols PROTOCOL_TLS (old name PROTOCOL_SSLv23), PROTOCOL_TLS_CLIENT and PROTOCOL_TLS_SERVER are going to have TLS 1.3 enabled by default. In order to disable TLS 1.3, let's add OP_NO_TLSv1_3 to _ssl.c and guard it with #ifdef SSL_OP_NO_TLSv1_3

https://github.com/openssl/openssl/blob/d2e491f225d465b11f18a466bf399d4a899cb50e/include/openssl/ssl.h#L346

Benjamin, Larry, Ned, are you ok with a new flag? OpenSSL 1.1.1 won't be available any time soon. I like to add the flag *after* the upcoming round of releases.

----------
assignee: christian.heimes
components: SSL
messages: 284504
nosy: benjamin.peterson, christian.heimes, larry, ned.deily
priority: normal
severity: normal
stage: needs patch
status: open
title: Add OP_NO_TLSv1_3
type: enhancement
versions: Python 2.7, Python 3.5, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29136>
_______________________________________

From report at bugs.python.org  Mon Jan  2 16:07:58 2017
From: report at bugs.python.org (William Gianopoulos)
Date: Mon, 02 Jan 2017 21:07:58 +0000
Subject: [issue29135] run_proces logs the command without escaping parmaeters
 such that the coammns logged are not valid
In-Reply-To: <1483390949.26.0.62389123764.issue29135@psf.upfronthosting.co.za>
Message-ID: <CAAyEg21CvXFDE70VugocfqsiiEvcMHEgk1-QcANhYjpFnaQyLw@mail.gmail.com>


William Gianopoulos added the comment:

well i could be screwed up perhaps it is something provided in the mozilla
python environment i was going by the fact that google searches on python
run-process returned things, including other reported issues, that made me
think it was not.

On Mon, Jan 2, 2017 at 4:02 PM, R. David Murray <report at bugs.python.org>
wrote:

>
> R. David Murray added the comment:
>
> What is run_process?  I'm not getting any hits from grep on the standard
> library.
>
> ----------
> nosy: +r.david.murray
>
> _______________________________________
> Python tracker <report at bugs.python.org>
> <http://bugs.python.org/issue29135>
> _______________________________________
>

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29135>
_______________________________________

From report at bugs.python.org  Mon Jan  2 16:11:29 2017
From: report at bugs.python.org (William Gianopoulos)
Date: Mon, 02 Jan 2017 21:11:29 +0000
Subject: [issue29135] run_proces logs the command without escaping parmaeters
 such that the coammns logged are not valid
In-Reply-To: <CAAyEg21CvXFDE70VugocfqsiiEvcMHEgk1-QcANhYjpFnaQyLw@mail.gmail.com>
Message-ID: <CAAyEg22fK-e9uis3y58bX-uWdLpLyecuQKpxYTUzLsOLq5T3=w@mail.gmail.com>


William Gianopoulos added the comment:

OK i t appears it might be a third party python utility library.  I will
try to trak this down and report it there and close this issue once i sort
it out.

On Mon, Jan 2, 2017 at 4:07 PM, William Gianopoulos <report at bugs.python.org>
wrote:

>
> William Gianopoulos added the comment:
>
> well i could be screwed up perhaps it is something provided in the mozilla
> python environment i was going by the fact that google searches on python
> run-process returned things, including other reported issues, that made me
> think it was not.
>
> On Mon, Jan 2, 2017 at 4:02 PM, R. David Murray <report at bugs.python.org>
> wrote:
>
> >
> > R. David Murray added the comment:
> >
> > What is run_process?  I'm not getting any hits from grep on the standard
> > library.
> >
> > ----------
> > nosy: +r.david.murray
> >
> > _______________________________________
> > Python tracker <report at bugs.python.org>
> > <http://bugs.python.org/issue29135>
> > _______________________________________
> >
>
> ----------
>
> _______________________________________
> Python tracker <report at bugs.python.org>
> <http://bugs.python.org/issue29135>
> _______________________________________
>

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29135>
_______________________________________

From report at bugs.python.org  Mon Jan  2 16:16:20 2017
From: report at bugs.python.org (William Gianopoulos)
Date: Mon, 02 Jan 2017 21:16:20 +0000
Subject: [issue29135] run_proces logs the command without escaping parmaeters
 such that the coammns logged are not valid
In-Reply-To: <1483389479.69.0.4543924586.issue29135@psf.upfronthosting.co.za>
Message-ID: <1483391780.35.0.621374597396.issue29135@psf.upfronthosting.co.za>


William Gianopoulos added the comment:

I would like to keep this open until I figure this out so I can provide a pointer to where the real issue is being tracked. I should have time to do that tomorrow. SOunds like this is part of some third-party add-on python library that is normally provided with  linux builds.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29135>
_______________________________________

From report at bugs.python.org  Mon Jan  2 16:38:42 2017
From: report at bugs.python.org (William Gianopoulos)
Date: Mon, 02 Jan 2017 21:38:42 +0000
Subject: [issue29135] run_proces logs the command without escaping parmaeters
 such that the coammns logged are not valid
In-Reply-To: <1483389479.69.0.4543924586.issue29135@psf.upfronthosting.co.za>
Message-ID: <1483393122.47.0.916650379507.issue29135@psf.upfronthosting.co.za>


William Gianopoulos added the comment:

It seems it is part of the Mozilla build system.  I closed this issue.

----------
resolution:  -> not a bug
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29135>
_______________________________________

From report at bugs.python.org  Mon Jan  2 16:58:01 2017
From: report at bugs.python.org (Benjamin Peterson)
Date: Mon, 02 Jan 2017 21:58:01 +0000
Subject: [issue29136] Add OP_NO_TLSv1_3
In-Reply-To: <1483391244.76.0.29431388133.issue29136@psf.upfronthosting.co.za>
Message-ID: <1483394278.4046232.835312409.25C3A392@webmail.messagingengine.com>


Benjamin Peterson added the comment:

I think that's fine for 2.7.

On Mon, Jan 2, 2017, at 13:07, Christian Heimes wrote:
> 
> New submission from Christian Heimes:
> 
> OpenSSL 1.1.1 is going to provide TLS 1.3. The preferred protocols
> PROTOCOL_TLS (old name PROTOCOL_SSLv23), PROTOCOL_TLS_CLIENT and
> PROTOCOL_TLS_SERVER are going to have TLS 1.3 enabled by default. In
> order to disable TLS 1.3, let's add OP_NO_TLSv1_3 to _ssl.c and guard it
> with #ifdef SSL_OP_NO_TLSv1_3
> 
> https://github.com/openssl/openssl/blob/d2e491f225d465b11f18a466bf399d4a899cb50e/include/openssl/ssl.h#L346
> 
> Benjamin, Larry, Ned, are you ok with a new flag? OpenSSL 1.1.1 won't be
> available any time soon. I like to add the flag *after* the upcoming
> round of releases.
> 
> ----------
> assignee: christian.heimes
> components: SSL
> messages: 284504
> nosy: benjamin.peterson, christian.heimes, larry, ned.deily
> priority: normal
> severity: normal
> stage: needs patch
> status: open
> title: Add OP_NO_TLSv1_3
> type: enhancement
> versions: Python 2.7, Python 3.5, Python 3.6, Python 3.7
> 
> _______________________________________
> Python tracker <report at bugs.python.org>
> <http://bugs.python.org/issue29136>
> _______________________________________

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29136>
_______________________________________

From report at bugs.python.org  Mon Jan  2 17:33:14 2017
From: report at bugs.python.org (Jim Fasarakis-Hilliard)
Date: Mon, 02 Jan 2017 22:33:14 +0000
Subject: [issue29012] __bases__ is a tuple (possibly empty or a singleton)
In-Reply-To: <1482134407.67.0.814127407079.issue29012@psf.upfronthosting.co.za>
Message-ID: <1483396394.41.0.188047797478.issue29012@psf.upfronthosting.co.za>


Jim Fasarakis-Hilliard added the comment:

Yes, I agree with that it seems redundant after the change. I'm attaching another small patch based on the committed one to trim that off.

----------
Added file: http://bugs.python.org/file46117/fixbasesdoc2.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29012>
_______________________________________

From report at bugs.python.org  Mon Jan  2 19:15:10 2017
From: report at bugs.python.org (Nathaniel Smith)
Date: Tue, 03 Jan 2017 00:15:10 +0000
Subject: [issue29137] Fix fpectl-induced ABI breakage
Message-ID: <1483402510.43.0.646850547969.issue29137@psf.upfronthosting.co.za>


New submission from Nathaniel Smith:

It turns out that CPython built with --with-fpectl has a different ABI than CPython built without --with-fpectl (which is the default). Specifically, if you have an extension module built against a --with-fpectl CPython, and it uses the PyFPE_{START,END}_PROTECT macros (as e.g. Cython modules do), then it ends up referring to some symbols that aren't provided by no-fpectl CPython builds.

These macros are part of the stable ABI, so it's possible (though unlikely?) that there are existing modules using the stable ABI that refer to these symbols.

Mailing list discussion:
   December: https://mail.python.org/pipermail/python-dev/2016-December/147065.html
   January: https://mail.python.org/pipermail/python-dev/2017-January/147092.html
   Guido's "let's get rid of it": https://mail.python.org/pipermail/python-dev/2017-January/147094.html

There are 3 parts to the fpectl machinery:
- macros PyFPE_{START,END}_PROTECT in Include/pyfpe.h, which are part of the public C API. Depending on --with-fpectl, these are either no-ops, or else refer to:
- global symbols PyFPE_{jbuf,counter,dummy}, defined in Python/pyfpe.c iff --with-fpectl is enabled.
- the stdlib module 'fpectl' (Modules/fpectlmodule.c) which provides some Python APIs that make use of the information that the macros put into the global symbols. (If the user doesn't use fpectl, then the macros do nothing except update the global variables.)

>From the python-dev discussion, I think the resolution is:

- for all supported CPython releases, we should modify Python/pyfpe.c so that the PyFPE_jbuf and PyFPE_counter are defined unconditionally. (I.e., remove the #ifdef WANT_SIGFPE_HANDLER that currently protects their definitions). After this change, all CPython builds will be able to import all CPython extension modules (though the actual fpectl functionality may or may not be available).

- in the next 3.5 and maybe 3.4 releases, we should add a deprecation warning to the fpectl module.

- in the next 2.7 release, we should add a Py3k warning to the fpectl module.

- in trunk / 3.7, we should remove --with-fpectl and the fpectl stdlib module entirely. For stable API compatibility we might want to leave the PyFPE_* macros (defined as no-ops) and/or the PyFPE_{jbuf,counter,dummy} symbols, but nothing will actually use them.

----------
components: Extension Modules
messages: 284511
nosy: njs
priority: normal
severity: normal
status: open
title: Fix fpectl-induced ABI breakage
versions: Python 2.7, Python 3.5, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29137>
_______________________________________

From report at bugs.python.org  Mon Jan  2 19:33:51 2017
From: report at bugs.python.org (Roundup Robot)
Date: Tue, 03 Jan 2017 00:33:51 +0000
Subject: [issue29012] __bases__ is a tuple (possibly empty or a singleton)
In-Reply-To: <1482134407.67.0.814127407079.issue29012@psf.upfronthosting.co.za>
Message-ID: <20170103003347.28640.60947.39BFE939@psf.io>


Roundup Robot added the comment:

New changeset 1aba7cbbcc27 by Berker Peksag in branch '3.5':
Issue #29012: Remove another outdated information
https://hg.python.org/cpython/rev/1aba7cbbcc27

New changeset 7ef5b02b228a by Berker Peksag in branch '3.6':
Issue #29012: Merge from 3.5
https://hg.python.org/cpython/rev/7ef5b02b228a

New changeset f05165a9cae6 by Berker Peksag in branch 'default':
Issue #29012: Merge from 3.6
https://hg.python.org/cpython/rev/f05165a9cae6

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29012>
_______________________________________

From report at bugs.python.org  Mon Jan  2 19:35:28 2017
From: report at bugs.python.org (Berker Peksag)
Date: Tue, 03 Jan 2017 00:35:28 +0000
Subject: [issue29012] __bases__ is a tuple (possibly empty or a singleton)
In-Reply-To: <1482134407.67.0.814127407079.issue29012@psf.upfronthosting.co.za>
Message-ID: <1483403728.9.0.571700035702.issue29012@psf.upfronthosting.co.za>


Berker Peksag added the comment:

Good point, Serhiy. Thanks for the patch, Jim. I've now removed the "possible a singleton" part.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29012>
_______________________________________

From report at bugs.python.org  Mon Jan  2 19:42:25 2017
From: report at bugs.python.org (Roundup Robot)
Date: Tue, 03 Jan 2017 00:42:25 +0000
Subject: [issue29035] regrtest: simplify regex to match test names for the
 --fromfile option
In-Reply-To: <1482330494.19.0.986324681201.issue29035@psf.upfronthosting.co.za>
Message-ID: <20170103004222.123644.65406.82EE3570@psf.io>


Roundup Robot added the comment:

New changeset a9fe5bee892b by Victor Stinner in branch 'default':
Issue #29035: Simplify a regex in libregrtest
https://hg.python.org/cpython/rev/a9fe5bee892b

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29035>
_______________________________________

From report at bugs.python.org  Mon Jan  2 19:44:11 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 03 Jan 2017 00:44:11 +0000
Subject: [issue29035] regrtest: simplify regex to match test names for the
 --fromfile option
In-Reply-To: <1482330494.19.0.986324681201.issue29035@psf.upfronthosting.co.za>
Message-ID: <1483404251.57.0.301505469433.issue29035@psf.upfronthosting.co.za>


STINNER Victor added the comment:

> I don't understand the use case of it ;)

When I modify a codec or something related to text codecs, I would like to run all codec tests, not the fully Python test suite. So I use "ls Lib/test/test_*codec*py", the list is quite list, it's annoying to have to copy/paste test names manually.

> You can call just .group() instead of .group(0).

Done.

Thanks for your review. I added a new unit test for filenames.

----------
resolution:  -> fixed
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29035>
_______________________________________

From report at bugs.python.org  Mon Jan  2 19:46:26 2017
From: report at bugs.python.org (Roundup Robot)
Date: Tue, 03 Jan 2017 00:46:26 +0000
Subject: [issue15812] inspect.getframeinfo() cannot show first line
In-Reply-To: <1346252079.7.0.669819400342.issue15812@psf.upfronthosting.co.za>
Message-ID: <20170103004624.21212.86520.638CBEDA@psf.io>


Roundup Robot added the comment:

New changeset 2b6bdd6cd3f8 by Berker Peksag in branch '3.5':
Issue #15812: Delete redundant max(start, 0)
https://hg.python.org/cpython/rev/2b6bdd6cd3f8

New changeset 7cbcee0c53e3 by Berker Peksag in branch '3.6':
Issue #15812: Merge from 3.5
https://hg.python.org/cpython/rev/7cbcee0c53e3

New changeset 5b0dee884b0b by Berker Peksag in branch 'default':
Issue #15812: Merge from 3.6
https://hg.python.org/cpython/rev/5b0dee884b0b

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue15812>
_______________________________________

From report at bugs.python.org  Mon Jan  2 19:47:13 2017
From: report at bugs.python.org (Berker Peksag)
Date: Tue, 03 Jan 2017 00:47:13 +0000
Subject: [issue15812] inspect.getframeinfo() cannot show first line
In-Reply-To: <1346252079.7.0.669819400342.issue15812@psf.upfronthosting.co.za>
Message-ID: <1483404433.61.0.377802229942.issue15812@psf.upfronthosting.co.za>


Berker Peksag added the comment:

You're right. Thanks for the review, Serhiy!

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue15812>
_______________________________________

From report at bugs.python.org  Mon Jan  2 19:48:39 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 03 Jan 2017 00:48:39 +0000
Subject: [issue28839] _PyFunction_FastCallDict(): replace PyTuple_New() with
 PyMem_Malloc()
In-Reply-To: <1480505567.26.0.13388381207.issue28839@psf.upfronthosting.co.za>
Message-ID: <1483404519.6.0.42442409681.issue28839@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Quick update on the fastcall work.

> I pushed th echange b9c9691c72c5 to replace PyObject_CallFunctionObjArgs() with _PyObject_CallNoArg() or _PyObject_CallArg1(). These functions are simpler and don't allocate memory on the C stack.

Using _PyObject_CallArg1() increased the usage of the C stack: see issue #28858. The fix was to remove the _PyObject_CallArg1() macro, replaced with PyObject_CallFunctionObjArgs(func, arg, NULL).

There is still an open issue #28870 to reduce the usage of the C stack memory even more, but it's more a general optimization.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28839>
_______________________________________

From report at bugs.python.org  Mon Jan  2 19:57:11 2017
From: report at bugs.python.org (Berker Peksag)
Date: Tue, 03 Jan 2017 00:57:11 +0000
Subject: [issue29013] zipfile: inconsistent doc for ZIP64 file size
In-Reply-To: <1482138273.34.0.375914911911.issue29013@psf.upfronthosting.co.za>
Message-ID: <1483405031.1.0.575413683793.issue29013@psf.upfronthosting.co.za>


Berker Peksag added the comment:

Right, that's my fault. Apparently I missed the "[...] zipfile will create [...]" part :) What do you think about Monte's suggested changes Serhiy? Should we just revert 4685cd33087b (1) or do both (1) and (2)?

----------
stage:  -> needs patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29013>
_______________________________________

From report at bugs.python.org  Mon Jan  2 20:00:12 2017
From: report at bugs.python.org (R. David Murray)
Date: Tue, 03 Jan 2017 01:00:12 +0000
Subject: [issue29135] run_proces logs the command without escaping parmaeters
 such that the coammns logged are not valid
In-Reply-To: <1483389479.69.0.4543924586.issue29135@psf.upfronthosting.co.za>
Message-ID: <1483405212.27.0.535034311197.issue29135@psf.upfronthosting.co.za>


Changes by R. David Murray <rdmurray at bitdance.com>:


----------
resolution: not a bug -> third party
stage:  -> resolved

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29135>
_______________________________________

From report at bugs.python.org  Mon Jan  2 20:00:13 2017
From: report at bugs.python.org (Berker Peksag)
Date: Tue, 03 Jan 2017 01:00:13 +0000
Subject: [issue27200] make doctest in CPython has failures
In-Reply-To: <1464988408.0.0.840317371896.issue27200@psf.upfronthosting.co.za>
Message-ID: <1483405213.8.0.522991512642.issue27200@psf.upfronthosting.co.za>


Berker Peksag added the comment:

I agree with Ezio.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue27200>
_______________________________________

From report at bugs.python.org  Mon Jan  2 20:05:49 2017
From: report at bugs.python.org (Roundup Robot)
Date: Tue, 03 Jan 2017 01:05:49 +0000
Subject: [issue28839] _PyFunction_FastCallDict(): replace PyTuple_New() with
 PyMem_Malloc()
In-Reply-To: <1480505567.26.0.13388381207.issue28839@psf.upfronthosting.co.za>
Message-ID: <20170103010546.25630.45293.780F6BB4@psf.io>


Roundup Robot added the comment:

New changeset 5f7cd3b6c9b1 by Victor Stinner in branch 'default':
Issue #28839: Optimize function_call()
https://hg.python.org/cpython/rev/5f7cd3b6c9b1

New changeset f9dd607dc04c by Victor Stinner in branch 'default':
Optimize _PyFunction_FastCallDict() when kwargs is {}
https://hg.python.org/cpython/rev/f9dd607dc04c

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28839>
_______________________________________

From report at bugs.python.org  Mon Jan  2 20:10:17 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 03 Jan 2017 01:10:17 +0000
Subject: [issue28839] _PyFunction_FastCallDict(): replace PyTuple_New() with
 PyMem_Malloc()
In-Reply-To: <1480505567.26.0.13388381207.issue28839@psf.upfronthosting.co.za>
Message-ID: <1483405817.67.0.821087561215.issue28839@psf.upfronthosting.co.za>


STINNER Victor added the comment:

I pushed the two obvious and safe optimization of fastcalldict-3.patch.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28839>
_______________________________________

From report at bugs.python.org  Mon Jan  2 20:22:40 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 03 Jan 2017 01:22:40 +0000
Subject: [issue28839] _PyFunction_FastCallDict(): replace PyTuple_New() with
 PyMem_Malloc()
In-Reply-To: <1480505567.26.0.13388381207.issue28839@psf.upfronthosting.co.za>
Message-ID: <1483406560.54.0.759476392206.issue28839@psf.upfronthosting.co.za>


STINNER Victor added the comment:

fastcalldict-4.patch: Rebased patch.

kw1: Median +- std dev: [ref] 290 ns +- 3 ns -> [patch] 253 ns +- 21 ns: 1.14x faster (-13%)
kw5: Median +- std dev: [ref] 438 ns +- 33 ns -> [patch] 394 ns +- 27 ns: 1.11x faster (-10%)
kw10: Median +- std dev: [ref] 663 ns +- 14 ns -> [patch] 617 ns +- 16 ns: 1.07x faster (-7%)

This patch still always allocated 10 pointers (80 bytes) on the C stack: see issue #28870 which discuss options to use less memory.

----------
Added file: http://bugs.python.org/file46118/fastcalldict-4.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28839>
_______________________________________

From report at bugs.python.org  Mon Jan  2 20:26:38 2017
From: report at bugs.python.org (Nick Coghlan)
Date: Tue, 03 Jan 2017 01:26:38 +0000
Subject: [issue29137] Fix fpectl-induced ABI breakage
In-Reply-To: <1483402510.43.0.646850547969.issue29137@psf.upfronthosting.co.za>
Message-ID: <1483406798.78.0.878586764651.issue29137@psf.upfronthosting.co.za>


Changes by Nick Coghlan <ncoghlan at gmail.com>:


----------
nosy: +ncoghlan

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29137>
_______________________________________

From report at bugs.python.org  Mon Jan  2 20:31:35 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 03 Jan 2017 01:31:35 +0000
Subject: [issue28870] Reduce stack consumption of
 PyObject_CallFunctionObjArgs() and like
In-Reply-To: <1480891813.94.0.54795440575.issue28870@psf.upfronthosting.co.za>
Message-ID: <1483407095.93.0.149737076879.issue28870@psf.upfronthosting.co.za>


STINNER Victor added the comment:

testcapi_stacksize.patch: add _testcapi.pyobjectl_callfunctionobjargs_stacksize(), function used to measure the stack consumption.

----------
Added file: http://bugs.python.org/file46119/testcapi_stacksize.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28870>
_______________________________________

From report at bugs.python.org  Mon Jan  2 20:32:55 2017
From: report at bugs.python.org (Eryk Sun)
Date: Tue, 03 Jan 2017 01:32:55 +0000
Subject: [issue29131] Calling printf from the cdll does not print the full
 string
In-Reply-To: <1483335941.28.0.90423143294.issue29131@psf.upfronthosting.co.za>
Message-ID: <1483407175.83.0.0172926383635.issue29131@psf.upfronthosting.co.za>


Eryk Sun added the comment:

> from ctypes import *
> msvcrt = cdll.msvcrt
> message_string = "Hello World!\n"
> msvcrt.printf("Testing: %s", message_string)

Avoid using libraries as attributes of cdll and windll. They get cached, and in turn they cache function pointers. Thus all modules contend for cdll and windll function prototype definitions (i.e. argtypes, restype, errcheck), and the last one to modify the definition wins. Even if you're not setting prototypes (in which case you must really like segfaults, data corruption, and difficult-to-diagnose errors), your code will still be at the mercy of whichever module does set them. Instead use the following:

    msvcrt = ctypes.CDLL('msvcrt', use_errno=True)
    msvcrt.printf.argtypes = (ctypes.c_char_p,)

CDLL uses the cdecl calling convention. This convention uses callee stack cleanup, so we allow passing a variable number of arguments even though the above argtypes definition only checks the first argument.

Examples

You'll get an exception if the first argument isn't bytes:

    >>> msvcrt.printf("spam and %d %s\n", 42, "eggs")
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    ctypes.ArgumentError: argument 1: <class 'TypeError'>: wrong type

Since we're not checking additional arguments, you might accidentally pass a wide-character string. In that case of course the output will be wrong:

    >>> msvcrt.printf(b"spam and %d %s\n", 42, "eggs")
    spam and 42 e
    14

Here we get it right:

    >>> msvcrt.printf(b"spam and %d %s\n", 42, b"eggs")
    spam and 42 eggs
    17

Or maybe you need to print a wide-character string, i.e. "%ls" or "%S":

    >>> msvcrt.printf(b"spam and %d %ls\n", 42, "eggs")
    spam and 42 eggs
    17

or

    >>> msvcrt.printf(b"spam and %d %S\n", 42, "eggs")
    spam and 42 eggs
    17

The "%s" and "%S" convention in MSVC predates standard C. In Microsoft's printf, "%s" expects a char pointer and "%S" expects a wchar_t pointer, but in their wprintf it's the other way around. In standard C "%s" and "%ls" are always a char string and a wchar_t string, respectively. MSVC also supports the standard "%ls" for wide-character strings, but its non-standard use of "%s" requires the introduction of a non-standard "%hs" for char strings.

----------
nosy: +eryksun

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29131>
_______________________________________

From report at bugs.python.org  Mon Jan  2 20:35:26 2017
From: report at bugs.python.org (Eryk Sun)
Date: Tue, 03 Jan 2017 01:35:26 +0000
Subject: [issue29131] Calling printf from the cdll does not print the full
 string
In-Reply-To: <1483335941.28.0.90423143294.issue29131@psf.upfronthosting.co.za>
Message-ID: <1483407326.97.0.142120651027.issue29131@psf.upfronthosting.co.za>


Changes by Eryk Sun <eryksun at gmail.com>:


----------
Removed message: http://bugs.python.org/msg284525

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29131>
_______________________________________

From report at bugs.python.org  Mon Jan  2 20:35:50 2017
From: report at bugs.python.org (Eryk Sun)
Date: Tue, 03 Jan 2017 01:35:50 +0000
Subject: [issue29131] Calling printf from the cdll does not print the full
 string
In-Reply-To: <1483335941.28.0.90423143294.issue29131@psf.upfronthosting.co.za>
Message-ID: <1483407350.69.0.149322078347.issue29131@psf.upfronthosting.co.za>


Eryk Sun added the comment:

> from ctypes import *
> msvcrt = cdll.msvcrt
> message_string = "Hello World!\n"
> msvcrt.printf("Testing: %s", message_string)

Avoid using libraries as attributes of cdll and windll. They get cached, and in turn they cache function pointers. Thus all modules contend for cdll and windll function prototype definitions (i.e. argtypes, restype, errcheck), and the last one to modify the definition wins. Even if you're not setting prototypes (in which case you must really like segfaults, data corruption, and difficult-to-diagnose errors), your code will still be at the mercy of whichever module does set them. Instead use the following:

    msvcrt = ctypes.CDLL('msvcrt', use_errno=True)
    msvcrt.printf.argtypes = (ctypes.c_char_p,)

CDLL uses the cdecl calling convention. This convention uses caller stack cleanup, so we allow passing a variable number of arguments even though the above argtypes definition only checks the first argument.

Examples

You'll get an exception if the first argument isn't bytes:

    >>> msvcrt.printf("spam and %d %s\n", 42, "eggs")
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    ctypes.ArgumentError: argument 1: <class 'TypeError'>: wrong type

Since we're not checking additional arguments, you might accidentally pass a wide-character string. In that case of course the output will be wrong:

    >>> msvcrt.printf(b"spam and %d %s\n", 42, "eggs")
    spam and 42 e
    14

Here we get it right:

    >>> msvcrt.printf(b"spam and %d %s\n", 42, b"eggs")
    spam and 42 eggs
    17

Or maybe you need to print a wide-character string, i.e. "%ls" or "%S":

    >>> msvcrt.printf(b"spam and %d %ls\n", 42, "eggs")
    spam and 42 eggs
    17

or

    >>> msvcrt.printf(b"spam and %d %S\n", 42, "eggs")
    spam and 42 eggs
    17

The "%s" and "%S" convention in MSVC predates standard C. In Microsoft's printf, "%s" expects a char pointer and "%S" expects a wchar_t pointer, but in their wprintf it's the other way around. In standard C "%s" and "%ls" are always a char string and a wchar_t string, respectively. MSVC also supports the standard "%ls" for wide-character strings, but its non-standard use of "%s" requires the introduction of a non-standard "%hs" for char strings.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29131>
_______________________________________

From report at bugs.python.org  Mon Jan  2 20:40:46 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 03 Jan 2017 01:40:46 +0000
Subject: [issue28870] Reduce stack consumption of
 PyObject_CallFunctionObjArgs() and like
In-Reply-To: <1480891813.94.0.54795440575.issue28870@psf.upfronthosting.co.za>
Message-ID: <1483407646.26.0.0527735337015.issue28870@psf.upfronthosting.co.za>


STINNER Victor added the comment:

no_small_stack.patch: And now something completely different, a patch to remove the "small stack" alllocated on the C stack, always use the heap memory. FYI I created no_small_stack.patch from less_stack.patch.

As expected, the stack usage is lower:

* less_stack.patch: 384 bytes/call
* no_small_stack.patch: 368 bytes/call

I didn't check the performance of no_small_stack.patch yet.

----------
Added file: http://bugs.python.org/file46120/no_small_stack.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28870>
_______________________________________

From report at bugs.python.org  Mon Jan  2 20:42:39 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 03 Jan 2017 01:42:39 +0000
Subject: [issue28870] Reduce stack consumption of
 PyObject_CallFunctionObjArgs() and like
In-Reply-To: <1480891813.94.0.54795440575.issue28870@psf.upfronthosting.co.za>
Message-ID: <1483407759.23.0.0779048683052.issue28870@psf.upfronthosting.co.za>


STINNER Victor added the comment:

In Python 3.5, PyObject_CallFunctionObjArgs() calls objargs_mktuple() which uses Py_VA_COPY(countva, va) and creates a tuple. The tuple constructor uses a free list to reduce the cost of heap memory allocations.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28870>
_______________________________________

From report at bugs.python.org  Mon Jan  2 20:47:58 2017
From: report at bugs.python.org (Nick Coghlan)
Date: Tue, 03 Jan 2017 01:47:58 +0000
Subject: [issue29137] Fix fpectl-induced ABI breakage
In-Reply-To: <1483402510.43.0.646850547969.issue29137@psf.upfronthosting.co.za>
Message-ID: <1483408078.24.0.445340769668.issue29137@psf.upfronthosting.co.za>


Nick Coghlan added the comment:

- 3.4 is already in security-fix only mode so we can safely ignore it for this purpose

- 3.5.3 is likely to be the last general bugfix release of 3.5, so we can probably skip that as well

- that would mean the ABI compability shims would only go in 3.6.1 and the next 2.7 release

At that point, does it actually make sense to provide the shims? Or should we instead just add the deprecation warnings and say "Don't use the --with-fpectl option, as it doesn't work properly, and breaks ABI compatibility for extension modules built with that Python"?

And then add a build time "#pragma message '--with-fpectl' is deprecated as it breaks extension module ABI compatibility" to the WANT_SIGFPE_HANDLER branch in Include/fpectl.h

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29137>
_______________________________________

From report at bugs.python.org  Mon Jan  2 20:58:51 2017
From: report at bugs.python.org (mike peremsky)
Date: Tue, 03 Jan 2017 01:58:51 +0000
Subject: [issue29131] Calling printf from the cdll does not print the full
 string
In-Reply-To: <1483335941.28.0.90423143294.issue29131@psf.upfronthosting.co.za>
Message-ID: <1483408731.04.0.637153951802.issue29131@psf.upfronthosting.co.za>


mike peremsky added the comment:

I appreciate the feedback, but as I had originally mentioned. I am following the "Gray Hat Python - Python Programming for Hackers and Reverse Engineers" book and was attempting to use Python 3 instead of Python 2 (as was used in the book). The cdll code is taken from the book itself. There is nothing that I personally plan on doing with this cdll other than to follow along in the book and become educated.

"Though if you do really want to use the (very) old msvcrt DLL rather than the proper functionality"

If there are better libraries to use I would be interested in knowing what those are. What is the "proper functionality"?. I like to educate myself and would like to attempt to follow the book using Python 3 and current libraries.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29131>
_______________________________________

From report at bugs.python.org  Mon Jan  2 21:18:08 2017
From: report at bugs.python.org (Evan)
Date: Tue, 03 Jan 2017 02:18:08 +0000
Subject: [issue29133] Minor inaccuracy in shlex.shlex punctuation_chars example
In-Reply-To: <1483362012.78.0.704413629859.issue29133@psf.upfronthosting.co.za>
Message-ID: <1483409888.18.0.264487649291.issue29133@psf.upfronthosting.co.za>


Evan added the comment:

Sorry for not being more clear in the original report - the error is in the code, not in the output. The old behavior is punct=False, the new is punct=True.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29133>
_______________________________________

From report at bugs.python.org  Mon Jan  2 21:26:36 2017
From: report at bugs.python.org (Nathaniel Smith)
Date: Tue, 03 Jan 2017 02:26:36 +0000
Subject: [issue29137] Fix fpectl-induced ABI breakage
In-Reply-To: <1483402510.43.0.646850547969.issue29137@psf.upfronthosting.co.za>
Message-ID: <1483410396.43.0.606242074722.issue29137@psf.upfronthosting.co.za>


Nathaniel Smith added the comment:

> At that point, does it actually make sense to provide the shims? Or should we instead just add the deprecation warnings and say "Don't use the --with-fpectl option, as it doesn't work properly, and breaks ABI compatibility for extension modules built with that Python"?

"Providing the shims" consists of deleting two lines of code, so eh, why not? And in theory there could be "stable ABI" extensions that depend on the shims. But I agree that it doesn't make much difference either way.

> And then add a build time "#pragma message '--with-fpectl' is deprecated as it breaks extension module ABI compatibility" to the WANT_SIGFPE_HANDLER branch in Include/fpectl.h

This would hassle every end user who builds extension modules on Debian/Ubuntu (b/c their default Python build uses --with-fpectl). But end users can't do anything about how Debian/Ubuntu build CPython. And in fact Debian/Ubuntu can't do anything about how Debian/Ubuntu build CPython either until 3.7 comes out and breaks ABI, because switching now would break installed systems...

So long as we keep the PyFPE_* macros as no-ops (which technically we have to to preserve the stable ABI), then there's no need to break compatibility at the C API level. The place we want to break compatibility is by dropping the Python-level fpectl package, so I think that's where we should warn.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29137>
_______________________________________

From report at bugs.python.org  Mon Jan  2 21:40:48 2017
From: report at bugs.python.org (Eryk Sun)
Date: Tue, 03 Jan 2017 02:40:48 +0000
Subject: [issue29131] Calling printf from the cdll does not print the full
 string
In-Reply-To: <1483335941.28.0.90423143294.issue29131@psf.upfronthosting.co.za>
Message-ID: <1483411248.33.0.858922595433.issue29131@psf.upfronthosting.co.za>


Eryk Sun added the comment:

> I am following the "Gray Hat Python - Python Programming for Hackers and Reverse Engineers"

>From what I've seen in Stack Overflow questions asked by people reading that book, its ctypes code is generally of poor quality. It only works in 32-bit Python 2 because the author has lazy habits about strings and pointers. Anyway, this issue is closed. There's no bug in Python here. If you have additional questions related to porting old ctypes code to work in Python 3 and 64-bit Windows, please ask on ctypes-users [1] or python-list [2]. Many people will be happy to help you.

[1]: https://lists.sourceforge.net/lists/listinfo/ctypes-users
[2]: https://mail.python.org/mailman/listinfo/python-list

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29131>
_______________________________________

From report at bugs.python.org  Mon Jan  2 22:44:47 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Tue, 03 Jan 2017 03:44:47 +0000
Subject: [issue29130] Exit code 120 returned from Python unit test testing
 SystemExit
In-Reply-To: <1483316898.11.0.0292789374137.issue29130@psf.upfronthosting.co.za>
Message-ID: <1483415087.31.0.722298011181.issue29130@psf.upfronthosting.co.za>


Xiang Zhang added the comment:

Behaviour changed in #5319. Add author Martin.

----------
nosy: +martin.panter, xiang.zhang

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29130>
_______________________________________

From report at bugs.python.org  Mon Jan  2 22:47:47 2017
From: report at bugs.python.org (Martin Panter)
Date: Tue, 03 Jan 2017 03:47:47 +0000
Subject: [issue28864] Add devnull file-like object
In-Reply-To: <1480822595.87.0.853337900572.issue28864@psf.upfronthosting.co.za>
Message-ID: <1483415267.72.0.480014750139.issue28864@psf.upfronthosting.co.za>


Martin Panter added the comment:

Example where an implementation like Serhiy?s was not good enough: <https://bugs.python.org/issue29130#msg284437>. In that case, the lack of flush() method causes a subtle problem.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28864>
_______________________________________

From report at bugs.python.org  Mon Jan  2 23:00:01 2017
From: report at bugs.python.org (Martin Panter)
Date: Tue, 03 Jan 2017 04:00:01 +0000
Subject: [issue29130] Exit code 120 returned from Python unit test testing
 SystemExit
In-Reply-To: <1483316898.11.0.0292789374137.issue29130@psf.upfronthosting.co.za>
Message-ID: <1483416001.0.0.332188132166.issue29130@psf.upfronthosting.co.za>


Martin Panter added the comment:

David is right. The 120 code was added in Issue 5319, as a way of indicating a problem in the final stages of the interpreter exiting. The two conditions that trigger this are calling the flush() method on sys.stdout and sys.stderr. If you add a dummy flush() implementation, it no longer exits with 120:

>>> subprocess.call((sys.executable, "-c", """
... class NullWriter:
...     def write(self, s): pass
... import sys; sys.stderr = NullWriter()"""))
120
>>> subprocess.call((sys.executable, "-c", """
... class NullWriter:
...     def write(self, s): pass
...     def flush(self): pass
... import sys; sys.stderr = NullWriter()"""))
0

It does not seem to be explicitly documented what you can set sys.stderr to, but I always thought it is safest to use an io.TextIOBase implementation. I would suggest to derive your NullWriter class from TextIOBase; that way you get a default flush() implementation for free.

Other options are to use StringIO() if you are not expecting too much output, or open(os.devnull, "w"). See also Issue 28864 about adding a predefined class like NullWriter to Python.

Having said all that, perhaps it would be reasonable to tolerate a missing flush() method, and not treat this as an error.

Stepping back a bit, I also suggest restoring sys.stderr after the test. Otherwise, you risk missing an error message. Try contextlib.redirect_stderr().

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29130>
_______________________________________

From report at bugs.python.org  Mon Jan  2 23:15:32 2017
From: report at bugs.python.org (Nick Coghlan)
Date: Tue, 03 Jan 2017 04:15:32 +0000
Subject: [issue28180] sys.getfilesystemencoding() should default to utf-8
In-Reply-To: <1474024622.65.0.838685461995.issue28180@psf.upfronthosting.co.za>
Message-ID: <1483416932.79.0.544058082005.issue28180@psf.upfronthosting.co.za>


Nick Coghlan added the comment:

Updated patch adds some tests showing that this change should also help with cases where SSH environment forwarding results in an unknown locale being requested in the server environment.

----------
Added file: http://bugs.python.org/file46121/pep538_coerce_legacy_c_locale_v2.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28180>
_______________________________________

From report at bugs.python.org  Mon Jan  2 23:42:41 2017
From: report at bugs.python.org (Ma Lin)
Date: Tue, 03 Jan 2017 04:42:41 +0000
Subject: [issue28518] execute("begin immediate") throwing OperationalError
In-Reply-To: <1477307118.31.0.70785963389.issue28518@psf.upfronthosting.co.za>
Message-ID: <1483418561.04.0.738502923863.issue28518@psf.upfronthosting.co.za>


Ma Lin added the comment:

I'm trying to catch up with you.
Does it mean, if I want execute VACUUM, I have to commit manually? Like this:

if conn.in_transaction:
    conn.commit()
conn.execute("vacuum")

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28518>
_______________________________________

From report at bugs.python.org  Tue Jan  3 01:27:46 2017
From: report at bugs.python.org (Martin Panter)
Date: Tue, 03 Jan 2017 06:27:46 +0000
Subject: [issue29138] No __hash__() inheritance warning with -Werror
Message-ID: <1483424866.48.0.00115230231317.issue29138@psf.upfronthosting.co.za>


New submission from Martin Panter:

Normally there is a Python 3 compatibility warning emitted when a class is based on object, implements __eq__(), but does not define __hash__():

$ python -3 -c 'class C(object): __eq__ = lambda self, other: True'
-c:1: DeprecationWarning: Overriding __eq__ blocks inheritance of __hash__ in 3.x

But when warnings are raised as exceptions, this warning seems to be suppressed:

$ python -3 -Werror -c 'class C(object): __eq__ = lambda self, other: True'

Perhaps there is bad exception handling at the warn() call site.

----------
messages: 284539
nosy: martin.panter
priority: normal
severity: normal
stage: needs patch
status: open
title: No __hash__() inheritance warning with -Werror
type: behavior
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29138>
_______________________________________

From report at bugs.python.org  Tue Jan  3 01:29:29 2017
From: report at bugs.python.org (Nick Coghlan)
Date: Tue, 03 Jan 2017 06:29:29 +0000
Subject: [issue29137] Fix fpectl-induced ABI breakage
In-Reply-To: <1483402510.43.0.646850547969.issue29137@psf.upfronthosting.co.za>
Message-ID: <1483424969.48.0.99866139163.issue29137@psf.upfronthosting.co.za>


Nick Coghlan added the comment:

Ah, cool - I didn't know that Debian built with fpectl enabled by default. In that case, +1 for:

- low maintenance ABI and API compatibility shims that are kept around indefinitely (since they leaked into the limited API/stable ABI) but don't actually do anything
- deprecating and removing the fpectl module
- adding the ABI compatibility shims in the maintenance releases where the retroactive deprecation warnings are introduced

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29137>
_______________________________________

From report at bugs.python.org  Tue Jan  3 01:33:50 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Tue, 03 Jan 2017 06:33:50 +0000
Subject: [issue29138] No __hash__() inheritance warning with -Werror
In-Reply-To: <1483424866.48.0.00115230231317.issue29138@psf.upfronthosting.co.za>
Message-ID: <1483425230.84.0.582334361486.issue29138@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

This is a duplicate of issue8627.

----------
nosy: +serhiy.storchaka
resolution:  -> duplicate
superseder:  -> Unchecked PyErr_WarnPy3k return value in Objects/typeobject.c

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29138>
_______________________________________

From report at bugs.python.org  Tue Jan  3 01:41:48 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Tue, 03 Jan 2017 06:41:48 +0000
Subject: [issue29130] Exit code 120 returned from Python unit test testing
 SystemExit
In-Reply-To: <1483316898.11.0.0292789374137.issue29130@psf.upfronthosting.co.za>
Message-ID: <1483425708.4.0.577648174924.issue29130@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

> Having said all that, perhaps it would be reasonable to tolerate a missing flush() method, and not treat this as an error.

In all cases the flush() method is called unconditionally. I think it can be considered as mandatory part of the writer protocol.

----------
nosy: +serhiy.storchaka

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29130>
_______________________________________

From report at bugs.python.org  Tue Jan  3 02:55:50 2017
From: report at bugs.python.org (Martin Panter)
Date: Tue, 03 Jan 2017 07:55:50 +0000
Subject: [issue29138] No __hash__() inheritance warning with -Werror
In-Reply-To: <1483424866.48.0.00115230231317.issue29138@psf.upfronthosting.co.za>
Message-ID: <1483430150.8.0.0780963369658.issue29138@psf.upfronthosting.co.za>


Martin Panter added the comment:

Thanks, I will try to look at that some time

----------
stage: needs patch -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29138>
_______________________________________

From report at bugs.python.org  Tue Jan  3 03:34:58 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Tue, 03 Jan 2017 08:34:58 +0000
Subject: [issue29035] regrtest: simplify regex to match test names for the
 --fromfile option
In-Reply-To: <1482330494.19.0.986324681201.issue29035@psf.upfronthosting.co.za>
Message-ID: <1483432498.58.0.0802198354054.issue29035@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

> So I use "ls Lib/test/test_*codec*py", the list is quite list, it's annoying to have to copy/paste test names manually.

I would use

    ./python -m test --list | grep codec | xargs ./python -m test

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29035>
_______________________________________

From report at bugs.python.org  Tue Jan  3 04:15:09 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Tue, 03 Jan 2017 09:15:09 +0000
Subject: [issue29116] Make str and bytes error messages on concatenation
 conform with other sequences
In-Reply-To: <1483124211.75.0.715869443236.issue29116@psf.upfronthosting.co.za>
Message-ID: <1483434909.67.0.795957330842.issue29116@psf.upfronthosting.co.za>


Xiang Zhang added the comment:

> As for `operator.concat`, any reason why the check is made beforehand to see if the first argument has a `__getitem__` method?

Concatenation is an operation of sequence (+ could be used both for concatenation and addition), so you have to check __getitem__ to make sure it's a sequence.

> Couldn't that just be removed allowing the exception from `concat(1, '')` to just propagate to the caller?

If removed, `concat(1, '')` could propagate exceptions but not `concat(1, 1)`.

----------
nosy: +xiang.zhang

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29116>
_______________________________________

From report at bugs.python.org  Tue Jan  3 04:58:30 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Tue, 03 Jan 2017 09:58:30 +0000
Subject: [issue29139] operator.concat/iconcat could only work if left operand
 is a sequence
Message-ID: <1483437510.5.0.880271510583.issue29139@psf.upfronthosting.co.za>


New submission from Xiang Zhang:

As the title, but I think it should also continue if only the right operand is a sequence since the right operand could get a __radd__ to do the concatenation:

>>> class S:
...     def __init__(self):
...             self.v = list(range(10))
...     def __radd__(self, other):
...             print('in __radd__')
...             self.v.extend(other)
...             return self.v
...     def __getitem__(self, idx):
...             return self.v[idx]
>>> def i():
...     yield from range(10, 20)
>>> i() + S()
in __radd__
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19]
>>> operator.concat(i(), S())
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'generator' object can't be concatenated
>>> a = i()
>>> a += S()
in __radd__
>>> a
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19]
>>> a = i()
>>> operator.iconcat(a, S())
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'generator' object can't be concatenated

----------
components: Library (Lib)
messages: 284546
nosy: xiang.zhang
priority: normal
severity: normal
status: open
title: operator.concat/iconcat could only work if left operand is a sequence
type: behavior
versions: Python 3.5, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29139>
_______________________________________

From report at bugs.python.org  Tue Jan  3 05:11:29 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 03 Jan 2017 10:11:29 +0000
Subject: [issue29139] operator.concat/iconcat could only work if left operand
 is a sequence
In-Reply-To: <1483437510.5.0.880271510583.issue29139@psf.upfronthosting.co.za>
Message-ID: <1483438289.02.0.234730927692.issue29139@psf.upfronthosting.co.za>


Changes by STINNER Victor <victor.stinner at gmail.com>:


----------
nosy: +haypo

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29139>
_______________________________________

From report at bugs.python.org  Tue Jan  3 05:16:10 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Tue, 03 Jan 2017 10:16:10 +0000
Subject: [issue29139] operator.concat/iconcat could only work if left operand
 is a sequence
In-Reply-To: <1483437510.5.0.880271510583.issue29139@psf.upfronthosting.co.za>
Message-ID: <1483438570.29.0.0923208540818.issue29139@psf.upfronthosting.co.za>


Changes by Xiang Zhang <angwerzx at 126.com>:


----------
nosy: +serhiy.storchaka, zach.ware

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29139>
_______________________________________

From report at bugs.python.org  Tue Jan  3 05:17:17 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Tue, 03 Jan 2017 10:17:17 +0000
Subject: [issue29139] operator.concat/iconcat could only work if left operand
 is a sequence
In-Reply-To: <1483437510.5.0.880271510583.issue29139@psf.upfronthosting.co.za>
Message-ID: <1483438637.38.0.346005577753.issue29139@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

I don't think it should. You can use operator.add() if you need to fall back to a __radd__ of the right operand.

operator.concat() is Python API to the sq_concat slot. It falls back to __add__() just because instances of user classes defining an __add__() method only have an nb_add slot, not an sq_concat slot.

Third-party classes (maybe NumPy arrays, I don't know) can have different implementations of sq_concat and nb_add.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29139>
_______________________________________

From report at bugs.python.org  Tue Jan  3 05:40:55 2017
From: report at bugs.python.org (SilentGhost)
Date: Tue, 03 Jan 2017 10:40:55 +0000
Subject: [issue29134] contextlib doc bug
In-Reply-To: <1483376051.72.0.678487921628.issue29134@psf.upfronthosting.co.za>
Message-ID: <1483440055.98.0.0345879147341.issue29134@psf.upfronthosting.co.za>


Changes by SilentGhost <ghost.adh at runbox.com>:


----------
assignee:  -> docs at python
components: +Documentation
nosy: +docs at python
stage:  -> patch review
type:  -> behavior
versions: +Python 3.5, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29134>
_______________________________________

From report at bugs.python.org  Tue Jan  3 06:01:44 2017
From: report at bugs.python.org (Marco Buttu)
Date: Tue, 03 Jan 2017 11:01:44 +0000
Subject: [issue27200] make doctest in CPython has failures
In-Reply-To: <1464988408.0.0.840317371896.issue27200@psf.upfronthosting.co.za>
Message-ID: <1483441304.45.0.553293395848.issue27200@psf.upfronthosting.co.za>


Marco Buttu added the comment:

Hi Ezio, thanks for your time :) The +SKIP option and the testsetup and teardown directives are not visible in the output. The only extra code in the output is sorted(). I agree with you about the use of sorted(), and I think we do not have to add extra code just to make the example passes. But as you can see in issue28860, I was asked to add it. Before going on, I think we should have a clear policy about it.

I also think the examples are there do document, but I believe there are good reasons to test them. In fact, unittests and doctests have complete diffent purposes: the goal of doctests is to verify the examples work properly, that's all. An example could be wrong also if all the related unittests pass and it is still updated: there could be a typo in the code or in the output, a missing import, and other kind of errors or lacks that make the example not working, getting confused the reader. Unluckily there is not just the downside you pointed out :/

Ok to break the patch in more patches, but I did not get if you want me to open a new issue for each patch, or attach all the patches here in this issue. Thanks again

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue27200>
_______________________________________

From report at bugs.python.org  Tue Jan  3 06:07:49 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Tue, 03 Jan 2017 11:07:49 +0000
Subject: [issue29139] operator.concat/iconcat could only work if left operand
 is a sequence
In-Reply-To: <1483437510.5.0.880271510583.issue29139@psf.upfronthosting.co.za>
Message-ID: <1483441668.99.0.123196699002.issue29139@psf.upfronthosting.co.za>


Xiang Zhang added the comment:

> You can use operator.add() if you need to fall back to a __radd__ of the right operand.

IMHO for sequences when defining __add__, __radd__, there is no difference between addition and concatenation, just as https://docs.python.org/3/reference/datamodel.html#emulating-container-types states. So it's confusing for me why must I use operator.add not operator.concate?

> operator.concat() is Python API to the sq_concat slot. It falls back to __add__() just because instances of user classes defining an __add__() method only have an nb_add slot, not an sq_concat slot.

More like a implementation detail for me. Users writing only Python won't realize such things.

> Third-party classes (maybe NumPy arrays, I don't know) can have different implementations of sq_concat and nb_add.

Does this matter? Anyway operator.concat will fall to '+'. My intent is proposing something like:

    if not hasattr(a, '__getitem__') and not hasattr(b, '__getitem__')

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29139>
_______________________________________

From report at bugs.python.org  Tue Jan  3 06:24:44 2017
From: report at bugs.python.org (Michael Felt)
Date: Tue, 03 Jan 2017 11:24:44 +0000
Subject: [issue28845] Clean up known issues for AIX
In-Reply-To: <1480542032.97.0.55863701863.issue28845@psf.upfronthosting.co.za>
Message-ID: <1483442684.24.0.164406889944.issue28845@psf.upfronthosting.co.za>


Michael Felt added the comment:

FWIW: just build python-2.7.13 using xlC - and ncurses-6.0 installed.

root at x064:[/data/prj/python/python2-2.7.13]
root at x064:[/data/prj/python/python2-2.7.13]./python
Python 2.7.13 (default, Jan  3 2017, 11:16:59) [C] on aix5
Type "help", "copyright", "credits" or "license" for more information.
>>> import curses
>>> from curses import panel
>>> def mkpanel(scr):
...  win = curses.newwin(8,8,1,1)
...  pan = panel.new_panel(win)
...
>>> curses.wrapper(mkpanel)
>>>

So, even for AIX 5.3 and Python2.7 this test seems solved. If the above is all that is needed!

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28845>
_______________________________________

From report at bugs.python.org  Tue Jan  3 07:17:30 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 03 Jan 2017 12:17:30 +0000
Subject: [issue29130] Exit code 120 returned from Python unit test testing
 SystemExit
In-Reply-To: <1483316898.11.0.0292789374137.issue29130@psf.upfronthosting.co.za>
Message-ID: <1483445850.03.0.877436888506.issue29130@psf.upfronthosting.co.za>


STINNER Victor added the comment:

This issue is not a bug in Python, but a bug in the author's code. I suggest to close the issue, since enough clues have been provided to fix the author's issue.

----------
nosy: +haypo

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29130>
_______________________________________

From report at bugs.python.org  Tue Jan  3 07:21:15 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 03 Jan 2017 12:21:15 +0000
Subject: [issue28945] email: get_boundary (and get_filename) invokes unquote
 twice
In-Reply-To: <1481547297.88.0.91240101349.issue28945@psf.upfronthosting.co.za>
Message-ID: <1483446075.93.0.245099719075.issue28945@psf.upfronthosting.co.za>


Changes by STINNER Victor <victor.stinner at gmail.com>:


----------
title: get_boundary (and get_filename) invokes unquote twice -> email: get_boundary (and get_filename) invokes unquote twice

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28945>
_______________________________________

From report at bugs.python.org  Tue Jan  3 07:21:51 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 03 Jan 2017 12:21:51 +0000
Subject: [issue29053] mailbox: Implement >From_ decoding on input from mbox
In-Reply-To: <1482502608.45.0.127449678115.issue29053@psf.upfronthosting.co.za>
Message-ID: <1483446111.66.0.0873815756544.issue29053@psf.upfronthosting.co.za>


Changes by STINNER Victor <victor.stinner at gmail.com>:


----------
title: Implement >From_ decoding on input from mbox -> mailbox: Implement >From_ decoding on input from mbox

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29053>
_______________________________________

From report at bugs.python.org  Tue Jan  3 07:22:37 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 03 Jan 2017 12:22:37 +0000
Subject: [issue29020] email: collapse_rfc2231_value has inconsistent unquoting
In-Reply-To: <1482193812.46.0.729778317823.issue29020@psf.upfronthosting.co.za>
Message-ID: <1483446157.79.0.80740257297.issue29020@psf.upfronthosting.co.za>


Changes by STINNER Victor <victor.stinner at gmail.com>:


----------
title: collapse_rfc2231_value has inconsistent unquoting -> email: collapse_rfc2231_value has inconsistent unquoting

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29020>
_______________________________________

From report at bugs.python.org  Tue Jan  3 07:24:01 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 03 Jan 2017 12:24:01 +0000
Subject: [issue29054] pty.py: pty.spawn hangs after client disconnect over nc
 (netcat)
In-Reply-To: <1482502637.76.0.304881442895.issue29054@psf.upfronthosting.co.za>
Message-ID: <1483446241.35.0.380321377948.issue29054@psf.upfronthosting.co.za>


Changes by STINNER Victor <victor.stinner at gmail.com>:


----------
nosy: +haypo

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29054>
_______________________________________

From report at bugs.python.org  Tue Jan  3 07:25:11 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 03 Jan 2017 12:25:11 +0000
Subject: [issue26228] pty.spawn hangs on FreeBSD 9.3, 10.x
In-Reply-To: <1453953861.58.0.104331734212.issue26228@psf.upfronthosting.co.za>
Message-ID: <1483446311.85.0.209835132306.issue26228@psf.upfronthosting.co.za>


Changes by STINNER Victor <victor.stinner at gmail.com>:


----------
nosy: +haypo

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue26228>
_______________________________________

From report at bugs.python.org  Tue Jan  3 07:26:59 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 03 Jan 2017 12:26:59 +0000
Subject: [issue26228] pty.spawn hangs on FreeBSD 9.3, 10.x
In-Reply-To: <1453953861.58.0.104331734212.issue26228@psf.upfronthosting.co.za>
Message-ID: <1483446419.47.0.978366247348.issue26228@psf.upfronthosting.co.za>


STINNER Victor added the comment:

> That used to be true in FBSD, but then someone fixed (?) it, and now the master side simply returns EOF when read-from.

Is it a behaviour change in a new version of Python? Or a change in FreeBSD? I don't understand.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue26228>
_______________________________________

From report at bugs.python.org  Tue Jan  3 07:27:11 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 03 Jan 2017 12:27:11 +0000
Subject: [issue29070] Integration tests for pty.spawn on Linux and all other
 platforms
In-Reply-To: <1482684225.17.0.467768789806.issue29070@psf.upfronthosting.co.za>
Message-ID: <1483446431.14.0.759790355834.issue29070@psf.upfronthosting.co.za>


Changes by STINNER Victor <victor.stinner at gmail.com>:


----------
nosy: +haypo

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29070>
_______________________________________

From report at bugs.python.org  Tue Jan  3 07:39:04 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 03 Jan 2017 12:39:04 +0000
Subject: [issue29063] Fixed timemodule compile warnings.
In-Reply-To: <1482576109.57.0.29725831282.issue29063@psf.upfronthosting.co.za>
Message-ID: <1483447144.29.0.920150372921.issue29063@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Downcasting to int doesn't seem good to me.

gmtoff_time_t.patch uses time_t instead of an int to store gmtoff. It raises an OverflowError if the value doesn't fit into a C long (at least max is at least 2^31).

I guess that the issue only impacts Windows, I expect that most platforms have a tm_zone field in the "tm" structure?

/* Define to 1 if `tm_zone' is a member of `struct tm'. */
#define HAVE_STRUCT_TM_TM_ZONE 1

----------
nosy: +haypo
Added file: http://bugs.python.org/file46122/gmtoff_time_t.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29063>
_______________________________________

From report at bugs.python.org  Tue Jan  3 07:41:02 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 03 Jan 2017 12:41:02 +0000
Subject: [issue29063] Fixed timemodule compile warnings.
In-Reply-To: <1482576109.57.0.29725831282.issue29063@psf.upfronthosting.co.za>
Message-ID: <1483447262.74.0.875161689087.issue29063@psf.upfronthosting.co.za>


STINNER Victor added the comment:

The warning was introduced by a recent change:

changeset:   103680:a96101dd105c
user:        Alexander Belopolsky <alexander.belopolsky at gmail.com>
date:        Sun Sep 11 22:55:16 2016 -0400
files:       Doc/library/time.rst Misc/NEWS Modules/timemodule.c
description:
Closes #25283: Make tm_gmtoff and tm_zone available on all platforms.

----------
nosy: +belopolsky
versions: +Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29063>
_______________________________________

From report at bugs.python.org  Tue Jan  3 07:47:46 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 03 Jan 2017 12:47:46 +0000
Subject: [issue29063] Fixed timemodule compile warnings.
In-Reply-To: <1482576109.57.0.29725831282.issue29063@psf.upfronthosting.co.za>
Message-ID: <1483447666.38.0.448819340406.issue29063@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Not checking for integer overflow is more likely to hide bugs. Handling time is an hard problem, see a recent funny (or not) story from Cloudfare with the latest leap second added at midnight the 2016-12-31:
https://blog.cloudflare.com/how-and-why-the-leap-second-affected-cloudflare-dns/

Extract: "RRDNS is written in Go and uses Go?s time.Now() function to get the time. Unfortunately, this function does not guarantee monotonicity. Go currently doesn?t offer a monotonic time source (see issue 12914 for discussion)."

Hopefully, Python has time.monotonic() since Python 3.3 ;-)

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29063>
_______________________________________

From report at bugs.python.org  Tue Jan  3 07:49:51 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 03 Jan 2017 12:49:51 +0000
Subject: [issue28911] Clarify the behaviour of assert_called_once_with
In-Reply-To: <1481233700.68.0.637550951007.issue28911@psf.upfronthosting.co.za>
Message-ID: <1483447791.37.0.151533745751.issue28911@psf.upfronthosting.co.za>


Changes by STINNER Victor <victor.stinner at gmail.com>:


----------
nosy: +haypo

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28911>
_______________________________________

From report at bugs.python.org  Tue Jan  3 07:52:04 2017
From: report at bugs.python.org (John Hagen)
Date: Tue, 03 Jan 2017 12:52:04 +0000
Subject: [issue29130] Exit code 120 returned from Python unit test testing
 SystemExit
In-Reply-To: <1483316898.11.0.0292789374137.issue29130@psf.upfronthosting.co.za>
Message-ID: <1483447924.29.0.677201590153.issue29130@psf.upfronthosting.co.za>


John Hagen added the comment:

I'm completely fine with closing this issue. Thanks for the help everyone.

If someone else doesn't close it in a couple days, I'll do it.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29130>
_______________________________________

From report at bugs.python.org  Tue Jan  3 07:52:38 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 03 Jan 2017 12:52:38 +0000
Subject: [issue29097] datetime.fromtimestamp(t) when 0 <= t <= 86399 fails on
 Python 3.6
In-Reply-To: <1482958160.11.0.32337159902.issue29097@psf.upfronthosting.co.za>
Message-ID: <1483447958.85.0.932074460195.issue29097@psf.upfronthosting.co.za>


Changes by STINNER Victor <victor.stinner at gmail.com>:


----------
nosy: +haypo

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29097>
_______________________________________

From report at bugs.python.org  Tue Jan  3 07:54:32 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 03 Jan 2017 12:54:32 +0000
Subject: [issue29028] Use-After-Free in PyString_FromStringAndSize() of
 stringobject.c
In-Reply-To: <1482247531.75.0.221402980867.issue29028@psf.upfronthosting.co.za>
Message-ID: <1483448072.59.0.279036895052.issue29028@psf.upfronthosting.co.za>


Changes by STINNER Victor <victor.stinner at gmail.com>:


----------
nosy: +haypo

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29028>
_______________________________________

From report at bugs.python.org  Tue Jan  3 07:54:28 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 03 Jan 2017 12:54:28 +0000
Subject: [issue27867] various issues due to misuse of PySlice_GetIndicesEx
In-Reply-To: <1472226685.15.0.242668202848.issue27867@psf.upfronthosting.co.za>
Message-ID: <1483448068.35.0.841656859094.issue27867@psf.upfronthosting.co.za>


Changes by STINNER Victor <victor.stinner at gmail.com>:


----------
nosy: +haypo

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue27867>
_______________________________________

From report at bugs.python.org  Tue Jan  3 08:07:43 2017
From: report at bugs.python.org (Michael Felt)
Date: Tue, 03 Jan 2017 13:07:43 +0000
Subject: [issue27435] ctypes library loading and AIX - also for 2.7.X (and
 later)
In-Reply-To: <1467383699.63.0.495142233463.issue27435@psf.upfronthosting.co.za>
Message-ID: <1483448863.5.0.268051914254.issue27435@psf.upfronthosting.co.za>


Michael Felt added the comment:

Again, I would like to draw attention to this issue - BECAUSE -

it is possible to use ctypes.CDLL() ASIS, on AIX - however, ctypes.util.find_library() always seems to fail (i.e., returns None even when a .so file exists)

When the library names are hard-coded, rather than found by find_library() everything seems to work fine.

In short, I hope this will finally be recognized as a bug (ctypes.util.find_library()).

Ideally, RTLD_MEMBER logic would be added to the Lib/ctypes/__init__.py so that CDLL does not have to have RTLD_MEMBER added manually as it does now.

(I have not yet figured out how to use LibraryLoader() properly, it seems to never complain, or behave the same as ctypes.CDLL(None) which returns the "python process" namespace - if I understand that correctly.

p.s. a) I have a newer patch; b) PEP 8, etc. pickyness is encouraged (i.e., expect multiple iterations before a patch is accepted - I do not mind learning PEP 8 et al.);

imho - nothing new is being added to what python2 (and python3!) already can do. However, a fix to make it easier and clearer to use (there must be a PEP for that case as well) - is warranted.

++++++++
Preparation:
a) the contents of the archive (example)
root at x064:[/data/prj/python/python-2.7.12.0]ar -Xany tv /usr/lib/libcurses.a
rwxr-xr-x   300/300   499495 Sep 26 22:35 2007 shr42.o
rwxr-xr-x   300/300   321459 Sep 26 22:04 2007 shr4.o
rwxr-xr-x   300/300   197552 Sep 26 22:04 2007 shr.o
rwxr-xr-x   300/300   591888 Sep 26 22:35 2007 shr42_64.o

b) create a empty .so file - MAYBE this will be found by the so-called "posix"
   routine in ctypes.util.find_library

root at x064:[/data/prj/python/python2-2.7.13]touch /usr/lib/libcurses.so

c) A test program
import sys
import ctypes
from ctypes import util

if sys.platform[:3] == "aix":
    print ctypes.util.find_library("curses")
    RTLD_MEMBER =  0x00040000
    if (sys.maxsize < 2**32):
        print 32
        print ctypes.CDLL("libcurses.a(shr.o)", RTLD_MEMBER)
    else:
        print 64
        print ctypes.CDLL("libcurses.a(shr42_64.o)", RTLD_MEMBER)

Testing:
a) notice "None" as value printed by find_library
b) notice that ctypes.CDLL can load the library when RTLD_MEMBER isd added
root at x064:[/data/prj/python/python2-2.7.13]./python ../python2-2.7.13/test*.py
None
64
<CDLL 'libcurses.a(shr42_64.o)', handle 10 at 7000000001280f0>

Changes:
-- since /usr/lib/libcurses.so is an empty file
   MAYBE - that is why the result was None
-- extract the 64-bit member and rename as /usr/lib/libcurses.so
-- test both find_library (still says None!) and CDLL of the extracted member
-- as it also loads, CDLL, i.e., dlopen(), works without RTLD_MEMBER
   (as it should)
-- there does not appear top be anything wrong with ctypes.CDLL,
   but only with the 'utils' included

root at x064:[/data/prj/python/python2-2.7.13]ar x /usr/lib/libcurses.a shr42_64.o
root at x064:[/data/prj/python/python2-2.7.13]mv shr42_64.o /usr/lib/libcurses.so

import sys
import ctypes
from ctypes import util

if sys.platform[:3] == "aix":
    print ctypes.util.find_library("curses")
    RTLD_MEMBER =  0x00040000
    if (sys.maxsize < 2**32):
        print 32
        print ctypes.CDLL("libcurses.a(shr.o)", RTLD_MEMBER)
    else:
        print 64
        print ctypes.CDLL("libcurses.a(shr42_64.o)", RTLD_MEMBER)
        print ctypes.CDLL("libcurses.so")

root at x064:[/data/prj/python/python2-2.7.13]./python ../python2-2.7.13/test*.py
None
64
<CDLL 'libcurses.a(shr42_64.o)', handle 10 at 700000000128128>
<CDLL 'libcurses.so', handle 11 at 700000000128128>

Verification of (expected) behavior:
a) CDLL in both formats returns the same function pointer for 'addch'
b) LibraryLoader always returns something, even when it is nonsense
c) CDLL - fails - aka Traceback - when an argument cannot be loaded

from ctypes import util

if sys.platform[:3] == "aix":
    print ctypes.util.find_library("curses")
    RTLD_MEMBER =  0x00040000
    if (sys.maxsize < 2**32):
        print 32
        print ctypes.CDLL("libcurses.a(shr.o)", RTLD_MEMBER)
    else:
        print 64
        print ctypes.CDLL("libcurses.a(shr42_64.o)", RTLD_MEMBER)
        lib = ctypes.CDLL("libcurses.a(shr42_64.o)", RTLD_MEMBER)
        if hasattr(lib, 'addch'):
            print "addch found"
            print lib.addch
        else:
            print "addch not found"
        print ctypes.CDLL(None)
        lib = ctypes.CDLL(None)
        if hasattr(lib, 'addch'):
            print "addch found"
            print lib.addch
        else:
            print "addch not found"
        print ctypes.CDLL("libcurses.so")
        lib = ctypes.CDLL("libcurses.so")
        if hasattr(lib, 'addch'):
            print "addch found"
            print lib.addch
        else:
            print "addch not found"
        print "LibraryLoader tests"
        print ctypes.LibraryLoader("libcurses.so")
        lib = ctypes.LibraryLoader("libcurses.so")
        print ctypes.LibraryLoader(None)
        print ctypes.LibraryLoader("libcurses.a(shr42_64.o)")
        print ctypes.LibraryLoader("libc.a(shr_64.o)")
        print "LibraryLoader XXX tests show LibraryLoader masks unloadable libraries"
        print ctypes.LibraryLoader("xxx.so")
        print ctypes.CDLL("xxx.so")

returns: (python2-2.7.12 and python2-2.7.13 return the same value)

root at x064:[/data/prj/python/python-2.7.12.0]./python ../python2-2.7.13/test*.py
None
64
<CDLL 'libcurses.a(shr42_64.o)', handle e at 700000000135a58>
addch found
<_FuncPtr object at 0x700000000174108>
<CDLL 'None', handle 10 at 7000000001354a8>
addch not found
<CDLL 'libcurses.so', handle 12 at 7000000001358d0>
addch found
<_FuncPtr object at 0x700000000174048>
LibraryLoader tests
<ctypes.LibraryLoader object at 0x7000000001354a8>
<ctypes.LibraryLoader object at 0x700000000135470>
<ctypes.LibraryLoader object at 0x700000000135470>
<ctypes.LibraryLoader object at 0x700000000135470>
LibraryLoader XXX tests show LibraryLoader masks unloadable libraries
<ctypes.LibraryLoader object at 0x700000000135470>
Traceback (most recent call last):
  File "../python2-2.7.13/test_ctypes.py", line 42, in <module>
    print ctypes.CDLL("xxx.so")
  File "/data/prj/python/python-2.7.12.0/Lib/ctypes/__init__.py", line 362, in __init__
    self._handle = _dlopen(self._name, mode)
OSError:        0509-022 Cannot load module .
        0509-026 System error: A file or directory in the path name does not exist.

root at x064:[/data/prj/python/python-2.7.12.0]cd ../py*13

root at x064:[/data/prj/python/python2-2.7.13]./python ../python2-2.7.13/test*.py
None
64
<CDLL 'libcurses.a(shr42_64.o)', handle 10 at 700000000127320>
addch found
<_FuncPtr object at 0x700000000158288>
<CDLL 'None', handle 12 at 700000000127710>
addch not found
<CDLL 'libcurses.so', handle 14 at 700000000127898>
addch found
<_FuncPtr object at 0x7000000001581c8>
LibraryLoader tests
<ctypes.LibraryLoader object at 0x700000000127908>
<ctypes.LibraryLoader object at 0x700000000127978>
<ctypes.LibraryLoader object at 0x700000000127978>
<ctypes.LibraryLoader object at 0x700000000127978>
LibraryLoader XXX tests show LibraryLoader masks unloadable libraries
<ctypes.LibraryLoader object at 0x700000000127978>
Traceback (most recent call last):
  File "../python2-2.7.13/test_ctypes.py", line 42, in <module>
    print ctypes.CDLL("xxx.so")
  File "/data/prj/python/src/python2-2.7.13/Lib/ctypes/__init__.py", line 362, in __init__
    self._handle = _dlopen(self._name, mode)
OSError:        0509-022 Cannot load module .
        0509-026 System error: A file or directory in the path name does not exist.
root at x064:[/data/prj/python/python2-2.7.13]

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue27435>
_______________________________________

From report at bugs.python.org  Tue Jan  3 08:12:47 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Tue, 03 Jan 2017 13:12:47 +0000
Subject: [issue29117] dir(<bound method>) should include dunder attributes of
 the unbound method
In-Reply-To: <1483134670.11.0.964598948516.issue29117@psf.upfronthosting.co.za>
Message-ID: <1483449167.0.0.716986659649.issue29117@psf.upfronthosting.co.za>


Changes by Xiang Zhang <angwerzx at 126.com>:


----------
nosy: +xiang.zhang

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29117>
_______________________________________

From report at bugs.python.org  Tue Jan  3 08:12:57 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 03 Jan 2017 13:12:57 +0000
Subject: [issue29104] Left bracket remains in format string result when '\'
 preceeds it
In-Reply-To: <1483012585.51.0.342717701316.issue29104@psf.upfronthosting.co.za>
Message-ID: <1483449177.74.0.278410578726.issue29104@psf.upfronthosting.co.za>


STINNER Victor added the comment:

The PEP 498 should be updated to define the expected behaviour of f'\{5}':
https://www.python.org/dev/peps/pep-0498/#escape-sequences

+        self.assertEqual(f'\{6*7}', '\\42')
+        self.assertEqual(f'\\{6*7}', '\\42')
+        self.assertEqual(fr'\{6*7}', '\\42')
+
+        AMPERSAND = 123
+        self.assertEqual(f'\N{AMPERSAND}', '&')
+        self.assertEqual(f'\\N{AMPERSAND}', '\\N123')
+        self.assertEqual(fr'\N{AMPERSAND}', '\\N123')
+        self.assertEqual(f'\\\N{AMPERSAND}', '\\&')

I'm not sure that I like this behaviour.

f'\\N{AMPERSAND}': reading a local variable looks like a typo or a security vulnerability, rather than a nice feature.

IMHO if you want an anti-slash (\) in the output string, it *must* be written "\\" since we *do* interpret other escape sequences:

* f'\n' returns '\n', the newline character U+000A
* f'\N{Snowman}' returns the nice snowman character U+2603 (?)
* etc.

What is the issue with having to write "\\{10}" to get "\\10" string? It's less error prone.

----------
nosy: +haypo

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29104>
_______________________________________

From report at bugs.python.org  Tue Jan  3 08:21:22 2017
From: report at bugs.python.org (Michael Felt)
Date: Tue, 03 Jan 2017 13:21:22 +0000
Subject: [issue28845] Clean up known issues for AIX
In-Reply-To: <1480542032.97.0.55863701863.issue28845@psf.upfronthosting.co.za>
Message-ID: <1483449682.05.0.737624194721.issue28845@psf.upfronthosting.co.za>


Michael Felt added the comment:

I request that you review issue27435 - in particular msg284557 - as I feel ctypes implementation for AIX is broken - at least as far as the supporting routines are concerned.

When you know the "magic" one can call ctypes.CDLL() and it 'works'. ctypes.util.find_library() always returns None (maybe it works if a stack of GNU tools are also loaded, but not on a 'generic' AIX install)

ctypes.LibraryLoader() is always successful (i.e., no Traceback) but I am (personally) unsure of the actual return value. It feels like a false positive.

Thank you for your time and consideration!

p.s. - a "fix/patch" is nearly accepted for Python3.7 (see issue26439) - needs more for the docs - however, I feel is is a great short-coming to limit this to versions that are yet to come!

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28845>
_______________________________________

From report at bugs.python.org  Tue Jan  3 08:22:46 2017
From: report at bugs.python.org (Ezio Melotti)
Date: Tue, 03 Jan 2017 13:22:46 +0000
Subject: [issue27200] make doctest in CPython has failures
In-Reply-To: <1464988408.0.0.840317371896.issue27200@psf.upfronthosting.co.za>
Message-ID: <1483449766.69.0.985068540923.issue27200@psf.upfronthosting.co.za>


Ezio Melotti added the comment:

> The +SKIP option and the testsetup and teardown directives are not visible in the output.

So feel free to use them when needed :)

> But as you can see in issue28860, I was asked to add it. Before going
> on, I think we should have a clear policy about it.

If the priority is making the test pass, then adding sorted() is a viable solution -- if keeping the examples simple and to the point has higher priority, then the test could be skipped.  The priority depends on the case.

> An example could be wrong also if all the related unittests pass and
> it is still updated: there could be a typo in the code or in the 
> output, a missing import, and other kind of errors or lacks that make > the example not working, getting confused the reader.

True.  In general, I think most of the current examples are fine, and whoever adds new ones or changes the behavior should take care that the examples (still) work.  If there is a mistake in an untested example, hopefully someone will report it.  While it's true that these mistakes might cause confusion, the same could be said for other changes done to the code to make the test pass (Why do I need to call sorted()/list()/etc.?  Why there are '...' in the output, is this an actual terminal session? etc.).     

> Ok to break the patch in more patches, but I did not get if you want
> me to open a new issue for each patch, or attach all the patches here 
> in this issue. Thanks again

It's probably fine to keep the discussion and patches on this issue, unless you prefer to handle discussions about specific parts separately -- just make sure that it's clear from the file description what patches should be applied.
As for the number of patches, probably 2-4 files are enough (e.g. one file could include all the trivial addition of +SKIP/.. doctest::, another the more "controversial" changes that add new files).

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue27200>
_______________________________________

From report at bugs.python.org  Tue Jan  3 08:57:09 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 03 Jan 2017 13:57:09 +0000
Subject: [issue29140] time_hash() reads the wrong bytes to get microseconds
Message-ID: <1483451829.62.0.434619516952.issue29140@psf.upfronthosting.co.za>


New submission from STINNER Victor:

When the time is folded, time_hash() uses DATE_xxx() macros, instead of TIME_xxx() macros, and so reads microseconds from the wrong bytes.

Bug introduced by the implementation of the PEP 495 (Local Time Disambiguation).

#define PyDateTime_DATE_GET_MICROSECOND(o)              \
    ((((PyDateTime_DateTime*)o)->data[7] << 16) |       \
     (((PyDateTime_DateTime*)o)->data[8] << 8)  |       \
      ((PyDateTime_DateTime*)o)->data[9])

#define PyDateTime_TIME_GET_MICROSECOND(o)              \
    ((((PyDateTime_Time*)o)->data[3] << 16) |           \
     (((PyDateTime_Time*)o)->data[4] << 8)  |           \
      ((PyDateTime_Time*)o)->data[5])

Attached patch fixes time_hash().

I guess that it's a dummy copy-paste issue.

----------
components: Library (Lib)
files: time_hash.patch
keywords: patch
messages: 284561
nosy: belopolsky, haypo
priority: normal
severity: normal
status: open
title: time_hash() reads the wrong bytes to get microseconds
versions: Python 3.6, Python 3.7
Added file: http://bugs.python.org/file46123/time_hash.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29140>
_______________________________________

From report at bugs.python.org  Tue Jan  3 09:04:28 2017
From: report at bugs.python.org (Sandeep Srinivasa)
Date: Tue, 03 Jan 2017 14:04:28 +0000
Subject: [issue29141] error in 2to3
Message-ID: <1483452268.26.0.428258207401.issue29141@psf.upfronthosting.co.za>


New submission from Sandeep Srinivasa:

docs_uploaded_at = dict(filter(lambda (x,y):True if len(y) == 3 else False,docs_uploaded_at.iteritems()))

produces

docs_uploaded_at = dict([x_y for x_y in iter(docs_uploaded_at.items()) if True if len(x_y[1]) == 3 else False])


without_transaction_users = filter(lambda x:False if x.phone in with_transaction_mobile else True,registered)

produces

without_transaction_users = [x for x in registered if False if x.phone in with_transaction_mobile else True]

----------
components: 2to3 (2.x to 3.x conversion tool)
messages: 284562
nosy: Sandeep Srinivasa
priority: normal
severity: normal
status: open
title: error in 2to3

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29141>
_______________________________________

From report at bugs.python.org  Tue Jan  3 09:07:25 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 03 Jan 2017 14:07:25 +0000
Subject: [issue29100] datetime.fromtimestamp() does crash for very small or
 very big timestamp
In-Reply-To: <1482972647.78.0.524140289263.issue29100@psf.upfronthosting.co.za>
Message-ID: <1483452445.1.0.995238596031.issue29100@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Attached datetime_check_args.patch fixes the issue: move argument checks into time(), date() and datetime() constructors of the _datetime module.

The patch requires the fix of the issue #29140 (time_hash bug), otherwise unit tests fail.

This issue is a regression introduced the implementation of the PEP 495. In Python 3.5, datetime_from_timet_and_us() calls datetime constructor which checks argument. In Python 3.6, datetime_from_timet_and_us() calls directly new_datetime_ex2() which doesn't check arguments anymore.

datetime_check_args.patch makes the _datetime module generally safer, not only for the report bug, because new_datetime_ex2() can be called by different ways, for example from the C API: "PyDateTime_CAPI".

The bug always shows a major lack of unit test in datetime.datetime.fromtimestamp() on dates out of the 0001-01-01 .. 9999-12-31 range. In Python 3.6, datetime.fromtimestamp() can create invalid datetime object greater than datetime.max!

$ python3.6 -c 'import datetime; print(datetime.datetime.fromtimestamp(2**38))'
10680-07-14 08:09:04

TODO: add unit tests.

----------
keywords: +patch
nosy: +haypo
title: Core dump / OverflowError for datetime.fromtimestamp with overly large timestamp in Ubuntu 12.04 -> datetime.fromtimestamp() does crash for very small or very big timestamp
versions: +Python 3.7
Added file: http://bugs.python.org/file46124/datetime_check_args.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29100>
_______________________________________

From report at bugs.python.org  Tue Jan  3 09:07:45 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 03 Jan 2017 14:07:45 +0000
Subject: [issue29100] datetime.fromtimestamp() doesn't check min/max year
 anymore: regression of Python 3.6
In-Reply-To: <1482972647.78.0.524140289263.issue29100@psf.upfronthosting.co.za>
Message-ID: <1483452465.07.0.281978774066.issue29100@psf.upfronthosting.co.za>


Changes by STINNER Victor <victor.stinner at gmail.com>:


----------
title: datetime.fromtimestamp() does crash for very small or very big timestamp -> datetime.fromtimestamp() doesn't check min/max year anymore: regression of Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29100>
_______________________________________

From report at bugs.python.org  Tue Jan  3 09:08:06 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 03 Jan 2017 14:08:06 +0000
Subject: [issue29100] datetime.fromtimestamp() doesn't check min/max year
 anymore: regression of Python 3.6
In-Reply-To: <1482972647.78.0.524140289263.issue29100@psf.upfronthosting.co.za>
Message-ID: <1483452486.09.0.364444967548.issue29100@psf.upfronthosting.co.za>


Changes by STINNER Victor <victor.stinner at gmail.com>:


----------
keywords: +3.6regression

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29100>
_______________________________________

From report at bugs.python.org  Tue Jan  3 09:08:16 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 03 Jan 2017 14:08:16 +0000
Subject: [issue29140] time_hash() reads the wrong bytes to get microseconds
In-Reply-To: <1483451829.62.0.434619516952.issue29140@psf.upfronthosting.co.za>
Message-ID: <1483452496.25.0.984577897929.issue29140@psf.upfronthosting.co.za>


Changes by STINNER Victor <victor.stinner at gmail.com>:


----------
keywords: +3.6regression

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29140>
_______________________________________

From report at bugs.python.org  Tue Jan  3 09:09:41 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 03 Jan 2017 14:09:41 +0000
Subject: [issue28961] unittest.mock._Call ignores `name` parameter
In-Reply-To: <1481636868.17.0.0411235027553.issue28961@psf.upfronthosting.co.za>
Message-ID: <1483452581.59.0.3070130549.issue28961@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Please remove "import ipdb; ipdb.set_trace() # TODO remove it" before posting patches ;-)

----------
nosy: +haypo

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28961>
_______________________________________

From report at bugs.python.org  Tue Jan  3 09:09:49 2017
From: report at bugs.python.org (Thomas Loetzer)
Date: Tue, 03 Jan 2017 14:09:49 +0000
Subject: [issue29142] urllib: no_proxy variable values with leading dot not
 properly handled
Message-ID: <1483452589.12.0.177615671636.issue29142@psf.upfronthosting.co.za>


New submission from Thomas Loetzer:

Hi,

the fix for issue 26864 changed the behavior of urllib for no_proxy values with a leading dot to no longer match anything. This seems to be caused by always adding an additional dot between the hostname being checked and the entry.

Example:
no_proxy = '.company.internal'

I would expect 'somehost.company.internal' to be accessed without proxy, but this is not actually the case. Changing no_proxy to company.internal changes this and the host is accessed without proxy.

Python 2.7.11 and curl both handle this case fine and the proxy is not used.

Regards,
Thomas

----------
components: Library (Lib)
messages: 284565
nosy: tloetzer
priority: normal
severity: normal
status: open
title: urllib: no_proxy variable values with leading dot not properly handled
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29142>
_______________________________________

From report at bugs.python.org  Tue Jan  3 09:10:55 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 03 Jan 2017 14:10:55 +0000
Subject: [issue28983] Windows: Python 3.5.2 won't install on my computer
In-Reply-To: <1481840742.78.0.554470867064.issue28983@psf.upfronthosting.co.za>
Message-ID: <1483452655.97.0.311595635256.issue28983@psf.upfronthosting.co.za>


Changes by STINNER Victor <victor.stinner at gmail.com>:


----------
title: Python 3.5.2 won't install on my computer -> Windows: Python 3.5.2 won't install on my computer

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28983>
_______________________________________

From report at bugs.python.org  Tue Jan  3 09:12:46 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 03 Jan 2017 14:12:46 +0000
Subject: [issue28768] Warning: implicit declaration of function '_setmode'
In-Reply-To: <1479758430.88.0.97682867264.issue28768@psf.upfronthosting.co.za>
Message-ID: <1483452766.79.0.821034670198.issue28768@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Masayuki Yamamoto: Do you consider that Python 2.7 should be fixed as well? What is your use case for compiling Python 2.7 on Windows using Cygwin?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28768>
_______________________________________

From report at bugs.python.org  Tue Jan  3 09:19:08 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 03 Jan 2017 14:19:08 +0000
Subject: [issue29096] unicode_concatenate() optimization is not signal-safe
 (not atomic)
In-Reply-To: <1482957515.7.0.441178310139.issue29096@psf.upfronthosting.co.za>
Message-ID: <1483453148.89.0.00877217958763.issue29096@psf.upfronthosting.co.za>


Changes by STINNER Victor <victor.stinner at gmail.com>:


----------
title: Signal Handlers reliably cause UnboundLocalErrors -> unicode_concatenate() optimization is not signal-safe (not atomic)
versions: +Python 3.5, Python 3.6, Python 3.7 -Python 3.3, Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29096>
_______________________________________

From report at bugs.python.org  Tue Jan  3 09:23:05 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 03 Jan 2017 14:23:05 +0000
Subject: [issue29090] python34.dll crash
In-Reply-To: <1482892800.26.0.562397151262.issue29090@psf.upfronthosting.co.za>
Message-ID: <1483453385.91.0.493748764287.issue29090@psf.upfronthosting.co.za>


STINNER Victor added the comment:

> OS is Windows XP SP3

Oh wow, that's old! XP is not more supported by Python, but it's no more supported by Microsoft neither.

A crash can have various reasons. Usually, it's a buffer overflow or a race condition "somewhere". You should try to collect more information using faulthandler, or maybe using Visual Studio.
https://docs.python.org/dev/library/faulthandler.html

I suggest you to try to upgrade Windows and Python to more recent versions.

Anyway, as Zachary wrote, Python 3.4 is no more supported (only security fixes, nor more bug fixes).

----------
nosy: +haypo

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29090>
_______________________________________

From report at bugs.python.org  Tue Jan  3 09:23:38 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 03 Jan 2017 14:23:38 +0000
Subject: [issue29090] PySerial crash on Windows XP and Python 3.4
In-Reply-To: <1482892800.26.0.562397151262.issue29090@psf.upfronthosting.co.za>
Message-ID: <1483453418.95.0.210407146713.issue29090@psf.upfronthosting.co.za>


Changes by STINNER Victor <victor.stinner at gmail.com>:


----------
resolution:  -> out of date
status: open -> closed
title: python34.dll crash -> PySerial crash on Windows XP and Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29090>
_______________________________________

From report at bugs.python.org  Tue Jan  3 09:23:51 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 03 Jan 2017 14:23:51 +0000
Subject: [issue28969] lru_cache is not threadsafe
In-Reply-To: <1481712564.52.0.530922236321.issue28969@psf.upfronthosting.co.za>
Message-ID: <1483453431.88.0.408348267278.issue28969@psf.upfronthosting.co.za>


Changes by STINNER Victor <victor.stinner at gmail.com>:


----------
nosy: +haypo

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28969>
_______________________________________

From report at bugs.python.org  Tue Jan  3 09:24:27 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 03 Jan 2017 14:24:27 +0000
Subject: [issue29092] Sync os.stat's doc and doc string
In-Reply-To: <1482914545.99.0.849195642081.issue29092@psf.upfronthosting.co.za>
Message-ID: <1483453467.66.0.23948125032.issue29092@psf.upfronthosting.co.za>


STINNER Victor added the comment:

It's a recent change, before path type was always str.

----------
nosy: +haypo, serhiy.storchaka

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29092>
_______________________________________

From report at bugs.python.org  Tue Jan  3 09:25:36 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 03 Jan 2017 14:25:36 +0000
Subject: [issue29092] Sync os.stat's doc and doc string
In-Reply-To: <1482914545.99.0.849195642081.issue29092@psf.upfronthosting.co.za>
Message-ID: <1483453536.73.0.924322329779.issue29092@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Ignore my comment, I was thinking to something else (recent os.scandir change on Windows).

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29092>
_______________________________________

From report at bugs.python.org  Tue Jan  3 09:27:40 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 03 Jan 2017 14:27:40 +0000
Subject: [issue25750] tp_descr_get(self, obj,
 type) is called without owning a reference to "self"
In-Reply-To: <1448654213.28.0.807225788685.issue25750@psf.upfronthosting.co.za>
Message-ID: <1483453660.08.0.822188083041.issue25750@psf.upfronthosting.co.za>


Changes by STINNER Victor <victor.stinner at gmail.com>:


----------
nosy: +haypo

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue25750>
_______________________________________

From report at bugs.python.org  Tue Jan  3 09:30:00 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 03 Jan 2017 14:30:00 +0000
Subject: [issue25750] tp_descr_get(self, obj,
 type) is called without owning a reference to "self"
In-Reply-To: <1448654213.28.0.807225788685.issue25750@psf.upfronthosting.co.za>
Message-ID: <1483453800.55.0.844304838649.issue25750@psf.upfronthosting.co.za>


STINNER Victor added the comment:

descr_ref-2.patch: patch rebased on the 2.7 branch.

----------
Added file: http://bugs.python.org/file46125/descr_ref-2.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue25750>
_______________________________________

From report at bugs.python.org  Tue Jan  3 09:31:06 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 03 Jan 2017 14:31:06 +0000
Subject: [issue25750] tp_descr_get(self, obj,
 type) is called without owning a reference to "self"
In-Reply-To: <1448654213.28.0.807225788685.issue25750@psf.upfronthosting.co.za>
Message-ID: <1483453866.06.0.0856403880616.issue25750@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Claudio: "We're currently testing to try and reproduce the segfaults on 2.7.13, after that I'll try jdemeyer's patch and report the results."

Cool! Keep us in touch ;-)

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue25750>
_______________________________________

From report at bugs.python.org  Tue Jan  3 09:34:54 2017
From: report at bugs.python.org (Jiajun Huang)
Date: Tue, 03 Jan 2017 14:34:54 +0000
Subject: [issue28961] unittest.mock._Call ignores `name` parameter
In-Reply-To: <1481636868.17.0.0411235027553.issue28961@psf.upfronthosting.co.za>
Message-ID: <1483454094.66.0.100242474447.issue28961@psf.upfronthosting.co.za>


Jiajun Huang added the comment:

sorry about that, fixed.

----------
Added file: http://bugs.python.org/file46126/mock.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28961>
_______________________________________

From report at bugs.python.org  Tue Jan  3 09:35:54 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 03 Jan 2017 14:35:54 +0000
Subject: [issue27671] FAQ: len() is still function for good reason.
In-Reply-To: <1470214886.13.0.870924776007.issue27671@psf.upfronthosting.co.za>
Message-ID: <1483454154.01.0.61242122109.issue27671@psf.upfronthosting.co.za>


Changes by STINNER Victor <victor.stinner at gmail.com>:


----------
nosy: +haypo

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue27671>
_______________________________________

From report at bugs.python.org  Tue Jan  3 09:36:05 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 03 Jan 2017 14:36:05 +0000
Subject: [issue28682] Bytes support in os.fwalk()
In-Reply-To: <1479029050.5.0.426622691368.issue28682@psf.upfronthosting.co.za>
Message-ID: <1483454165.46.0.907521881691.issue28682@psf.upfronthosting.co.za>


Changes by STINNER Victor <victor.stinner at gmail.com>:


----------
nosy: +haypo

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28682>
_______________________________________

From report at bugs.python.org  Tue Jan  3 09:36:42 2017
From: report at bugs.python.org (Claudio Freire)
Date: Tue, 03 Jan 2017 14:36:42 +0000
Subject: [issue25750] tp_descr_get(self, obj,
 type) is called without owning a reference to "self"
In-Reply-To: <1448654213.28.0.807225788685.issue25750@psf.upfronthosting.co.za>
Message-ID: <1483454202.69.0.0886790442231.issue25750@psf.upfronthosting.co.za>


Claudio Freire added the comment:

The crash (the one we're experiencing) still happens with 2.7.13. But at this point it's not clear whether it's a Python bug or a Cython bug, as jdemeyer's patch doesn't fix it. We're having a hard time getting accurate backtraces to actually debug this thing, as it only happens on production servers.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue25750>
_______________________________________

From report at bugs.python.org  Tue Jan  3 09:38:16 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 03 Jan 2017 14:38:16 +0000
Subject: [issue25750] tp_descr_get(self, obj,
 type) is called without owning a reference to "self"
In-Reply-To: <1448654213.28.0.807225788685.issue25750@psf.upfronthosting.co.za>
Message-ID: <1483454296.86.0.775397541511.issue25750@psf.upfronthosting.co.za>


STINNER Victor added the comment:

> The crash (the one we're experiencing) still happens with 2.7.13. But at this point it's not clear whether it's a Python bug or a Cython bug, as jdemeyer's patch doesn't fix it. We're having a hard time getting accurate backtraces to actually debug this thing, as it only happens on production servers.

I suggest you to try faulthandler to try to get a traceback ;-)

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue25750>
_______________________________________

From report at bugs.python.org  Tue Jan  3 09:42:34 2017
From: report at bugs.python.org (Jeroen Demeyer)
Date: Tue, 03 Jan 2017 14:42:34 +0000
Subject: [issue25750] tp_descr_get(self, obj,
 type) is called without owning a reference to "self"
In-Reply-To: <1448654213.28.0.807225788685.issue25750@psf.upfronthosting.co.za>
Message-ID: <1483454554.67.0.665569344098.issue25750@psf.upfronthosting.co.za>


Jeroen Demeyer added the comment:

<shameless plug> If you are on POSIX, you could also use cysignals to get a traceback (simply import cysignals, which will install a handler for fatal signals like SIGSEGV).

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue25750>
_______________________________________

From report at bugs.python.org  Tue Jan  3 09:44:09 2017
From: report at bugs.python.org (Claudio Freire)
Date: Tue, 03 Jan 2017 14:44:09 +0000
Subject: [issue25750] tp_descr_get(self, obj,
 type) is called without owning a reference to "self"
In-Reply-To: <1448654213.28.0.807225788685.issue25750@psf.upfronthosting.co.za>
Message-ID: <1483454649.79.0.697584425417.issue25750@psf.upfronthosting.co.za>


Claudio Freire added the comment:

Nice ideas, will give them a try

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue25750>
_______________________________________

From report at bugs.python.org  Tue Jan  3 10:02:01 2017
From: report at bugs.python.org (Barry A. Warsaw)
Date: Tue, 03 Jan 2017 15:02:01 +0000
Subject: [issue28180] sys.getfilesystemencoding() should default to utf-8
In-Reply-To: <1474024622.65.0.838685461995.issue28180@psf.upfronthosting.co.za>
Message-ID: <1483455721.25.0.516848856186.issue28180@psf.upfronthosting.co.za>


Changes by Barry A. Warsaw <barry at python.org>:


----------
nosy: +barry

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28180>
_______________________________________

From report at bugs.python.org  Tue Jan  3 10:20:56 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 03 Jan 2017 15:20:56 +0000
Subject: [issue29139] operator.concat/iconcat could only work if left operand
 is a sequence
In-Reply-To: <1483441668.99.0.123196699002.issue29139@psf.upfronthosting.co.za>
Message-ID: <CAMpsgwYQGEaB75KZsHKguKSahQcDVe-Fs3Y9DpemP=F8_MZaLw@mail.gmail.com>


STINNER Victor added the comment:

> More like a implementation detail for me.

IMHO it's a deliberate choice that the operator module behaves
differently for concat() and add().

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29139>
_______________________________________

From report at bugs.python.org  Tue Jan  3 10:31:40 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Tue, 03 Jan 2017 15:31:40 +0000
Subject: [issue29104] Left bracket remains in format string result when '\'
 preceeds it
In-Reply-To: <1483012585.51.0.342717701316.issue29104@psf.upfronthosting.co.za>
Message-ID: <1483457500.36.0.039312335623.issue29104@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

> f'\\N{AMPERSAND}': reading a local variable looks like a typo or a security vulnerability, rather than a nice feature.

This can look as a typo, but how would you write this if this is not a typo? f'\\N{AMPERSAND}' is legitimate syntax, and it would be weird to interpret this expression in any other way than as '\\' + 'N' + format(AMPERSAND).

> What is the issue with having to write "\\{10}" to get "\\10" string? It's less error prone.

There is no issue with f'\\{10}'. There is an issue with f'\{10}'.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29104>
_______________________________________

From report at bugs.python.org  Tue Jan  3 10:33:18 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 03 Jan 2017 15:33:18 +0000
Subject: [issue29104] Left bracket remains in format string result when '\'
 preceeds it
In-Reply-To: <1483012585.51.0.342717701316.issue29104@psf.upfronthosting.co.za>
Message-ID: <1483457598.39.0.321280958587.issue29104@psf.upfronthosting.co.za>


STINNER Victor added the comment:

> There is no issue with f'\\{10}'. There is an issue with f'\{10}'.

Hum, I'm not sure that my previous comment is explicit: I suggest to raise a syntax error on f'\{10}' (or emit a warning, and raise an error in 3.7 if you insist ;-)).

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29104>
_______________________________________

From report at bugs.python.org  Tue Jan  3 10:59:07 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Tue, 03 Jan 2017 15:59:07 +0000
Subject: [issue29104] Left bracket remains in format string result when '\'
 preceeds it
In-Reply-To: <1483012585.51.0.342717701316.issue29104@psf.upfronthosting.co.za>
Message-ID: <1483459147.57.0.756924847325.issue29104@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

Updated patch adds a warning.

----------
Added file: http://bugs.python.org/file46127/fstring_backslash_2.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29104>
_______________________________________

From report at bugs.python.org  Tue Jan  3 11:36:19 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Tue, 03 Jan 2017 16:36:19 +0000
Subject: [issue29139] operator.concat/iconcat could only work if left operand
 is a sequence
In-Reply-To: <1483437510.5.0.880271510583.issue29139@psf.upfronthosting.co.za>
Message-ID: <1483461379.94.0.669183150974.issue29139@psf.upfronthosting.co.za>


Xiang Zhang added the comment:

Actually this issue was discussed when import the pure Python implementation in #16694. There seems once an effort checking also the right operand but was later removed, but I couldn't understand why (though there is an explanation, but it's more about why removing __len__ checking). :-(

----------
nosy: +rhettinger

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29139>
_______________________________________

From report at bugs.python.org  Tue Jan  3 11:52:28 2017
From: report at bugs.python.org (Marco Buttu)
Date: Tue, 03 Jan 2017 16:52:28 +0000
Subject: [issue29133] Minor inaccuracy in shlex.shlex punctuation_chars example
In-Reply-To: <1483362012.78.0.704413629859.issue29133@psf.upfronthosting.co.za>
Message-ID: <1483462348.17.0.14058561118.issue29133@psf.upfronthosting.co.za>


Marco Buttu added the comment:

Yes, you are right. Here is a patch.

----------
Added file: http://bugs.python.org/file46128/issue29133_2nd.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29133>
_______________________________________

From report at bugs.python.org  Tue Jan  3 12:49:34 2017
From: report at bugs.python.org (Oleg Serov)
Date: Tue, 03 Jan 2017 17:49:34 +0000
Subject: [issue29143] Logger should ignore propagate property for disabled
 handlers.
Message-ID: <1483465774.47.0.809201158116.issue29143@psf.upfronthosting.co.za>


New submission from Oleg Serov:

Looks like if I have a logger handler with propagate = False and it is disabled, the "propagate" is still in affect.

I am considering this is a bug, because:
1. I have some random logging configuration with some logging handlers that have "propagate = True"
2. I am setting a new logging configuration by using logging.config.dictConfig with "disable_existing_loggers = True"
3. After that I still "feel" the effect of old config, because some log messages do not reach my new loggers.

What I am doing wrong? Or this is a bug? A bug in documentation?

----------
components: Library (Lib)
messages: 284583
nosy: Oleg Serov
priority: normal
severity: normal
status: open
title: Logger should ignore propagate property for disabled handlers.
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29143>
_______________________________________

From report at bugs.python.org  Tue Jan  3 12:52:07 2017
From: report at bugs.python.org (Oleg Serov)
Date: Tue, 03 Jan 2017 17:52:07 +0000
Subject: [issue29143] Logger should ignore propagate property for disabled
 handlers.
In-Reply-To: <1483465774.47.0.809201158116.issue29143@psf.upfronthosting.co.za>
Message-ID: <1483465927.77.0.545931812224.issue29143@psf.upfronthosting.co.za>


Oleg Serov added the comment:

> 1. I have some random logging configuration with some logging handlers that have "propagate = True"
Read as: 1. I have some random logging configuration with some logging handlers that have "propagate = False"

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29143>
_______________________________________

From report at bugs.python.org  Tue Jan  3 13:02:02 2017
From: report at bugs.python.org (Aviv Palivoda)
Date: Tue, 03 Jan 2017 18:02:02 +0000
Subject: [issue28518] execute("begin immediate") throwing OperationalError
In-Reply-To: <1477307118.31.0.70785963389.issue28518@psf.upfronthosting.co.za>
Message-ID: <1483466522.04.0.852349183999.issue28518@psf.upfronthosting.co.za>


Aviv Palivoda added the comment:

Yes. If a transaction is open you will need to explicitly commit before doing vacuum.
I am not sure if that was intentional or not. From quick look in the sqlite source code there are few things that cannot happen from a transaction:
1. VACUUM
2. ATTACH
3. DETACH
4. BEGIN
5. Few PRAGMAs (temp_store, synchronous)

All of the above worked with implicit commit before commit 284676cf2ac8 but now has to call commit explicitly.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28518>
_______________________________________

From report at bugs.python.org  Tue Jan  3 13:15:43 2017
From: report at bugs.python.org (Elias Zamaria)
Date: Tue, 03 Jan 2017 18:15:43 +0000
Subject: [issue29076] Py 3.6 Mac installer doesn't update "python3" shell
 command
In-Reply-To: <1482778000.76.0.584407589583.issue29076@psf.upfronthosting.co.za>
Message-ID: <1483467343.21.0.151954462037.issue29076@psf.upfronthosting.co.za>


Elias Zamaria added the comment:

Ned, that sounds plausible. I think I started using fish after I installed Python 3.5, but before I installed Python 3.6.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29076>
_______________________________________

From report at bugs.python.org  Tue Jan  3 13:27:01 2017
From: report at bugs.python.org (=?utf-8?q?Lum=C3=ADr_Balhar?=)
Date: Tue, 03 Jan 2017 18:27:01 +0000
Subject: [issue29144] Implicit namespace packages in Python 3.6
Message-ID: <1483468021.78.0.379899829208.issue29144@psf.upfronthosting.co.za>


New submission from Lum?r Balhar:

Hello.

I've found a really strange difference between Python 3.5 and 3.6 related to namespace packages and I cannot find any note in changelogs.

I've created a simple bash script which can reproduce my issue using virtual environments: http://pastebin.com/j3fXMtR4

This script creates a virtual environment, installs marrow.util from PyPI which is one part of namespace package and creates the other part (marrow.mailer) in the local folder. Then script tries to import marrow.util and marrow.mailer - and there is the difference.

In Python 3.5 - I can import marrow.util (installed via pip) but I cannot import local marrow.mailer module. I think that this is right behavior.
In Python 3.6 - I cannot import marrow.util (installed via pip) but I can import local marrow.mailer module.

Python versions are 3.5.2 (installed from Fedora repositories) and 3.6.0 (compiled manually).

Why there is such a big difference? Am I missing something?

I tried to ask on IRC, stackoverflow etc. but without success. Could you please explain the reason for this behavior to me?

Thank you and have a nice day.
Lum?r

Bash script attached

----------
files: test_namespace_package.sh
messages: 284587
nosy: frenzy
priority: normal
severity: normal
status: open
title: Implicit namespace packages in Python 3.6
type: behavior
versions: Python 3.6
Added file: http://bugs.python.org/file46129/test_namespace_package.sh

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29144>
_______________________________________

From report at bugs.python.org  Tue Jan  3 13:30:23 2017
From: report at bugs.python.org (Eric V. Smith)
Date: Tue, 03 Jan 2017 18:30:23 +0000
Subject: [issue29144] Implicit namespace packages in Python 3.6
In-Reply-To: <1483468021.78.0.379899829208.issue29144@psf.upfronthosting.co.za>
Message-ID: <1483468223.98.0.540567453334.issue29144@psf.upfronthosting.co.za>


Changes by Eric V. Smith <eric at trueblade.com>:


----------
nosy: +eric.smith

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29144>
_______________________________________

From report at bugs.python.org  Tue Jan  3 13:36:42 2017
From: report at bugs.python.org (jan matejek)
Date: Tue, 03 Jan 2017 18:36:42 +0000
Subject: [issue29145] failing overflow checks in replace_*
Message-ID: <1483468602.77.0.231507148809.issue29145@psf.upfronthosting.co.za>


New submission from jan matejek:

Related to http://bugs.python.org/issue1621 and http://bugs.python.org/issue27473
GCC 6 optimizes away broken overflow checks. This leads to segfaults on test_replace_overflow, at least for strings and bytearrays.

----------
components: Interpreter Core
messages: 284588
nosy: matejcik
priority: normal
severity: normal
status: open
title: failing overflow checks in replace_*
type: crash
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29145>
_______________________________________

From report at bugs.python.org  Tue Jan  3 14:25:05 2017
From: report at bugs.python.org (Berker Peksag)
Date: Tue, 03 Jan 2017 19:25:05 +0000
Subject: [issue28961] unittest.mock._Call ignores `name` parameter
In-Reply-To: <1481636868.17.0.0411235027553.issue28961@psf.upfronthosting.co.za>
Message-ID: <1483471505.43.0.938496397541.issue28961@psf.upfronthosting.co.za>


Berker Peksag added the comment:

The patch looks good to me. There are some styling issues in the test code (e.g. no need to add trailing spaces after commas in some cases), but I can take care of them before committing the patch :) Thanks!

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28961>
_______________________________________

From report at bugs.python.org  Tue Jan  3 14:45:46 2017
From: report at bugs.python.org (Martin Panter)
Date: Tue, 03 Jan 2017 19:45:46 +0000
Subject: [issue26228] pty.spawn hangs on FreeBSD 9.3, 10.x
In-Reply-To: <1453953861.58.0.104331734212.issue26228@psf.upfronthosting.co.za>
Message-ID: <1483472746.56.0.60647716945.issue26228@psf.upfronthosting.co.za>


Martin Panter added the comment:

Behaviour change in Free BSD as I understand. Nothing changed in Python, but perhaps older versions of Free BSD behaved like Linux and raised EIO (or another errno; it is not clear).

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue26228>
_______________________________________

From report at bugs.python.org  Tue Jan  3 16:12:51 2017
From: report at bugs.python.org (John Parejko)
Date: Tue, 03 Jan 2017 21:12:51 +0000
Subject: [issue29146] Confusing "invalid token" exception when integers have
 leading zero
Message-ID: <1483477971.7.0.132370199799.issue29146@psf.upfronthosting.co.za>


New submission from John Parejko:

As described in PEP-3127, the "leading-zeros" formatting for octal was removed from python 3. This is a good thing(tm), but the recommendation of that PEP to improve the error message of the raised exception[1] was apparently never implemented. I just ran into this while with some recently-ported python2 code, and it took a while to figure out the problem.

Although this is going to be less of a problem with time as people convert to pure python3, it will be very helpful during the transition period.

>>> 0o007
7
>>> 007
  File "<stdin>", line 1
    007
      ^
SyntaxError: invalid token


1: https://www.python.org/dev/peps/pep-3127/#id17

----------
assignee: docs at python
components: 2to3 (2.x to 3.x conversion tool), Documentation, Interpreter Core
messages: 284591
nosy: John Parejko, docs at python
priority: normal
severity: normal
status: open
title: Confusing "invalid token" exception when integers have leading zero
type: behavior
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29146>
_______________________________________

From report at bugs.python.org  Tue Jan  3 16:31:04 2017
From: report at bugs.python.org (John Parejko)
Date: Tue, 03 Jan 2017 21:31:04 +0000
Subject: [issue29146] Confusing "invalid token" exception when integers have
 leading zero
In-Reply-To: <1483477971.7.0.132370199799.issue29146@psf.upfronthosting.co.za>
Message-ID: <1483479064.81.0.527151005614.issue29146@psf.upfronthosting.co.za>


John Parejko added the comment:

Ah, I finally found a related issue, and it looks like it has patches! https://bugs.python.org/issue20608

If someone could check that over and merge it, that would be wonderful!

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29146>
_______________________________________

From report at bugs.python.org  Tue Jan  3 16:49:54 2017
From: report at bugs.python.org (Petr Viktorin)
Date: Tue, 03 Jan 2017 21:49:54 +0000
Subject: [issue27961] remove support for platforms without "long long"
In-Reply-To: <1473119429.61.0.683650779093.issue27961@psf.upfronthosting.co.za>
Message-ID: <1483480194.26.0.270742029315.issue27961@psf.upfronthosting.co.za>


Petr Viktorin added the comment:

The backwards compatibility is not as strong as it could be: previously, HAVE_LONG_LONG was defined to 1; now it's defined but empty.

At least that's the case on Fedora.
Found in the Python plugin for GCC: https://bugzilla.redhat.com/show_bug.cgi?id=1409009

----------
nosy: +encukou

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue27961>
_______________________________________

From report at bugs.python.org  Tue Jan  3 17:02:00 2017
From: report at bugs.python.org (Brett Cannon)
Date: Tue, 03 Jan 2017 22:02:00 +0000
Subject: [issue29141] error in 2to3
In-Reply-To: <1483452268.26.0.428258207401.issue29141@psf.upfronthosting.co.za>
Message-ID: <1483480920.48.0.106413950031.issue29141@psf.upfronthosting.co.za>


Brett Cannon added the comment:

To help you work around this you can simplify your original code such that you don't trigger the issue:

dict((key, val) for key, val in docs_uploaded_at.iteritems() if len(val) == 3)

and

[x for x in registered if x.phone not in with_transaction_mobile]

----------
nosy: +brett.cannon

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29141>
_______________________________________

From report at bugs.python.org  Tue Jan  3 17:12:24 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 03 Jan 2017 22:12:24 +0000
Subject: [issue28961] unittest.mock._Call ignores `name` parameter
In-Reply-To: <1481636868.17.0.0411235027553.issue28961@psf.upfronthosting.co.za>
Message-ID: <1483481544.45.0.843755172539.issue28961@psf.upfronthosting.co.za>


STINNER Victor added the comment:

The latest patch (Jiajun Huang, 2017-01-03 14:34) LGTM.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28961>
_______________________________________

From report at bugs.python.org  Tue Jan  3 17:45:29 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 03 Jan 2017 22:45:29 +0000
Subject: [issue29140] time_hash() reads the wrong bytes to get microseconds
In-Reply-To: <1483451829.62.0.434619516952.issue29140@psf.upfronthosting.co.za>
Message-ID: <1483483529.72.0.69842745814.issue29140@psf.upfronthosting.co.za>


STINNER Victor added the comment:

I used attached check.patch to check if macros are misused in other functions. Hopefully, only time_hash() has bugs.

I don't think that it's worth it to apply check.patch, I dislike such complex macro.

----------
Added file: http://bugs.python.org/file46130/check.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29140>
_______________________________________

From report at bugs.python.org  Tue Jan  3 17:48:19 2017
From: report at bugs.python.org (Roundup Robot)
Date: Tue, 03 Jan 2017 22:48:19 +0000
Subject: [issue29140] time_hash() reads the wrong bytes to get microseconds
In-Reply-To: <1483451829.62.0.434619516952.issue29140@psf.upfronthosting.co.za>
Message-ID: <20170103224816.28542.32882.43F9FE3A@psf.io>


Roundup Robot added the comment:

New changeset 72e48ff7b169 by Victor Stinner in branch '3.6':
Issue #29140: Fix hash(datetime.time)
https://hg.python.org/cpython/rev/72e48ff7b169

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29140>
_______________________________________

From report at bugs.python.org  Tue Jan  3 17:48:40 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 03 Jan 2017 22:48:40 +0000
Subject: [issue29140] time_hash() reads the wrong bytes to get microseconds
In-Reply-To: <1483451829.62.0.434619516952.issue29140@psf.upfronthosting.co.za>
Message-ID: <1483483720.75.0.237157158486.issue29140@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Note: I found this bug while working on the issue #29100.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29140>
_______________________________________

From report at bugs.python.org  Tue Jan  3 17:58:20 2017
From: report at bugs.python.org (Evan)
Date: Tue, 03 Jan 2017 22:58:20 +0000
Subject: [issue29133] Minor inaccuracy in shlex.shlex punctuation_chars example
In-Reply-To: <1483362012.78.0.704413629859.issue29133@psf.upfronthosting.co.za>
Message-ID: <1483484300.23.0.763777692003.issue29133@psf.upfronthosting.co.za>


Evan added the comment:

Second patch looks good, thanks. Do you also want to doctest that?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29133>
_______________________________________

From report at bugs.python.org  Tue Jan  3 18:12:41 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 03 Jan 2017 23:12:41 +0000
Subject: [issue29100] datetime.fromtimestamp() doesn't check min/max year
 anymore: regression of Python 3.6
In-Reply-To: <1482972647.78.0.524140289263.issue29100@psf.upfronthosting.co.za>
Message-ID: <1483485161.94.0.763876502127.issue29100@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Oh, datetime.datetime.timestamp() also has a bug for values close to 0001-01-01 (year 1), at least in my timezone (CET).

$ ./python -c 'import datetime; datetime.datetime.min.timestamp()'
python: /home/haypo/prog/python/default/Modules/_datetimemodule.c:251: days_before_year: Assertion `year >= 1' failed.
Aborted (core dumped)

datetime_check_args.patch doesn't fix this issue. A solution is to check that year>=1 in utc_to_seconds():

+    if (year < MINYEAR || year > MAXYEAR) {
+        PyErr_SetString(PyExc_ValueError,
+                        "year is out of range");
+        return -1;
+    }

Maybe it would solve the following local() comment:

   /* XXX: add bounds checking */

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29100>
_______________________________________

From report at bugs.python.org  Tue Jan  3 18:14:17 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 03 Jan 2017 23:14:17 +0000
Subject: [issue29100] datetime.fromtimestamp() doesn't check min/max year
 anymore: regression of Python 3.6
In-Reply-To: <1482972647.78.0.524140289263.issue29100@psf.upfronthosting.co.za>
Message-ID: <1483485257.75.0.725127448463.issue29100@psf.upfronthosting.co.za>


STINNER Victor added the comment:

unitttest.patch: my attempt to write an unit test, but I was bitten by the datetime.timestamp() bug.

----------
Added file: http://bugs.python.org/file46131/unittest.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29100>
_______________________________________

From report at bugs.python.org  Tue Jan  3 18:21:42 2017
From: report at bugs.python.org (Berker Peksag)
Date: Tue, 03 Jan 2017 23:21:42 +0000
Subject: [issue29146] Confusing "invalid token" exception when integers have
 leading zero
In-Reply-To: <1483477971.7.0.132370199799.issue29146@psf.upfronthosting.co.za>
Message-ID: <1483485702.68.0.651041909286.issue29146@psf.upfronthosting.co.za>


Berker Peksag added the comment:

Thanks for finding the duplicate. I'm closing this as a duplicate of issue 20608.

----------
components:  -2to3 (2.x to 3.x conversion tool), Documentation
nosy: +berker.peksag -docs at python
resolution:  -> duplicate
stage:  -> resolved
status: open -> closed
superseder:  -> 'SyntaxError: invalid token' is unfriendly

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29146>
_______________________________________

From report at bugs.python.org  Tue Jan  3 18:24:16 2017
From: report at bugs.python.org (Berker Peksag)
Date: Tue, 03 Jan 2017 23:24:16 +0000
Subject: [issue20608] 'SyntaxError: invalid token' is unfriendly
In-Reply-To: <1392213800.84.0.505532685284.issue20608@psf.upfronthosting.co.za>
Message-ID: <1483485856.32.0.894140933978.issue20608@psf.upfronthosting.co.za>


Changes by Berker Peksag <berker.peksag at gmail.com>:


----------
versions: +Python 3.7 -Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue20608>
_______________________________________

From report at bugs.python.org  Tue Jan  3 18:38:43 2017
From: report at bugs.python.org (John Parejko)
Date: Tue, 03 Jan 2017 23:38:43 +0000
Subject: [issue20608] 'SyntaxError: invalid token' is unfriendly
In-Reply-To: <1392213800.84.0.505532685284.issue20608@psf.upfronthosting.co.za>
Message-ID: <1483486723.07.0.232209638879.issue20608@psf.upfronthosting.co.za>


Changes by John Parejko <parejkoj at gmail.com>:


----------
nosy: +John Parejko

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue20608>
_______________________________________

From report at bugs.python.org  Tue Jan  3 18:40:34 2017
From: report at bugs.python.org (John Parejko)
Date: Tue, 03 Jan 2017 23:40:34 +0000
Subject: [issue20608] 'SyntaxError: invalid token' is unfriendly
In-Reply-To: <1392213800.84.0.505532685284.issue20608@psf.upfronthosting.co.za>
Message-ID: <1483486834.52.0.090301875289.issue20608@psf.upfronthosting.co.za>


John Parejko added the comment:

I had filed issue 29146 but eventually found this, which has patches to fix the exception messages. Could someone please look at this and get it incorporated into python?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue20608>
_______________________________________

From report at bugs.python.org  Tue Jan  3 19:58:14 2017
From: report at bugs.python.org (Carlos Alberto Lopez Perez)
Date: Wed, 04 Jan 2017 00:58:14 +0000
Subject: [issue22417] PEP 476: verify HTTPS certificates by default
In-Reply-To: <1410784469.95.0.475937020841.issue22417@psf.upfronthosting.co.za>
Message-ID: <1483491494.56.0.695863332416.issue22417@psf.upfronthosting.co.za>


Carlos Alberto Lopez Perez added the comment:

The python 2.7 documentation for urrlib still has a big warning notice at the top saying:

"""
Warning

When opening HTTPS URLs, it does not attempt to validate the server certificate. Use at your own risk! 
"""
^^ https://docs.python.org/2/library/urllib.html

I believe this is incorrect since this patch was backported to the 2.7  branch. I checked it, and it verifies SSL certs by default.


I guess the documentation for urllib should be updated to remove that warning?

----------
nosy: +clopez

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue22417>
_______________________________________

From report at bugs.python.org  Tue Jan  3 20:02:24 2017
From: report at bugs.python.org (INADA Naoki)
Date: Wed, 04 Jan 2017 01:02:24 +0000
Subject: [issue28180] sys.getfilesystemencoding() should default to utf-8
In-Reply-To: <1474024622.65.0.838685461995.issue28180@psf.upfronthosting.co.za>
Message-ID: <1483491744.41.0.760927522523.issue28180@psf.upfronthosting.co.za>


INADA Naoki added the comment:

I read PEP 538 but I can't understand why just using UTF-8 when locale is C like macOS is bad idea.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28180>
_______________________________________

From report at bugs.python.org  Tue Jan  3 21:23:15 2017
From: report at bugs.python.org (R. David Murray)
Date: Wed, 04 Jan 2017 02:23:15 +0000
Subject: [issue29130] Exit code 120 returned from Python unit test testing
 SystemExit
In-Reply-To: <1483316898.11.0.0292789374137.issue29130@psf.upfronthosting.co.za>
Message-ID: <1483496595.71.0.903808609966.issue29130@psf.upfronthosting.co.za>


Changes by R. David Murray <rdmurray at bitdance.com>:


----------
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29130>
_______________________________________

From report at bugs.python.org  Tue Jan  3 21:57:54 2017
From: report at bugs.python.org (Greg Bengeult)
Date: Wed, 04 Jan 2017 02:57:54 +0000
Subject: [issue16026] csv.DictReader argument names documented incorrectly
In-Reply-To: <1348505583.77.0.632308357055.issue16026@psf.upfronthosting.co.za>
Message-ID: <1483498674.53.0.73466442702.issue16026@psf.upfronthosting.co.za>


Greg Bengeult added the comment:

Here's a revised patch file for Python 3.7. This is my first ever submission to an open source project, so please be gentle.

----------
nosy: +gbengeult
versions:  -Python 2.7, Python 3.5, Python 3.6
Added file: http://bugs.python.org/file46132/csv.rst-patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue16026>
_______________________________________

From report at bugs.python.org  Tue Jan  3 22:51:41 2017
From: report at bugs.python.org (Benjamin Peterson)
Date: Wed, 04 Jan 2017 03:51:41 +0000
Subject: [issue29145] failing overflow checks in replace_*
In-Reply-To: <1483468602.77.0.231507148809.issue29145@psf.upfronthosting.co.za>
Message-ID: <1483501901.97.0.563556052429.issue29145@psf.upfronthosting.co.za>


Benjamin Peterson added the comment:

Does that mean it no longer respects -fwrapv?

----------
nosy: +benjamin.peterson

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29145>
_______________________________________

From report at bugs.python.org  Tue Jan  3 23:48:12 2017
From: report at bugs.python.org (jha.amit6666)
Date: Wed, 04 Jan 2017 04:48:12 +0000
Subject: [issue29147] registry value to be cleared when python is uninstalled
Message-ID: <1483505292.36.0.945723017008.issue29147@psf.upfronthosting.co.za>


New submission from jha.amit6666:

I have installed python 3.5 on Windows and then uninstalled it. THe registry value HKEY_LOCAL_MACHINE\SOFTWARE\Python\PythonCore\3.5 is still present.

This is solved in python 3.6. So the work to be done here is: if there was any previous version of python installed and not present now, then the registry values should be cleared.

The behaviour of uninstallation of Python 3.6 is that the registry value is cleared.

----------
components: Windows
messages: 284608
nosy: jha.amit6666, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: registry value to be cleared when python is uninstalled
type: behavior
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29147>
_______________________________________

From report at bugs.python.org  Wed Jan  4 00:09:06 2017
From: report at bugs.python.org (Senthil Kumaran)
Date: Wed, 04 Jan 2017 05:09:06 +0000
Subject: [issue29134] contextlib doc bug
In-Reply-To: <1483376051.72.0.678487921628.issue29134@psf.upfronthosting.co.za>
Message-ID: <1483506546.83.0.111829364604.issue29134@psf.upfronthosting.co.za>


Senthil Kumaran added the comment:

I am going to vote for "not a bug" here. From the context is it understood that the `ContextBaseClass` is to be provided by the user.

> Existing context managers that already have a base class can be extended by using ContextDecorator as a mixin class:

The idea here is the illustrate the Mixin possibility with a Base class and not to provide a full example with Base class code as well.

@Marco, also not a good ticket add more doc strings to this module.rst. If the docs of those can be improved, they can done separately, concentrating on all the possible improvements.

----------
nosy: +orsenthil
resolution:  -> not a bug
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29134>
_______________________________________

From report at bugs.python.org  Wed Jan  4 00:14:47 2017
From: report at bugs.python.org (Yugansh Marwah)
Date: Wed, 04 Jan 2017 05:14:47 +0000
Subject: [issue29148] Inheritance behaviour ambiguos
Message-ID: <1483506887.2.0.751213470204.issue29148@psf.upfronthosting.co.za>


New submission from Yugansh Marwah:

Inheritance property behaves ambiguously on inheriting multiple inheritance over a class
It even calls the function of un-inherited class
code is attached below kindly run it and tell me the exact behavior of the same.

----------
components: Tests
files: oops.py
messages: 284610
nosy: yugansh94
priority: normal
severity: normal
status: open
title: Inheritance behaviour ambiguos
type: behavior
versions: Python 3.5
Added file: http://bugs.python.org/file46133/oops.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29148>
_______________________________________

From report at bugs.python.org  Wed Jan  4 00:19:48 2017
From: report at bugs.python.org (irdb)
Date: Wed, 04 Jan 2017 05:19:48 +0000
Subject: [issue17301] An in-place version of many bytearray methods is needed
In-Reply-To: <1361910523.06.0.491264384267.issue17301@psf.upfronthosting.co.za>
Message-ID: <1483507188.22.0.613642827544.issue17301@psf.upfronthosting.co.za>


Changes by irdb <electro.nnn at gmail.com>:


----------
nosy: +irdb

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue17301>
_______________________________________

From report at bugs.python.org  Wed Jan  4 00:43:10 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Wed, 04 Jan 2017 05:43:10 +0000
Subject: [issue29148] Inheritance behaviour ambiguos
In-Reply-To: <1483506887.2.0.751213470204.issue29148@psf.upfronthosting.co.za>
Message-ID: <1483508590.84.0.992005751385.issue29148@psf.upfronthosting.co.za>


Xiang Zhang added the comment:

It seems you are surprised `super(C, self).foo2()` works but it is actually how super() and class.__mro__ works. :-) So not a bug. You could learn more about it but sorry I don't have any good reference for you. :-(

----------
nosy: +xiang.zhang
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29148>
_______________________________________

From report at bugs.python.org  Wed Jan  4 00:47:54 2017
From: report at bugs.python.org (jha.amit6666)
Date: Wed, 04 Jan 2017 05:47:54 +0000
Subject: [issue29147] registry value to be cleared when python is uninstalled
In-Reply-To: <1483505292.36.0.945723017008.issue29147@psf.upfronthosting.co.za>
Message-ID: <1483508874.03.0.424685572877.issue29147@psf.upfronthosting.co.za>


jha.amit6666 added the comment:

The start menu shortcuts and the files in installation path "C:\Program Files\Python35" are also present. The thing should go when python3.5 is uninstalled.

This can be solved when newer version of python, i.e., python 3.6 is installed/uninstalled. During this time, a prompt should be shown.

----------
Added file: http://bugs.python.org/file46134/1.png

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29147>
_______________________________________

From report at bugs.python.org  Wed Jan  4 00:55:19 2017
From: report at bugs.python.org (Berker Peksag)
Date: Wed, 04 Jan 2017 05:55:19 +0000
Subject: [issue16026] csv.DictReader argument names documented incorrectly
In-Reply-To: <1348505583.77.0.632308357055.issue16026@psf.upfronthosting.co.za>
Message-ID: <1483509319.74.0.604623710606.issue16026@psf.upfronthosting.co.za>


Berker Peksag added the comment:

Welcome and thanks for the patch, Greg. csv.rst-patch looks good to me. I will commit it this week if there are no objections.

Documentation updates can go into all active branches so I'm adding 3.5 and 3.6 back (we can pass 2.7 at this point since this is a trivial change)

----------
stage: needs patch -> patch review
versions: +Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue16026>
_______________________________________

From report at bugs.python.org  Wed Jan  4 01:05:07 2017
From: report at bugs.python.org (Sumner Hearth)
Date: Wed, 04 Jan 2017 06:05:07 +0000
Subject: [issue29149] SSL.py recursion limit crash
Message-ID: <1483509907.61.0.802720332804.issue29149@psf.upfronthosting.co.za>


New submission from Sumner Hearth:

Error in python 3.6 ssl.py:

[...]
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 737, in __init__
    self._context.verify_mode = cert_reqs
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 479, in verify_mode
    super(SSLContext, SSLContext).verify_mode.__set__(self, value)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 479, in verify_mode
    super(SSLContext, SSLContext).verify_mode.__set__(self, value)
[Repeated]

Crash is infinite recursion on super calls having to do with the verify_mode property

Crash found when creating an SSL context for eventlet with flask-socketio, unfortunately easiest way to reproduce involves installing eventlet, flask, flask-socketio, and pyopenssl. I've included the smallest working example of the crash in python 3.6, attached code works in python 3.5.

Note:
I'm attempting to create a dependency-free version of the file which exhibits the same behavior, it's tricky to unwrap all the libraries leading to the issue. See https://github.com/miguelgrinberg/Flask-SocketIO/issues/193 for original post discovering bug.

----------
assignee: christian.heimes
components: SSL
files: crash.py
messages: 284614
nosy: Sumner Hearth, christian.heimes
priority: normal
severity: normal
status: open
title: SSL.py recursion limit crash
type: crash
versions: Python 3.6
Added file: http://bugs.python.org/file46135/crash.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29149>
_______________________________________

From report at bugs.python.org  Wed Jan  4 01:06:44 2017
From: report at bugs.python.org (Steve Dower)
Date: Wed, 04 Jan 2017 06:06:44 +0000
Subject: [issue29147] registry value to be cleared when python is uninstalled
In-Reply-To: <1483505292.36.0.945723017008.issue29147@psf.upfronthosting.co.za>
Message-ID: <1483510004.53.0.331794080907.issue29147@psf.upfronthosting.co.za>


Steve Dower added the comment:

Interesting. I don't suppose you could find all the log files in your %TEMP% directory, put them in a zip file and attach them here? There may be some clue as to why this happened - it's certainly meant to clean everything up when you uninstall (except for site-packages, but definitely shortcuts and registry entries).

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29147>
_______________________________________

From report at bugs.python.org  Wed Jan  4 01:47:48 2017
From: report at bugs.python.org (Masayuki Yamamoto)
Date: Wed, 04 Jan 2017 06:47:48 +0000
Subject: [issue28768] Warning: implicit declaration of function '_setmode'
In-Reply-To: <1479758430.88.0.97682867264.issue28768@psf.upfronthosting.co.za>
Message-ID: <1483512468.5.0.0229381227438.issue28768@psf.upfronthosting.co.za>


Masayuki Yamamoto added the comment:

Building 2.7 is for testing purposes. It is used to judge whether it is a problem (specifically for Cygwin-specific) originated in the old version. _setmode is an important function for setting input and output of CPython, so in Cygwin I feel annoying that warnings are issued even though there is actually no problem.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28768>
_______________________________________

From report at bugs.python.org  Wed Jan  4 01:58:07 2017
From: report at bugs.python.org (Sumner Hearth)
Date: Wed, 04 Jan 2017 06:58:07 +0000
Subject: [issue29149] SSL.py recursion limit crash
In-Reply-To: <1483509907.61.0.802720332804.issue29149@psf.upfronthosting.co.za>
Message-ID: <1483513087.53.0.0408138878329.issue29149@psf.upfronthosting.co.za>


Sumner Hearth added the comment:

Recursion crash disappears in eventlet 0.17.4

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29149>
_______________________________________

From report at bugs.python.org  Wed Jan  4 02:40:39 2017
From: report at bugs.python.org (INADA Naoki)
Date: Wed, 04 Jan 2017 07:40:39 +0000
Subject: [issue17301] An in-place version of many bytearray methods is needed
In-Reply-To: <1361910523.06.0.491264384267.issue17301@psf.upfronthosting.co.za>
Message-ID: <1483515639.11.0.250761365473.issue17301@psf.upfronthosting.co.za>


INADA Naoki added the comment:

I prefer ".inplace_translate" to ".mtranslate", but I'm not sure about it's worth enough to add it to bytearray.

In last year, I proposed adding `bytes.frombuffer()` constructor to avoid
unnecessary memory copy.
https://mail.python.org/pipermail/python-dev/2016-October/146668.html

Nick Coghlan againsted for adding methods to builtin type only for low level
programming, and suggested to implement it in 3rd party library first.
https://mail.python.org/pipermail/python-dev/2016-October/146746.html
https://mail.python.org/pipermail/python-dev/2016-October/146763.html

----------
nosy: +inada.naoki

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue17301>
_______________________________________

From report at bugs.python.org  Wed Jan  4 02:48:08 2017
From: report at bugs.python.org (Sumner Hearth)
Date: Wed, 04 Jan 2017 07:48:08 +0000
Subject: [issue29149] SSL.py recursion limit crash
In-Reply-To: <1483509907.61.0.802720332804.issue29149@psf.upfronthosting.co.za>
Message-ID: <1483516088.2.0.203411332871.issue29149@psf.upfronthosting.co.za>


Sumner Hearth added the comment:

Attached a simpler version of the code with same error using only eventlet.

----------
Added file: http://bugs.python.org/file46136/crash_simple.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29149>
_______________________________________

From report at bugs.python.org  Wed Jan  4 03:01:45 2017
From: report at bugs.python.org (Nick Coghlan)
Date: Wed, 04 Jan 2017 08:01:45 +0000
Subject: [issue28180] sys.getfilesystemencoding() should default to utf-8
In-Reply-To: <1474024622.65.0.838685461995.issue28180@psf.upfronthosting.co.za>
Message-ID: <1483516905.86.0.104790994099.issue28180@psf.upfronthosting.co.za>


Nick Coghlan added the comment:

On Mac OS X, the XCode libc already ignores the locale settings and just uses UTF-8 as the default text encoding, so the hardcoding in CPython aligns with that behaviour.

That isn't the case on other *nix systems - there, we need CPython to be consistent with the configured C/C++ locale, *and* we need it to be using something other than ASCII as the default encoding.

Answer: coerce the default locale from C to C.UTF-8 (if available), or to en_US.UTF-8 (for older distros that don't provide C.UTF-8). (The latter aspect isn't in the PEP yet, it's an improvement that came up in the linux-sig discussions: https://github.com/python/peps/issues/171 )

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28180>
_______________________________________

From report at bugs.python.org  Wed Jan  4 03:29:24 2017
From: report at bugs.python.org (INADA Naoki)
Date: Wed, 04 Jan 2017 08:29:24 +0000
Subject: [issue28180] sys.getfilesystemencoding() should default to utf-8
In-Reply-To: <1474024622.65.0.838685461995.issue28180@psf.upfronthosting.co.za>
Message-ID: <1483518564.33.0.994685284994.issue28180@psf.upfronthosting.co.za>


INADA Naoki added the comment:

> That isn't the case on other *nix systems - there, we need CPython to be consistent with the configured C/C++ locale, *and* we need it to be using something other than ASCII as the default encoding.

Isn't using UTF-8 as filesystem encoding and stdin/stdout encoding consistent with C or POSIX locale?

Don't "modern" programming environments (Rust, Go, node.js) use UTF-8 even if locale is C or POSIX?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28180>
_______________________________________

From report at bugs.python.org  Wed Jan  4 04:14:58 2017
From: report at bugs.python.org (Marco Buttu)
Date: Wed, 04 Jan 2017 09:14:58 +0000
Subject: [issue29133] Minor inaccuracy in shlex.shlex punctuation_chars example
In-Reply-To: <1483362012.78.0.704413629859.issue29133@psf.upfronthosting.co.za>
Message-ID: <1483521298.67.0.658350031768.issue29133@psf.upfronthosting.co.za>


Marco Buttu added the comment:

I did not add the doctest directive on purpose. In fact, if in the future we start running the doctests with buildbot (see issue27200), the tests related to this issue will pass only for Python >= 3.6. If we _currently_ want the doctests to pass for every Python version, I see only two solutions:

1) we do not have to doctest the example, as I did in issue29133_2nd.patch
2) we have to apply two patches, one with doctests (for the doc to be tested with Python >= 3.6), and another one without doctests (for the doc to be tested with Python < 3.6).

I think the best solution is to add an option to the doctest directive, that allows us to specify for which Python version the tests have to be run:

https://github.com/sphinx-doc/sphinx/issues/3303

I would like the commiters to give a feedback about this, because if they think  this solution is doable, then I will spend the time to work on the Sphinx issue.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29133>
_______________________________________

From report at bugs.python.org  Wed Jan  4 04:27:25 2017
From: report at bugs.python.org (Marco Buttu)
Date: Wed, 04 Jan 2017 09:27:25 +0000
Subject: [issue29133] Minor inaccuracy in shlex.shlex punctuation_chars example
In-Reply-To: <1483362012.78.0.704413629859.issue29133@psf.upfronthosting.co.za>
Message-ID: <1483522045.15.0.839247203392.issue29133@psf.upfronthosting.co.za>


Changes by Marco Buttu <marco.buttu at gmail.com>:


----------
nosy: +r.david.murray

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29133>
_______________________________________

From report at bugs.python.org  Wed Jan  4 04:42:57 2017
From: report at bugs.python.org (Christian Heimes)
Date: Wed, 04 Jan 2017 09:42:57 +0000
Subject: [issue22417] PEP 476: verify HTTPS certificates by default
In-Reply-To: <1410784469.95.0.475937020841.issue22417@psf.upfronthosting.co.za>
Message-ID: <1483522977.95.0.0887562011571.issue22417@psf.upfronthosting.co.za>


Christian Heimes added the comment:

Carlos, you are correct. Please create a new issue and make it a documentation issue for 2.7. Thanks!

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue22417>
_______________________________________

From report at bugs.python.org  Wed Jan  4 04:53:39 2017
From: report at bugs.python.org (Christian Heimes)
Date: Wed, 04 Jan 2017 09:53:39 +0000
Subject: [issue29149] SSL.py recursion limit crash
In-Reply-To: <1483509907.61.0.802720332804.issue29149@psf.upfronthosting.co.za>
Message-ID: <1483523619.23.0.391131091833.issue29149@psf.upfronthosting.co.za>


Christian Heimes added the comment:

It's a bug in eventlet's monkey patch, not a bug in Python's ssl module. I've seen a similar issue with gevent's monkey patch.

----------
resolution:  -> third party

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29149>
_______________________________________

From report at bugs.python.org  Wed Jan  4 05:40:40 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Wed, 04 Jan 2017 10:40:40 +0000
Subject: [issue17301] An in-place version of many bytearray methods is needed
In-Reply-To: <1361910523.06.0.491264384267.issue17301@psf.upfronthosting.co.za>
Message-ID: <1483526440.43.0.687007592469.issue17301@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

> The important reasons for this are memory use and cache thrashing, not just
> CPU time.

Memory use is not an issue unless you translate hundreds of megabytes at a time. Cache trashing at the end is performance issue.

The original patch is no longer applied cleanly. Here is a patch synchronized with current sources and with fixed one error. I didn't look at it closely and don't know whether it has other bugs.

Here are results of microbenchmarking.

$ ./python -m perf timeit -s "table = bytes(range(256)).swapcase(); data = bytearray(range(256))*1000" -- "data = data.translate(table)"
Median +- std dev: 1.48 ms +- 0.02 ms
$ ./python -m perf timeit -s "table = bytes(range(256)).swapcase(); data = bytearray(range(256))*1000" -- "data[:] = data.translate(table)"
Median +- std dev: 1.60 ms +- 0.09 ms
$ ./python -m perf timeit -s "table = bytes(range(256)).swapcase(); data = bytearray(range(256))*1000" -- "data.mtranslate(table)"
Median +- std dev: 1.79 ms +- 0.07 ms

In-place translate don't have benefits. It is slower that translate with creating a new copy, and even is slower that translate with copying a new copy back.

----------
Added file: http://bugs.python.org/file46137/issue17301-2.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue17301>
_______________________________________

From report at bugs.python.org  Wed Jan  4 05:49:01 2017
From: report at bugs.python.org (Christian Heimes)
Date: Wed, 04 Jan 2017 10:49:01 +0000
Subject: [issue17301] An in-place version of many bytearray methods is needed
In-Reply-To: <1361910523.06.0.491264384267.issue17301@psf.upfronthosting.co.za>
Message-ID: <1483526941.9.0.667667196469.issue17301@psf.upfronthosting.co.za>


Christian Heimes added the comment:

Not every machine has hundreds of MB of memory to waste. Some devices (IoT, MicroPython, Raspberry Pi, OLPC) have limited resources.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue17301>
_______________________________________

From report at bugs.python.org  Wed Jan  4 05:55:42 2017
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 04 Jan 2017 10:55:42 +0000
Subject: [issue17301] An in-place version of many bytearray methods is needed
In-Reply-To: <1361910523.06.0.491264384267.issue17301@psf.upfronthosting.co.za>
Message-ID: <1483527342.48.0.648146420799.issue17301@psf.upfronthosting.co.za>


Changes by STINNER Victor <victor.stinner at gmail.com>:


----------
nosy: +haypo

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue17301>
_______________________________________

From report at bugs.python.org  Wed Jan  4 06:02:38 2017
From: report at bugs.python.org (Roundup Robot)
Date: Wed, 04 Jan 2017 11:02:38 +0000
Subject: [issue24773] Implement PEP 495 (Local Time Disambiguation)
In-Reply-To: <1438456647.64.0.191464539374.issue24773@psf.upfronthosting.co.za>
Message-ID: <20170104110235.123978.8022.EBC6B19C@psf.io>


Roundup Robot added the comment:

New changeset 5c02f689c62c by Victor Stinner in branch '3.6':
Issue #24773: fix datetime.time constructor docstring
https://hg.python.org/cpython/rev/5c02f689c62c

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24773>
_______________________________________

From report at bugs.python.org  Wed Jan  4 06:06:08 2017
From: report at bugs.python.org (Marco Buttu)
Date: Wed, 04 Jan 2017 11:06:08 +0000
Subject: [issue27200] make doctest in CPython has failures
In-Reply-To: <1464988408.0.0.840317371896.issue27200@psf.upfronthosting.co.za>
Message-ID: <1483527968.46.0.480800815127.issue27200@psf.upfronthosting.co.za>


Marco Buttu added the comment:

Thinking a little bit about it, I believe that the easiest way to proceed is to make one patch for every file or two, with the only purpose to make their doctests to pass. Otherwise if I make a patch with all the doctest directives, or just with the +SKIP options, we will have hundreds of failures, and it will be really hard to understand what is going wrong, also because the tests are not isolated, and we will have some random failures. Reducing the number of failures by making a patch with all the doctest/testsetup/testcleanup directives is not a solution, because we eventually fall again into a big patch, and the doctests are still not pass too. 

That is why I think having one patch for two files will reduce a lot the effort in making and reviewing the patch. We will end up in about 9 patches, and it will be really easy to review most of them (I think couple of minutes). It will be also easy to test them. For instance, if the patch involves the files Doc/library/copyreg.rst and Doc/library/ctypes.rst, we just have to to to the Doc/ directory and run: 

  $ sphinx-build -b doctest . build/doctest \
  > library/copyreg.rst \
  > library/ctypes.rst

expecting 0 failures. 

Maybe having one new issue for each patch will also simplify the process, because the purpose of the patch will be well explained by the issue description and title. Let me know what you think, and thank you very much for your time.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue27200>
_______________________________________

From report at bugs.python.org  Wed Jan  4 06:26:34 2017
From: report at bugs.python.org (zeroinside)
Date: Wed, 04 Jan 2017 11:26:34 +0000
Subject: [issue29150] Bad cast@ _mysql_ResultObject_Initialize() result in
 code execution
Message-ID: <1483529194.23.0.906683794217.issue29150@psf.upfronthosting.co.za>


New submission from zeroinside:

Hello.
I found a vulnerability in _mysql module.
PoC below:
#!/usr/bin/python2.7
import _mysql
RDX=0x66666666
payload="A"*2048
_mysql.result(payload,RDX)


It's exploitable bug, I'm working on exploit.
(gdb) run mysql.py 
Starting program: /usr/bin/python2.7 mysql.py
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/usr/lib/libthread_db.so.1".

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff38e7f3c in mysql_use_result () from /usr/lib/libmysqlclient.so.18
(gdb) info reg
rax            0x4141414141414141       4702111234474983745
rbx            0x7ffff7e91b90   140737352637328
rcx            0x0      0
rdx            0x66666666       1717986918
rsi            0x7ffff7eb1ec0   140737352769216
rdi            0x5555557f9890   93824995006608
rbp            0x7fffffffe120   0x7fffffffe120
rsp            0x7fffffffe0a8   0x7fffffffe0a8
r8             0x7fffffffdd00   140737488346368
r9             0x7fffffffdd80   140737488346496
r10            0x5555557824f0   93824994518256
r11            0x2      2
r12            0x5555557560a0   93824994336928
r13            0x0      0
r14            0x7ffff7e939c7   140737352645063
r15            0x7ffff7e91b90   140737352637328
rip            0x7ffff38e7f3c   0x7ffff38e7f3c <mysql_use_result+12>
eflags         0x10206  [ PF IF RF ]
cs             0x33     51
ss             0x2b     43
ds             0x0      0
es             0x0      0
fs             0x0      0
gs             0x0      0
(gdb) disas $rip
Dump of assembler code for function mysql_use_result:
   0x00007ffff38e7f30 <+0>:     push   %rbp
   0x00007ffff38e7f31 <+1>:     mov    0x4d0(%rdi),%rax
   0x00007ffff38e7f38 <+8>:     mov    %rsp,%rbp
   0x00007ffff38e7f3b <+11>:    pop    %rbp
=> 0x00007ffff38e7f3c <+12>:    mov    0x18(%rax),%rax
   0x00007ffff38e7f40 <+16>:    jmpq   *%rax
End of assembler dump.
(gdb)

----------
components: Library (Lib)
messages: 284629
nosy: zeroinside
priority: normal
severity: normal
status: open
title: Bad cast@ _mysql_ResultObject_Initialize() result in code execution
type: security
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29150>
_______________________________________

From report at bugs.python.org  Wed Jan  4 06:30:44 2017
From: report at bugs.python.org (Christian Heimes)
Date: Wed, 04 Jan 2017 11:30:44 +0000
Subject: [issue29150] Bad cast@ _mysql_ResultObject_Initialize() result in
 code execution
In-Reply-To: <1483529194.23.0.906683794217.issue29150@psf.upfronthosting.co.za>
Message-ID: <1483529444.21.0.542210003835.issue29150@psf.upfronthosting.co.za>


Christian Heimes added the comment:

mysql is a 3rd party extension and not part of the Python standard library. Please report the issue with MySQL.

----------
nosy: +christian.heimes
resolution:  -> third party
stage:  -> resolved
status: open -> closed
type: security -> crash

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29150>
_______________________________________

From report at bugs.python.org  Wed Jan  4 06:41:12 2017
From: report at bugs.python.org (INADA Naoki)
Date: Wed, 04 Jan 2017 11:41:12 +0000
Subject: [issue28180] sys.getfilesystemencoding() should default to utf-8
In-Reply-To: <1474024622.65.0.838685461995.issue28180@psf.upfronthosting.co.za>
Message-ID: <1483530072.22.0.792368833829.issue28180@psf.upfronthosting.co.za>


INADA Naoki added the comment:

I'm sorry.
I must search old discussion about why we can't simply use utf-8
for fsencoding when C locale, instead of asking here.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28180>
_______________________________________

From report at bugs.python.org  Wed Jan  4 06:55:55 2017
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 04 Jan 2017 11:55:55 +0000
Subject: [issue24773] Implement PEP 495 (Local Time Disambiguation)
In-Reply-To: <1438456647.64.0.191464539374.issue24773@psf.upfronthosting.co.za>
Message-ID: <1483530955.48.0.762491153224.issue24773@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Two bugs were found in the implementation: issue #29100 and issue #29140.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24773>
_______________________________________

From report at bugs.python.org  Wed Jan  4 07:09:11 2017
From: report at bugs.python.org (Tomas Orsava)
Date: Wed, 04 Jan 2017 12:09:11 +0000
Subject: [issue29144] Implicit namespace packages in Python 3.6
In-Reply-To: <1483468021.78.0.379899829208.issue29144@psf.upfronthosting.co.za>
Message-ID: <1483531751.22.0.315268446642.issue29144@psf.upfronthosting.co.za>


Changes by Tomas Orsava <tomas.n at orsava.cz>:


----------
nosy: +torsava

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29144>
_______________________________________

From report at bugs.python.org  Wed Jan  4 07:22:05 2017
From: report at bugs.python.org (patrila)
Date: Wed, 04 Jan 2017 12:22:05 +0000
Subject: [issue29151] test_asyncore.TestAPI_UseIPv4Select /
 test_asyncore.TestAPI_UseIPv6Select fails test_handle_close_after_conn_broken
Message-ID: <1483532525.73.0.454850752566.issue29151@psf.upfronthosting.co.za>


New submission from patrila:

Dear Python developers

python-2.7.12, python-3.4.5 and the current "default" branch fail the tests

* test.test_asyncore.TestAPI_UseIPv4Select.test_handle_close_after_conn_broken
* test.test_asyncore.TestAPI_UseIPv6Select.test_handle_close_after_conn_broken

(difference is only if IPv4 or IPv6 is used). The output of the test-suite is

$ ./python -m unittest test.test_asyncore.TestAPI_UseIPv4Select.test_handle_close_after_conn_broken test.test_asyncore.TestAPI_UseIPv6Select.test_handle_close_after_conn_broken
FF
======================================================================
FAIL: test_handle_close_after_conn_broken (test.test_asyncore.TestAPI_UseIPv4Select)
----------------------------------------------------------------------
Traceback (most recent call last):
  File ".../Lib/test/test_asyncore.py", line 648, in test_handle_close_after_conn_broken
    self.loop_waiting_for_flag(client)
  File ".../Lib/test/test_asyncore.py", line 511, in loop_waiting_for_flag
    self.fail("flag not set")
AssertionError: flag not set

======================================================================
FAIL: test_handle_close_after_conn_broken (test.test_asyncore.TestAPI_UseIPv6Select)
----------------------------------------------------------------------
Traceback (most recent call last):
  File ".../Lib/test/test_asyncore.py", line 648, in test_handle_close_after_conn_broken
    self.loop_waiting_for_flag(client)
  File ".../Lib/test/test_asyncore.py", line 511, in loop_waiting_for_flag
    self.fail("flag not set")
AssertionError: flag not set

----------------------------------------------------------------------
Ran 2 tests in 10.000s

FAILED (failures=2)


However, I also encounter sometimes a success (for both IPv6 and also IPv4). Therefore, I re-run the tests 100 times, this was the result:

success rate of IPv4 test: 2/100
success rate of IPv6 test: 4/100

I conjecture that there is some kind of race condition behind the scenes which sometimes is "won" but more often lost.

I'm running this test on Linux 4.8. I also did find nothing in the system logs related to the issue.

----------
messages: 284633
nosy: patrila
priority: normal
severity: normal
status: open
title: test_asyncore.TestAPI_UseIPv4Select / test_asyncore.TestAPI_UseIPv6Select fails test_handle_close_after_conn_broken
versions: Python 2.7, Python 3.5, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29151>
_______________________________________

From report at bugs.python.org  Wed Jan  4 07:27:45 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Wed, 04 Jan 2017 12:27:45 +0000
Subject: [issue17301] An in-place version of many bytearray methods is needed
In-Reply-To: <1361910523.06.0.491264384267.issue17301@psf.upfronthosting.co.za>
Message-ID: <1483532865.54.0.787647906852.issue17301@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

Python interpreter itself takes 20-30 MB of memory. Using it on a machine that has no few tens of free memory doesn't make much sense. MicroPython can be an exception, but it has special modules for low-level programming.

If you need to proceed so large bytearray that creating yet one copy of it is not possible, it can be translated by separate elements or smaller chunks.

$ ./python -m perf timeit -s "table = bytes(range(256)).swapcase(); data = bytearray(range(256))*1000" -- "for i in range(len(data)): data[i] = table[data[i]]"
Median +- std dev: 205 ms +- 10 ms
$ ./python -m perf timeit -s "table = bytes(range(256)).swapcase(); data = bytearray(range(256))*1000" -- "for i in range(0, len(data), 100): data[i:i+100] = data[i:i+100].translate(table)"
Median +- std dev: 12.9 ms +- 0.6 ms
$ ./python -m perf timeit -s "table = bytes(range(256)).swapcase(); data = bytearray(range(256))*1000" -- "for i in range(0, len(data), 1000): data[i:i+1000] = data[i:i+1000].translate(table)"
Median +- std dev: 3.12 ms +- 0.22 ms
$ ./python -m perf timeit -s "table = bytes(range(256)).swapcase(); data = bytearray(range(256))*1000" -- "for i in range(0, len(data), 10000): data[i:i+10000] = data[i:i+10000].translate(table)"
Median +- std dev: 1.79 ms +- 0.14 ms

Translating by chunks also works with data that can't be fit in memory at all. You can mmap large file or read and write it by blocks. In-place bytearray method wouldn't help in this case.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue17301>
_______________________________________

From report at bugs.python.org  Wed Jan  4 07:32:36 2017
From: report at bugs.python.org (R. David Murray)
Date: Wed, 04 Jan 2017 12:32:36 +0000
Subject: [issue29149] SSL.py recursion limit crash
In-Reply-To: <1483509907.61.0.802720332804.issue29149@psf.upfronthosting.co.za>
Message-ID: <1483533156.01.0.424509796459.issue29149@psf.upfronthosting.co.za>


Changes by R. David Murray <rdmurray at bitdance.com>:


----------
stage:  -> resolved
status: open -> closed
type: crash -> behavior

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29149>
_______________________________________

From report at bugs.python.org  Wed Jan  4 07:57:35 2017
From: report at bugs.python.org (kristall)
Date: Wed, 04 Jan 2017 12:57:35 +0000
Subject: [issue29152] unittest subTest does not call addFailure
Message-ID: <1483534655.28.0.0626841658388.issue29152@psf.upfronthosting.co.za>


New submission from kristall:

unittests subTests do not call addFailure in the case of a failure.

Please see http://stackoverflow.com/questions/41432353/python3-4-unittest-addfailure-not-called-when-subtests-are-used for detailed problemdescription.

----------
components: Tests
messages: 284635
nosy: kristall
priority: normal
severity: normal
status: open
title: unittest subTest does not call addFailure
versions: Python 3.4, Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29152>
_______________________________________

From report at bugs.python.org  Wed Jan  4 08:50:54 2017
From: report at bugs.python.org (patrila)
Date: Wed, 04 Jan 2017 13:50:54 +0000
Subject: [issue29153] Test test.test_asynchat.TestAsynchat /
 test.test_asynchat.TestAsynchat_WithPoll fail test_close_when_done
Message-ID: <1483537854.64.0.339355431872.issue29153@psf.upfronthosting.co.za>


New submission from patrila:

Dear Python developers

The tests

    test.test_asynchat.TestAsynchat.test_close_when_done
    test.test_asynchat.TestAsynchat_WitrhPoll.test_close_when_done

fail with

======================================================================
FAIL: test_close_when_done (test.test_asynchat.TestAsynchat)
----------------------------------------------------------------------
Traceback (most recent call last):
  File ".../Lib/test/test_asynchat.py", line 256, in test_close_when_done
    self.assertGreater(len(s.buffer), 0)
AssertionError: 0 not greater than 0

======================================================================
FAIL: test_close_when_done (test.test_asynchat.TestAsynchat_WithPoll)
----------------------------------------------------------------------
Traceback (most recent call last):
  File ".../Lib/test/test_asynchat.py", line 256, in test_close_when_done
    self.assertGreater(len(s.buffer), 0)
AssertionError: 0 not greater than 0

Looking in the source code we see

        self.assertEqual(c.contents, [])
        # the server might have been able to send a byte or two back, but this
        # at least checks that it received something and didn't just fail
        # (which could still result in the client not having received anything)
        self.assertGreater(len(s.buffer), 0)

We therefore conjecture that the server was able to send all data back.
The relevant part is around line 61, the run() method of echo_server.

                # this may fail on some tests, such as test_close_when_done,
                # since the client closes the channel when it's done sending
                while self.buffer:
                    n = conn.send(self.buffer[:self.chunk_size])
                    time.sleep(0.001)
                    self.buffer = self.buffer[n:]


Indeed, if we change the sleeping time to something larger, e.g. 1sec. The tests pass fine.

In general it is a bad habit to relay on "hardware to be slow enough" to work/pass.
I therefore propose to introduce a separate field in echo_server which is set to True if the server has received some data.
Patch is attached and also included below. It is tested against Python-3.6.0 (sorry no "default" branch).
Python 3.5.3rc1 also works, but the patch below has an offset of -1 (the import warnings line was added/removed depending on the point of view).
For Python 2.7.13 the "same patch" also works but needs different line numbers and also different context. Patch is attached.

Patch for 3.6.0:

--- a/Lib/test/test_asynchat.py
+++ b/Lib/test/test_asynchat.py
@@ -42,12 +42,15 @@
             self.event.set()
             conn, client = self.sock.accept()
             self.buffer = b"" 
+            self.received_something = False
             # collect data until quit message is seen
             while SERVER_QUIT not in self.buffer:
                 data = conn.recv(1)
                 if not data:
                     break
                 self.buffer = self.buffer + data
+            if self.buffer:
+                self.received_something = True

             # remove the SERVER_QUIT message
             self.buffer = self.buffer.replace(SERVER_QUIT, b'')
@@ -252,7 +255,7 @@
         # the server might have been able to send a byte or two back, but this
         # at least checks that it received something and didn't just fail
         # (which could still result in the client not having received anything)
-        self.assertGreater(len(s.buffer), 0)
+        self.assertTrue(s.received_something)

     def test_push(self):
         # Issue #12523: push() should raise a TypeError if it doesn't get

----------
files: test_asynchat_add_receive_something_flag_3.6.patch
keywords: patch
messages: 284636
nosy: patrila
priority: normal
severity: normal
status: open
title: Test test.test_asynchat.TestAsynchat / test.test_asynchat.TestAsynchat_WithPoll fail test_close_when_done
versions: Python 2.7, Python 3.5, Python 3.6, Python 3.7
Added file: http://bugs.python.org/file46138/test_asynchat_add_receive_something_flag_3.6.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29153>
_______________________________________

From report at bugs.python.org  Wed Jan  4 08:52:06 2017
From: report at bugs.python.org (patrila)
Date: Wed, 04 Jan 2017 13:52:06 +0000
Subject: [issue29153] Test test.test_asynchat.TestAsynchat /
 test.test_asynchat.TestAsynchat_WithPoll fail test_close_when_done
In-Reply-To: <1483537854.64.0.339355431872.issue29153@psf.upfronthosting.co.za>
Message-ID: <1483537926.68.0.817611500487.issue29153@psf.upfronthosting.co.za>


Changes by patrila <patrila at fake-box.com>:


Added file: http://bugs.python.org/file46139/test_asynchat_add_receive_something_flag_2.7.13.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29153>
_______________________________________

From report at bugs.python.org  Wed Jan  4 09:04:23 2017
From: report at bugs.python.org (Jeroen Demeyer)
Date: Wed, 04 Jan 2017 14:04:23 +0000
Subject: [issue5322] Python 2.6 object.__new__ argument calling autodetection
 faulty
In-Reply-To: <1235075665.37.0.301694124748.issue5322@psf.upfronthosting.co.za>
Message-ID: <1483538663.43.0.00363349700947.issue5322@psf.upfronthosting.co.za>


Jeroen Demeyer added the comment:

It worries me that nothing in the Python docs nor in any PEP describes how tp_new is inherited. In my opinion, ?this patch makes a significant change which should be subject to a PEP. However, neither the old nor new behaviour is described anywhere. This also makes it harder to argue which behaviour is correct.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue5322>
_______________________________________

From report at bugs.python.org  Wed Jan  4 09:13:18 2017
From: report at bugs.python.org (jan matejek)
Date: Wed, 04 Jan 2017 14:13:18 +0000
Subject: [issue29145] failing overflow checks in replace_*
In-Reply-To: <1483468602.77.0.231507148809.issue29145@psf.upfronthosting.co.za>
Message-ID: <1483539198.3.0.0421777408641.issue29145@psf.upfronthosting.co.za>


jan matejek added the comment:

It does, but "-fwrapv" is not automatically added when you specify custom OPT flags. I should have clarified that in the original report.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29145>
_______________________________________

From report at bugs.python.org  Wed Jan  4 09:31:02 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Wed, 04 Jan 2017 14:31:02 +0000
Subject: [issue29145] failing overflow checks in replace_*
In-Reply-To: <1483468602.77.0.231507148809.issue29145@psf.upfronthosting.co.za>
Message-ID: <1483540262.63.0.590289499261.issue29145@psf.upfronthosting.co.za>


Xiang Zhang added the comment:

> It does, but "-fwrapv" is not automatically added when you specify custom OPT flags.

Indeed I think this is why the changes in #27473 and #1621 make sense.

> GCC 6 optimizes away broken overflow checks.

I am sorry but I don't understand this. What do you mean by broken overflow checks? Is `if (size > PY_SSIZE_T_MAX - vo.len)` broken?

----------
nosy: +xiang.zhang

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29145>
_______________________________________

From report at bugs.python.org  Wed Jan  4 09:37:55 2017
From: report at bugs.python.org (jan matejek)
Date: Wed, 04 Jan 2017 14:37:55 +0000
Subject: [issue29145] failing overflow checks in replace_*
In-Reply-To: <1483468602.77.0.231507148809.issue29145@psf.upfronthosting.co.za>
Message-ID: <1483540675.04.0.48166761848.issue29145@psf.upfronthosting.co.za>


jan matejek added the comment:

No, your changes from issue 27473 are OK. However functions like replace_interleave and replace_single_character etc. still use the broken code:

/* use the difference between current and new, hence the "-1" */
/*   result_len = self_len + count * (to_len-1)  */
product = count * (to_len-1);
if (product / (to_len-1) != count) {
    PyErr_SetString(PyExc_OverflowError, "replace bytes is too long");
    return NULL;
}

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29145>
_______________________________________

From report at bugs.python.org  Wed Jan  4 09:46:13 2017
From: report at bugs.python.org (Nick Coghlan)
Date: Wed, 04 Jan 2017 14:46:13 +0000
Subject: [issue28180] sys.getfilesystemencoding() should default to utf-8
In-Reply-To: <1474024622.65.0.838685461995.issue28180@psf.upfronthosting.co.za>
Message-ID: <1483541173.33.0.931483633089.issue28180@psf.upfronthosting.co.za>


Nick Coghlan added the comment:

The default encoding in the C/POSIX locale is ASCII (which is the entire source of the problem).

The initial verison of the PEP I uploaded didn't explain that background, but I added a section about it in the update earlier this week: https://www.python.org/dev/peps/pep-0538/#background

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28180>
_______________________________________

From report at bugs.python.org  Wed Jan  4 10:32:49 2017
From: report at bugs.python.org (Gerrit Holl)
Date: Wed, 04 Jan 2017 15:32:49 +0000
Subject: [issue23680] Sporadic freeze in test_interrupted_write_retry_text
In-Reply-To: <1426521207.32.0.580722848653.issue23680@psf.upfronthosting.co.za>
Message-ID: <1483543969.22.0.10620668364.issue23680@psf.upfronthosting.co.za>


Gerrit Holl added the comment:

I experience this problem when trying to build/test Python 3.6 on the [JASMIN Analysis Platform](http://www.jasmin.ac.uk/services/jasmin-analysis-platform/) which runs Red Hat Enterprise Linux Server release 6.8 on a machine with 48 ? Intel(R) Xeon(R) CPU E7-4860 v2 @ 2.60GHz, 2 TiB RAM, and a [PanFS?](http://www.panasas.com/products/panfs) distributed file system.  I experience this problem regardless whether I build/test Python 3.6 from within a PanFS filesystem or otherwise.

./python -m test -v test_io
== CPython 3.6.0 (default, Jan 4 2017, 15:08:07) [GCC 4.4.7 20120313 (Red Hat 4.4.7-17)]
==   Linux-2.6.32-642.6.2.el6.x86_64-x86_64-with-redhat-6.8-Santiago little-endian
==   hash algorithm: siphash24 64bit
==  cwd: /dev/shm/gerrit/Python-3.6.0/build/test_python_34342
==  encodings: locale=UTF-8, FS=utf-8
Testing with flags: sys.flags(debug=0, inspect=0, interactive=0, optimize=0, dont_write_bytecode=0, no_user_site=0, no_site=0, ignore_environment=0, verbose=0, bytes_warning=0, quiet=0, hash_randomization=1, isolated=0)
Run tests sequentially
0:00:00 [1/1] test_io
[?snip many lines?]
test_interrupted_write_retry_text (test.test_io.CSignalsTest) ...

after which there is no more output.

----------
nosy: +Gerrit.Holl

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23680>
_______________________________________

From report at bugs.python.org  Wed Jan  4 10:48:48 2017
From: report at bugs.python.org (Gerrit Holl)
Date: Wed, 04 Jan 2017 15:48:48 +0000
Subject: [issue29154] 5 failures in test_doctest: ModuleNotFoundError: No
 module named 'IPython'
Message-ID: <1483544927.48.0.740005145144.issue29154@psf.upfronthosting.co.za>


New submission from Gerrit Holl:

I am building and testing Python 3.6 on the JASMIN Analysis Platform <http://www.jasmin.ac.uk/services/jasmin-analysis-platform/>, which runs Red Hat Enterprise Linux Server release 6.8 on a machine with 48 ? Intel(R) Xeon(R) CPU E7-4860 v2 @ 2.60GHz, 2 TiB RAM, and a PanFS? <http://www.panasas.com/products/panfs> distributed file system.  I am experiencing failures in test_doctest, specifically with `ModuleNotFoundError: No module named 'IPython'`.  I don't know why it comes up with IPython at all.

Below is the non-verbose test output.  Attached is the verbose test output.


$ ./python -m test test_doctest
Run tests sequentially
0:00:00 [1/1] test_doctest
**********************************************************************
File "/dev/shm/gerrit/Python-3.6.0/Lib/test/test_doctest.py", line 1841, in test.test_doctest.test_debug
Failed example:
    try: doctest.debug_src(s)
    finally: sys.stdin = real_stdin
Expected:
    > <string>(1)<module>()
    (Pdb) next
    12
    --Return--
    > <string>(1)<module>()->None
    (Pdb) print(x)
    12
    (Pdb) continue
Got:
    *** ModuleNotFoundError: No module named 'IPython'
    > <string>(1)<module>()
    (Pdb) next
    12
    --Return--
    > <string>(1)<module>()->None
    (Pdb) print(x)
    12
    (Pdb) continue
**********************************************************************
File "/dev/shm/gerrit/Python-3.6.0/Lib/test/test_doctest.py", line 1885, in test.test_doctest.test_pdb_set_trace
Failed example:
    try: runner.run(test)
    finally: sys.stdin = real_stdin
Expected:
    --Return--
    > <doctest foo-bar at baz[2]>(1)<module>()->None
    -> import pdb; pdb.set_trace()
    (Pdb) print(x)
    42
    (Pdb) continue
    TestResults(failed=0, attempted=3)
Got:
    --Return--
    *** ModuleNotFoundError: No module named 'IPython'
    > <doctest foo-bar at baz[2]>(1)<module>()->None
    -> import pdb; pdb.set_trace()
    (Pdb) print(x)
    42
    (Pdb) continue
    TestResults(failed=0, attempted=3)
**********************************************************************
File "/dev/shm/gerrit/Python-3.6.0/Lib/test/test_doctest.py", line 1914, in test.test_doctest.test_pdb_set_trace
Failed example:
    try:
        runner.run(test)
    finally:
        sys.stdin = real_stdin
Expected:
    --Return--
    > <doctest test.test_doctest.test_pdb_set_trace[7]>(3)calls_set_trace()->None
    -> import pdb; pdb.set_trace()
    (Pdb) print(y)
    2
    (Pdb) up
    > <doctest foo-bar at baz[1]>(1)<module>()
    -> calls_set_trace()
    (Pdb) print(x)
    1
    (Pdb) continue
    TestResults(failed=0, attempted=2)
Got:
    --Return--
    *** ModuleNotFoundError: No module named 'IPython'
    > <doctest test.test_doctest.test_pdb_set_trace[7]>(3)calls_set_trace()->None
    -> import pdb; pdb.set_trace()
    (Pdb) print(y)
    2
    (Pdb) up
    > <doctest foo-bar at baz[1]>(1)<module>()
    -> calls_set_trace()
    (Pdb) print(x)
    1
    (Pdb) continue
    TestResults(failed=0, attempted=2)
**********************************************************************
File "/dev/shm/gerrit/Python-3.6.0/Lib/test/test_doctest.py", line 1952, in test.test_doctest.test_pdb_set_trace
Failed example:
    try: runner.run(test)
    finally: sys.stdin = real_stdin
    # doctest: +NORMALIZE_WHITESPACE
Expected:
    --Return--
    > <doctest foo-bar at baz[1]>(3)g()->None
    -> import pdb; pdb.set_trace()
    (Pdb) list
      1     def g(x):
      2         print(x+3)
      3  ->     import pdb; pdb.set_trace()
    [EOF]
    (Pdb) next
    --Return--
    > <doctest foo-bar at baz[0]>(2)f()->None
    -> g(x*2)
    (Pdb) list
      1     def f(x):
      2  ->     g(x*2)
    [EOF]
    (Pdb) next
    --Return--
    > <doctest foo-bar at baz[2]>(1)<module>()->None
    -> f(3)
    (Pdb) list
      1  -> f(3)
    [EOF]
    (Pdb) continue
    **********************************************************************
    File "foo-bar at baz.py", line 7, in foo-bar at baz
    Failed example:
        f(3)
    Expected nothing
    Got:
        9
    TestResults(failed=1, attempted=3)
Got:
    --Return--
    *** ModuleNotFoundError: No module named 'IPython'
    > <doctest foo-bar at baz[1]>(3)g()->None
    -> import pdb; pdb.set_trace()
    (Pdb) list
      1         def g(x):
      2             print(x+3)
      3  ->         import pdb; pdb.set_trace()
    [EOF]
    (Pdb) next
    --Return--
    > <doctest foo-bar at baz[0]>(2)f()->None
    -> g(x*2)
    (Pdb) list
      1         def f(x):
      2  ->         g(x*2)
    [EOF]
    (Pdb) next
    --Return--
    > <doctest foo-bar at baz[2]>(1)<module>()->None
    -> f(3)
    (Pdb) list
      1  ->     f(3)
    [EOF]
    (Pdb) continue
    **********************************************************************
    File "foo-bar at baz.py", line 7, in foo-bar at baz
    Failed example:
        f(3)
    Expected nothing
    Got:
        9
    TestResults(failed=1, attempted=3)
File "/dev/shm/gerrit/Python-3.6.0/Lib/test/test_doctest.py", line 2025, in test.test_doctest.test_pdb_set_trace_nested
Failed example:
    try:
        runner.run(test)
    finally:
        sys.stdin = real_stdin
    # doctest: +REPORT_NDIFF
Differences (ndiff with -expected +actual):
    + *** ModuleNotFoundError: No module named 'IPython'
      > <doctest test.test_doctest.test_pdb_set_trace_nested[0]>(5)calls_set_trace()
      -> self.f1()
      (Pdb) print(y)
      1
      (Pdb) step
      --Call--
      > <doctest test.test_doctest.test_pdb_set_trace_nested[0]>(7)f1()
      -> def f1(self):
      (Pdb) step
      > <doctest test.test_doctest.test_pdb_set_trace_nested[0]>(8)f1()
      -> x = 1
      (Pdb) step
      > <doctest test.test_doctest.test_pdb_set_trace_nested[0]>(9)f1()
      -> self.f2()
      (Pdb) step
      --Call--
      > <doctest test.test_doctest.test_pdb_set_trace_nested[0]>(11)f2()
      -> def f2(self):
      (Pdb) step
      > <doctest test.test_doctest.test_pdb_set_trace_nested[0]>(12)f2()
      -> z = 1
      (Pdb) step
      > <doctest test.test_doctest.test_pdb_set_trace_nested[0]>(13)f2()
      -> z = 2
      (Pdb) print(z)
      1
      (Pdb) up
      > <doctest test.test_doctest.test_pdb_set_trace_nested[0]>(9)f1()
      -> self.f2()
      (Pdb) print(x)
      1
      (Pdb) up
      > <doctest test.test_doctest.test_pdb_set_trace_nested[0]>(5)calls_set_trace()
      -> self.f1()
      (Pdb) print(y)
      1
      (Pdb) up
      > <doctest foo-bar at baz[1]>(1)<module>()
      -> calls_set_trace()
      (Pdb) print(foo)
      *** NameError: name 'foo' is not defined
      (Pdb) continue
      TestResults(failed=0, attempted=2)
**********************************************************************
3 items had failures:
   1 of   4 in test.test_doctest.test_debug
   3 of  18 in test.test_doctest.test_pdb_set_trace
   1 of   9 in test.test_doctest.test_pdb_set_trace_nested
***Test Failed*** 5 failures.
test test_doctest failed -- 5 of 515 doctests failed
test_doctest failed

1 test failed:
    test_doctest

Total duration: 3 sec
Tests result: FAILURE

----------
files: test_doctest_failure
messages: 284643
nosy: Gerrit.Holl
priority: normal
severity: normal
status: open
title: 5 failures in test_doctest: ModuleNotFoundError: No module named 'IPython'
versions: Python 3.6
Added file: http://bugs.python.org/file46140/test_doctest_failure

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29154>
_______________________________________

From report at bugs.python.org  Wed Jan  4 10:56:37 2017
From: report at bugs.python.org (Kyle Altendorf)
Date: Wed, 04 Jan 2017 15:56:37 +0000
Subject: [issue29134] contextlib doc bug
In-Reply-To: <1483376051.72.0.678487921628.issue29134@psf.upfronthosting.co.za>
Message-ID: <1483545397.35.0.595536295683.issue29134@psf.upfronthosting.co.za>


Kyle Altendorf added the comment:

I would think that the idea of simply adding some reference to `User` such as was suggested with `UserDefinedContextClass` would be reasonable and helpful.  This doesn't involve any more code merely a more explanatory name.

----------
nosy: +altendky

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29134>
_______________________________________

From report at bugs.python.org  Wed Jan  4 10:59:56 2017
From: report at bugs.python.org (Roundup Robot)
Date: Wed, 04 Jan 2017 15:59:56 +0000
Subject: [issue28768] Warning: implicit declaration of function '_setmode'
In-Reply-To: <1479758430.88.0.97682867264.issue28768@psf.upfronthosting.co.za>
Message-ID: <20170104155953.5815.11581.F0A28EC2@psf.io>


Roundup Robot added the comment:

New changeset 5ea0fef6ec53 by Steve Dower in branch '2.7':
Issue #28768: Fix implicit declaration of function _setmode. Patch by Masayuki Yamamoto
https://hg.python.org/cpython/rev/5ea0fef6ec53

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28768>
_______________________________________

From report at bugs.python.org  Wed Jan  4 11:00:39 2017
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 04 Jan 2017 16:00:39 +0000
Subject: [issue28768] Warning: implicit declaration of function '_setmode'
In-Reply-To: <1479758430.88.0.97682867264.issue28768@psf.upfronthosting.co.za>
Message-ID: <1483545639.22.0.232411514886.issue28768@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Well, the change is not going to hurt. I backported the change. Thanks for the fix Masayuki!

----------
resolution:  -> fixed
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28768>
_______________________________________

From report at bugs.python.org  Wed Jan  4 11:06:09 2017
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 04 Jan 2017 16:06:09 +0000
Subject: [issue28180] sys.getfilesystemencoding() should default to utf-8
In-Reply-To: <1483541173.33.0.931483633089.issue28180@psf.upfronthosting.co.za>
Message-ID: <CAMpsgwYA3Cs8ofigMC+NoUEJmbi9Gia45CFeD-WdY2mEd501JQ@mail.gmail.com>


STINNER Victor added the comment:

> The default encoding in the C/POSIX locale is ASCII (which is the entire source of the problem).

The reality is more complex than that :-) It depends on the OS.

Some OS uses Latin1 for the POSIX locale. Some OS announces to use
Latin1 for the POSIX locale, but use ASCII in practice :-) On these
lying OS, Python decodes bytes 0x80..0xff using mbstowcs() to check if
we get ASCII or Latin1: see the check_force_ascii() function.

/* Workaround FreeBSD and OpenIndiana locale encoding issue with the C locale.
   On these operating systems, nl_langinfo(CODESET) announces an alias of the
   ASCII encoding, whereas mbstowcs() and wcstombs() functions use the
   ISO-8859-1 encoding. The problem is that os.fsencode() and os.fsdecode() use
   locale.getpreferredencoding() codec. For example, if command line arguments
   are decoded by mbstowcs() and encoded back by os.fsencode(), we get a
   UnicodeEncodeError instead of retrieving the original byte string.

   The workaround is enabled if setlocale(LC_CTYPE, NULL) returns "C",
   nl_langinfo(CODESET) announces "ascii" (or an alias to ASCII), and at least
   one byte in range 0x80-0xff can be decoded from the locale encoding. The
   workaround is also enabled on error, for example if getting the locale
   failed.

    (...) */

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28180>
_______________________________________

From report at bugs.python.org  Wed Jan  4 11:26:10 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Wed, 04 Jan 2017 16:26:10 +0000
Subject: [issue29145] failing overflow checks in replace_*
In-Reply-To: <1483468602.77.0.231507148809.issue29145@psf.upfronthosting.co.za>
Message-ID: <1483547170.62.0.0298745436813.issue29145@psf.upfronthosting.co.za>


Xiang Zhang added the comment:

Ohh sorry I misunderstand your intention. :-( Attach a patch. :-)

----------
keywords: +patch
stage:  -> patch review
Added file: http://bugs.python.org/file46141/replace-overflow-check.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29145>
_______________________________________

From report at bugs.python.org  Wed Jan  4 11:35:53 2017
From: report at bugs.python.org (Gerrit Holl)
Date: Wed, 04 Jan 2017 16:35:53 +0000
Subject: [issue29154] 5 failures in test_doctest: ModuleNotFoundError: No
 module named 'IPython'
In-Reply-To: <1483544927.48.0.740005145144.issue29154@psf.upfronthosting.co.za>
Message-ID: <1483547753.15.0.566830297091.issue29154@psf.upfronthosting.co.za>


Gerrit Holl added the comment:

I get the same failure cause in test_pdb and test_zipimport_support, also in situations with tests based on doctest, again getting an unexpected `*** ModuleNotFoundError: No module named 'IPython'`

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29154>
_______________________________________

From report at bugs.python.org  Wed Jan  4 11:39:03 2017
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 04 Jan 2017 16:39:03 +0000
Subject: [issue29100] datetime.fromtimestamp() doesn't check min/max year
 anymore: regression of Python 3.6
In-Reply-To: <1482972647.78.0.524140289263.issue29100@psf.upfronthosting.co.za>
Message-ID: <1483547943.53.0.977065106468.issue29100@psf.upfronthosting.co.za>


STINNER Victor added the comment:

timestamp_limits.patch: Add many checks in the _datetime module to avoid overflows or creating a object out of the range. The patch adds unit tests.

The unit test checks dates around year 1 and year 9999. I expect failures on platforms which only work well on year in the range 1970..2038. My plan is to wait for buildbot reports to check on which platforms the test fails and with which exception.

Since the patch is likely to break buildbots, I will apply first to the default branch, fix it, and then backport to Python 3.6.

----------
Added file: http://bugs.python.org/file46142/timestamp_limits.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29100>
_______________________________________

From report at bugs.python.org  Wed Jan  4 11:42:46 2017
From: report at bugs.python.org (Gerrit Holl)
Date: Wed, 04 Jan 2017 16:42:46 +0000
Subject: [issue29155] test.test_spwd.TestSpwdNonRoot failure with
 FileNotFoundError:
Message-ID: <1483548166.48.0.314208460749.issue29155@psf.upfronthosting.co.za>


New submission from Gerrit Holl:

I am building and testing Python 3.6 on the JASMIN Analysis Platform <http://www.jasmin.ac.uk/services/jasmin-analysis-platform/>, which runs Red Hat Enterprise Linux Server release 6.8 on a machine with 48 ? Intel(R) Xeon(R) CPU E7-4860 v2 @ 2.60GHz, 2 TiB RAM, and a PanFS? <http://www.panasas.com/products/panfs> distributed file system.  test.test_spwd.TestSpwdNonRoot is failing with FileNotFoundError.  Below is the full verbose output of `./python -m test -v test_spwd`:

== CPython 3.6.0 (default, Jan 4 2017, 14:11:04) [GCC 4.4.7 20120313 (Red Hat 4.4.7-17)]
==   Linux-2.6.32-642.6.2.el6.x86_64-x86_64-with-redhat-6.8-Santiago little-endian
==   hash algorithm: siphash24 64bit
==  cwd: /home/users/gholl/src/Python-3.6.0/build/test_python_40357
==  encodings: locale=UTF-8, FS=utf-8
Testing with flags: sys.flags(debug=0, inspect=0, interactive=0, optimize=0, dont_write_bytecode=0, no_user_site=0, no_site=0, ignore_environment=0, verbose=0, bytes_warning=0, quiet=0, hash_randomization=1, isolated=0)
Run tests sequentially
0:00:00 [1/1] test_spwd
test_getspnam_exception (test.test_spwd.TestSpwdNonRoot) ... ERROR
test_getspall (test.test_spwd.TestSpwdRoot) ... skipped 'root privileges required'
test_getspnam (test.test_spwd.TestSpwdRoot) ... skipped 'root privileges required'

======================================================================
ERROR: test_getspnam_exception (test.test_spwd.TestSpwdNonRoot)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/users/gholl/src/Python-3.6.0/Lib/test/test_spwd.py", line 67, in test_getspnam_exception
    spwd.getspnam(name)
FileNotFoundError: [Errno 2] No such file or directory

----------------------------------------------------------------------
Ran 3 tests in 0.002s

FAILED (errors=1, skipped=2)
test_spwd failed

1 test failed:
    test_spwd

Total duration: 66 ms
Tests result: FAILURE

----------
messages: 284651
nosy: Gerrit.Holl
priority: normal
severity: normal
status: open
title: test.test_spwd.TestSpwdNonRoot  failure with FileNotFoundError:

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29155>
_______________________________________

From report at bugs.python.org  Wed Jan  4 11:42:51 2017
From: report at bugs.python.org (Gerrit Holl)
Date: Wed, 04 Jan 2017 16:42:51 +0000
Subject: [issue29155] test.test_spwd.TestSpwdNonRoot failure with
 FileNotFoundError:
In-Reply-To: <1483548166.48.0.314208460749.issue29155@psf.upfronthosting.co.za>
Message-ID: <1483548171.4.0.103780837709.issue29155@psf.upfronthosting.co.za>


Changes by Gerrit Holl <gerrit.holl at gmail.com>:


----------
versions: +Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29155>
_______________________________________

From report at bugs.python.org  Wed Jan  4 11:45:56 2017
From: report at bugs.python.org (Lukas Schwaighofer)
Date: Wed, 04 Jan 2017 16:45:56 +0000
Subject: [issue29156] remove superfluous pow test for (nonexistent) long
Message-ID: <1483548356.52.0.249977190995.issue29156@psf.upfronthosting.co.za>


New submission from Lukas Schwaighofer:

Since python 3 there is no longer a type long. There is still a legacy test formerly used for longs present in test_pow.py. As the same test is already present (as also visible at the beginning of the patch) the test can safely be removed.

----------
components: Tests
files: test_pow.patch
keywords: patch
messages: 284652
nosy: lukas.schwaighofer
priority: normal
severity: normal
status: open
title: remove superfluous pow test for (nonexistent) long
type: enhancement
versions: Python 3.7
Added file: http://bugs.python.org/file46143/test_pow.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29156>
_______________________________________

From report at bugs.python.org  Wed Jan  4 11:50:17 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Wed, 04 Jan 2017 16:50:17 +0000
Subject: [issue29156] remove superfluous pow test for (nonexistent) long
In-Reply-To: <1483548356.52.0.249977190995.issue29156@psf.upfronthosting.co.za>
Message-ID: <1483548617.59.0.647991537731.issue29156@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

LGTM. Thank you for your patch Lukas.

----------
assignee:  -> serhiy.storchaka
nosy: +serhiy.storchaka
stage:  -> commit review

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29156>
_______________________________________

From report at bugs.python.org  Wed Jan  4 11:53:19 2017
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 04 Jan 2017 16:53:19 +0000
Subject: [issue29156] remove superfluous pow test for (nonexistent) long
In-Reply-To: <1483548356.52.0.249977190995.issue29156@psf.upfronthosting.co.za>
Message-ID: <1483548799.24.0.579636194322.issue29156@psf.upfronthosting.co.za>


STINNER Victor added the comment:

LGTM.

----------
nosy: +haypo

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29156>
_______________________________________

From report at bugs.python.org  Wed Jan  4 11:54:35 2017
From: report at bugs.python.org (Roundup Robot)
Date: Wed, 04 Jan 2017 16:54:35 +0000
Subject: [issue29156] remove superfluous pow test for (nonexistent) long
In-Reply-To: <1483548356.52.0.249977190995.issue29156@psf.upfronthosting.co.za>
Message-ID: <20170104165431.54949.82512.EA44C614@psf.io>


Roundup Robot added the comment:

New changeset f159ab8a9360 by Serhiy Storchaka in branch '3.5':
Issue #29156: Remove superfluous pow test.
https://hg.python.org/cpython/rev/f159ab8a9360

New changeset ef3440585a17 by Serhiy Storchaka in branch '3.6':
Issue #29156: Remove superfluous pow test.
https://hg.python.org/cpython/rev/ef3440585a17

New changeset 29d28f1bdc7c by Serhiy Storchaka in branch 'default':
Issue #29156: Remove superfluous pow test.
https://hg.python.org/cpython/rev/29d28f1bdc7c

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29156>
_______________________________________

From report at bugs.python.org  Wed Jan  4 11:55:13 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Wed, 04 Jan 2017 16:55:13 +0000
Subject: [issue29156] remove superfluous pow test for (nonexistent) long
In-Reply-To: <1483548356.52.0.249977190995.issue29156@psf.upfronthosting.co.za>
Message-ID: <1483548913.01.0.237145015984.issue29156@psf.upfronthosting.co.za>


Changes by Serhiy Storchaka <storchaka+cpython at gmail.com>:


----------
resolution:  -> fixed
stage: commit review -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29156>
_______________________________________

From report at bugs.python.org  Wed Jan  4 11:59:20 2017
From: report at bugs.python.org (R. David Murray)
Date: Wed, 04 Jan 2017 16:59:20 +0000
Subject: [issue29155] test.test_spwd.TestSpwdNonRoot failure with
 FileNotFoundError:
In-Reply-To: <1483548166.48.0.314208460749.issue29155@psf.upfronthosting.co.za>
Message-ID: <1483549160.06.0.976445789137.issue29155@psf.upfronthosting.co.za>


R. David Murray added the comment:

Since you are seeing errors on your platform that we are not seeing, and we do not have access to your platform, you are going to have to track down the cause, I'm afraid.

----------
nosy: +r.david.murray

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29155>
_______________________________________

From report at bugs.python.org  Wed Jan  4 12:06:50 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Wed, 04 Jan 2017 17:06:50 +0000
Subject: [issue29155] test.test_spwd.TestSpwdNonRoot failure with
 FileNotFoundError:
In-Reply-To: <1483548166.48.0.314208460749.issue29155@psf.upfronthosting.co.za>
Message-ID: <1483549610.25.0.835448537508.issue29155@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

It is easy to add FileNotFoundError to the list of expected exceptions. But it seems to me that the only error that gespname() can return on Linux is EACCES. [1] May be this is a bug of your platform.

[1] http://man7.org/linux/man-pages/man3/getspnam.3.html

----------
nosy: +serhiy.storchaka

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29155>
_______________________________________

From report at bugs.python.org  Wed Jan  4 12:17:05 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Wed, 04 Jan 2017 17:17:05 +0000
Subject: [issue29145] failing overflow checks in replace_*
In-Reply-To: <1483468602.77.0.231507148809.issue29145@psf.upfronthosting.co.za>
Message-ID: <1483550225.69.0.601181690981.issue29145@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

It would be better to write the code in the same form as in 3.x. This could help backporting other patches if needed.

----------
nosy: +serhiy.storchaka

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29145>
_______________________________________

From report at bugs.python.org  Wed Jan  4 12:33:01 2017
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 04 Jan 2017 17:33:01 +0000
Subject: [issue29157] random.c: Prefer getrandom() over getentropy(),
 handle ENOSYS in py_getentropy()
Message-ID: <1483551181.22.0.980001779001.issue29157@psf.upfronthosting.co.za>


New submission from STINNER Victor:

A new getentropy() function was recently added to the glibc:
https://sourceware.org/bugzilla/show_bug.cgi?id=17252

When the Python/random.c file was written (by me), the getentropy() function was only supported on OpenBSD. Later, random.c was modified to *not* use getentropy() on Solaris (Issue #25003).

The problem is that py_getentropy() doesn't handle ENOSYS, and so Python fails at startup with a fatal error (Python 3.6):
   Fatal Python error: failed to get random numbers to initialize Python
or (Python 3.5):
   Fatal Python error: getentropy() failed

The bug was first reported in Fedora 26 (rawhide):
https://bugzilla.redhat.com/show_bug.cgi?id=1410175

Attached patch (written for the default branch) should fix these issues:

* Prefer getrandom() syscall over getentropy() function: getrandom() supports blocking and non-blocking mode on Linux, whereas getentropy() doesn't
* Enhance py_getentropy() to handle ENOSYS: fallback on reading from /dev/urandom and remember that the function doesn't work

I'm not sure that handling ENOSYS is required, since it's no more used on Linux, but it shouldn't hurt. I don't know if py_getentropy() should also handle EPERM?

py_getrandom() catchs errors: EAGAIN, EINTR, EPERM and ENOSYS.

With the patch, py_getentropy() catchs ENOSYS error.

----------
files: getentropy.patch
keywords: patch
messages: 284659
nosy: christian.heimes, haypo
priority: normal
severity: normal
status: open
title: random.c: Prefer getrandom() over getentropy(), handle ENOSYS in py_getentropy()
type: security
versions: Python 2.7, Python 3.5, Python 3.6, Python 3.7
Added file: http://bugs.python.org/file46144/getentropy.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29157>
_______________________________________

From report at bugs.python.org  Wed Jan  4 12:34:05 2017
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 04 Jan 2017 17:34:05 +0000
Subject: [issue29157] random.c: Prefer getrandom() over getentropy(),
 handle ENOSYS in py_getentropy()
In-Reply-To: <1483551181.22.0.980001779001.issue29157@psf.upfronthosting.co.za>
Message-ID: <1483551245.32.0.831197580324.issue29157@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Python 2.7, 3.5, 3.6 and 3.7 are impacted: they should fail on Linux if compiled with a recent glibc which has getentropy().

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29157>
_______________________________________

From report at bugs.python.org  Wed Jan  4 14:19:03 2017
From: report at bugs.python.org (Luciano Dionisio)
Date: Wed, 04 Jan 2017 19:19:03 +0000
Subject: [issue29158] Possible glitch in the interaction of a thread and a
 multiprocessing manager
Message-ID: <1483557543.41.0.26156873729.issue29158@psf.upfronthosting.co.za>


New submission from Luciano Dionisio:

After spending a lot of time trying to understand why my code will not execute as expected and after not getting any help from StackOverflow:

http://stackoverflow.com/questions/41444081/python-multiprocessing-manager-delegate-client-connection-and-dont-wait

I assumed that it may be a possible glitch in the interaction of a thread and a multiprocessing manager.

I have tried this in Python 2.7.13 and Python 3.6.0 and assume the problem still exists in between and beyond.

The problem appears when I try to delegate the connect() method of a multiprocessing manager client to a thread. The first time the procedure takes place, everything works fine and there is no problem of shared memory or anything. The problem arises on the second and forth trials of connect, when there seems to be a memory sharing problem. To reproduce the problem you have to run the server.py and client.py modules. You can see that the client is capable of populating the server's queue. If you terminate the server.py process and start it again, the client can no longer reassign the remote queue. Actually, reconnecting to the server always take place, as well as the correct linkage to the remote queue on line 53 of bridge.py:

self.items_buffer = self.remote_manager.items_buffer()

but the problem is that this procedure no longer works after the first time. Even though the connection is re-established, and at the moment of reconnection it is possible to send info to the server, whenever the thread dies, the pipe gets somehow broken.

----------
components: Interpreter Core
files: case.zip
messages: 284661
nosy: Luciano Dionisio
priority: normal
severity: normal
status: open
title: Possible glitch in the interaction of a thread and a multiprocessing manager
type: crash
versions: Python 2.7, Python 3.6
Added file: http://bugs.python.org/file46145/case.zip

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29158>
_______________________________________

From report at bugs.python.org  Wed Jan  4 14:27:23 2017
From: report at bugs.python.org (R. David Murray)
Date: Wed, 04 Jan 2017 19:27:23 +0000
Subject: [issue29158] Possible glitch in the interaction of a thread and a
 multiprocessing manager
In-Reply-To: <1483557543.41.0.26156873729.issue29158@psf.upfronthosting.co.za>
Message-ID: <1483558043.61.0.201236066764.issue29158@psf.upfronthosting.co.za>


R. David Murray added the comment:

My understanding is that the basic rule of thumb is: don't mix threads and multiprocessing.  You may find that if you use spawn, it won't ever work.  But I haven't used multiprocessing myself.

----------
nosy: +r.david.murray
type: crash -> behavior

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29158>
_______________________________________

From report at bugs.python.org  Wed Jan  4 14:49:54 2017
From: report at bugs.python.org (Alexander Belopolsky)
Date: Wed, 04 Jan 2017 19:49:54 +0000
Subject: [issue29159] Regression in bytes constructor
Message-ID: <1483559394.32.0.335371111214.issue29159@psf.upfronthosting.co.za>


New submission from Alexander Belopolsky:

Consider the following code:

$ cat bug.py
from array import array
class C(array):
    def __new__(cls):
        return array.__new__(cls, 'B', b'abc')
    def __index__(self):
        raise TypeError
x = C()
print(bytes(x))

It works under python 3.5:

$ python3.5 bug.py
b'abc'

but raises a TypeError under python 3.6:

$ python3.6 bug.py
Traceback (most recent call last):
  File "bug.py", line 8, in <module>
    print(bytes(x))
  File "bug.py", line 6, in __index__
    raise TypeError
TypeError

It looks like this was introduced in issue #27704.

(Ref: e/pyq#827)

----------
keywords: 3.6regression
messages: 284663
nosy: belopolsky, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Regression in bytes constructor
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29159>
_______________________________________

From report at bugs.python.org  Wed Jan  4 15:12:51 2017
From: report at bugs.python.org (Alexander Belopolsky)
Date: Wed, 04 Jan 2017 20:12:51 +0000
Subject: [issue29159] Regression in bytes constructor
In-Reply-To: <1483559394.32.0.335371111214.issue29159@psf.upfronthosting.co.za>
Message-ID: <1483560771.56.0.453820022342.issue29159@psf.upfronthosting.co.za>


Alexander Belopolsky added the comment:

My test code may seem contrived, but numpy arrays exhibit similar behavior:

>>> a = numpy.array([2, 2])
>>> bytes(a)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: only integer arrays with one element can be converted to an index

Under python 3.5, the result was

>>> bytes(a)
b'\x02\x00\x00\x00\x02\x00\x00\x00'

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29159>
_______________________________________

From report at bugs.python.org  Wed Jan  4 15:24:47 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Wed, 04 Jan 2017 20:24:47 +0000
Subject: [issue29159] Regression in bytes constructor
In-Reply-To: <1483559394.32.0.335371111214.issue29159@psf.upfronthosting.co.za>
Message-ID: <1698135.JeCfcheUoZ@raxxla>


Serhiy Storchaka added the comment:

Indeed, there is behavior change. It is easy to revert the old behavior. But 
was the old behavior designed? It looks as a side effect of the implementation.

What is a reason of making an array an integer type?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29159>
_______________________________________

From report at bugs.python.org  Wed Jan  4 15:27:53 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Wed, 04 Jan 2017 20:27:53 +0000
Subject: [issue29159] Regression in bytes constructor
In-Reply-To: <1483559394.32.0.335371111214.issue29159@psf.upfronthosting.co.za>
Message-ID: <1483561673.07.0.0892617797381.issue29159@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

Ah, if numpy arrays are affected, this should be restored. But swallowing an arbitrary exception doesn't look good to me.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29159>
_______________________________________

From report at bugs.python.org  Wed Jan  4 15:28:05 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Wed, 04 Jan 2017 20:28:05 +0000
Subject: [issue29159] Regression in bytes constructor
In-Reply-To: <1483559394.32.0.335371111214.issue29159@psf.upfronthosting.co.za>
Message-ID: <1483561685.71.0.664295505334.issue29159@psf.upfronthosting.co.za>


Changes by Serhiy Storchaka <storchaka+cpython at gmail.com>:


----------
nosy: +inada.naoki

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29159>
_______________________________________

From report at bugs.python.org  Wed Jan  4 15:32:09 2017
From: report at bugs.python.org (Xavier de Gaye)
Date: Wed, 04 Jan 2017 20:32:09 +0000
Subject: [issue26851] android compilation and link flags
In-Reply-To: <1461673211.54.0.945034926098.issue26851@psf.upfronthosting.co.za>
Message-ID: <1483561929.94.0.912510714676.issue26851@psf.upfronthosting.co.za>


Changes by Xavier de Gaye <xdegaye at gmail.com>:


----------
versions: +Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue26851>
_______________________________________

From report at bugs.python.org  Wed Jan  4 15:54:34 2017
From: report at bugs.python.org (Roundup Robot)
Date: Wed, 04 Jan 2017 20:54:34 +0000
Subject: [issue26851] android compilation and link flags
In-Reply-To: <1461673211.54.0.945034926098.issue26851@psf.upfronthosting.co.za>
Message-ID: <20170104205431.10575.96455.3BDB9170@psf.io>


Roundup Robot added the comment:

New changeset fa2bc63e64c6 by Xavier de Gaye in branch '3.6':
Issue #26851: Set Android compilation and link flags.
https://hg.python.org/cpython/rev/fa2bc63e64c6

New changeset af363b5200ff by Xavier de Gaye in branch 'default':
Issue #26851: Merge 3.6.
https://hg.python.org/cpython/rev/af363b5200ff

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue26851>
_______________________________________

From report at bugs.python.org  Wed Jan  4 15:58:21 2017
From: report at bugs.python.org (Xavier de Gaye)
Date: Wed, 04 Jan 2017 20:58:21 +0000
Subject: [issue26851] android compilation and link flags
In-Reply-To: <1461673211.54.0.945034926098.issue26851@psf.upfronthosting.co.za>
Message-ID: <1483563501.1.0.215741401134.issue26851@psf.upfronthosting.co.za>


Xavier de Gaye added the comment:

Latest patch committed with the following simplification: BASECFLAGS and LDFLAGS are now set for armv7 at the same place, as suggested by Martin in msg271518.

----------
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue26851>
_______________________________________

From report at bugs.python.org  Wed Jan  4 16:04:51 2017
From: report at bugs.python.org (Martin Panter)
Date: Wed, 04 Jan 2017 21:04:51 +0000
Subject: [issue29070] Integration tests for pty.spawn on Linux and all other
 platforms
In-Reply-To: <1482684225.17.0.467768789806.issue29070@psf.upfronthosting.co.za>
Message-ID: <1483563891.66.0.957176500318.issue29070@psf.upfronthosting.co.za>


Martin Panter added the comment:

Hi Cornelius and thanks for the work. Since the patch adds a significant amount of code, I think you might have to sign the contributor agreement: <http://www.python.org/psf/contrib/contrib-form/>. You can do it in a web browser.

I would like to review your tests, but because there is a lot of code to understand and I don?t have much time, it might take me a while. If you can find any way to simplify it, that would be great.

I have some comments on the code review, and will probably post more as I begin to understand what you are proposing.

It looks like you depend on fixing Issue 26228, but the patch there will conflict with your changes. Maybe merge with the other patch, or propose an alternative fix.

The documentation currently mentions the code is only tested on Linux, so it would be nice to update that.

The patch does introduce behavioural changes, if you consider abuse like monkey-patching os.exec() after importing the pty module. It is best not to make unnecessary changes in a bug fix.

Why does the patch slow the tests down so much? Ideally, it is nice to keep the tests as fast as possible.

What is the problem with using a genuine exec() call? Why do we need PtyMockingExecTestBase?

----------
versions: +Python 2.7, Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29070>
_______________________________________

From report at bugs.python.org  Wed Jan  4 16:05:40 2017
From: report at bugs.python.org (Martin Panter)
Date: Wed, 04 Jan 2017 21:05:40 +0000
Subject: [issue29070] Integration tests for pty.spawn on Linux and all other
 platforms
In-Reply-To: <1482684225.17.0.467768789806.issue29070@psf.upfronthosting.co.za>
Message-ID: <1483563940.71.0.783169277232.issue29070@psf.upfronthosting.co.za>


Martin Panter added the comment:

Ignore my comment about contrib agreement, that must have come through recently :)

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29070>
_______________________________________

From report at bugs.python.org  Wed Jan  4 16:38:21 2017
From: report at bugs.python.org (Alexander Belopolsky)
Date: Wed, 04 Jan 2017 21:38:21 +0000
Subject: [issue29159] Regression in bytes constructor
In-Reply-To: <1483559394.32.0.335371111214.issue29159@psf.upfronthosting.co.za>
Message-ID: <1483565901.8.0.806547644677.issue29159@psf.upfronthosting.co.za>


Alexander Belopolsky added the comment:

I don't think we should put too much effort into preserving numpy behavior.  Consider this python 3.5 session:

>>> import numpy
>>> a = numpy.array([1])
>>> bytes(a)
__main__:1: VisibleDeprecationWarning: converting an array with ndim > 0 to an index will result in an error in the future
b'\x00'
>>> a = numpy.array([2, 2])
>>> bytes(a)
b'\x02\x00\x00\x00\x02\x00\x00\x00'

It looks like this behavior is just an artifact of ndarray providing both __index__ and buffer protocol and not something thought out by numpy developers.

I wonder if we could check for buffer protocol support before detecting an integer argument?  I also recall a discussion of deprecating bytes(int) altogether.  See <https://mail.python.org/pipermail/python-ideas/2014-March/027295.html>.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29159>
_______________________________________

From report at bugs.python.org  Wed Jan  4 16:38:36 2017
From: report at bugs.python.org (Alexander Belopolsky)
Date: Wed, 04 Jan 2017 21:38:36 +0000
Subject: [issue29159] Regression in bytes constructor
In-Reply-To: <1483559394.32.0.335371111214.issue29159@psf.upfronthosting.co.za>
Message-ID: <1483565916.6.0.157081702293.issue29159@psf.upfronthosting.co.za>


Changes by Alexander Belopolsky <alexander.belopolsky at gmail.com>:


----------
nosy: +ncoghlan

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29159>
_______________________________________

From report at bugs.python.org  Wed Jan  4 16:51:37 2017
From: report at bugs.python.org (Alexander Belopolsky)
Date: Wed, 04 Jan 2017 21:51:37 +0000
Subject: [issue29159] Regression in bytes constructor
In-Reply-To: <1483559394.32.0.335371111214.issue29159@psf.upfronthosting.co.za>
Message-ID: <1483566697.63.0.141922889063.issue29159@psf.upfronthosting.co.za>


Alexander Belopolsky added the comment:

Also relevant:

  * #20895 - Add bytes.empty_buffer and deprecate bytes(17) for the same purpose
  * PEP 467 - Minor API improvements for binary sequences

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29159>
_______________________________________

From report at bugs.python.org  Wed Jan  4 17:07:23 2017
From: report at bugs.python.org (Aviv Palivoda)
Date: Wed, 04 Jan 2017 22:07:23 +0000
Subject: [issue29021] Custom functions in sqlite receive None on invalid UTF-8
In-Reply-To: <1482225891.27.0.11020777474.issue29021@psf.upfronthosting.co.za>
Message-ID: <1483567643.56.0.71569603838.issue29021@psf.upfronthosting.co.za>


Aviv Palivoda added the comment:

The problem is in _pysqlite_build_py_params function at connection.c. In case we have text we do the following code:

case SQLITE_TEXT:
      val_str = (const char*)sqlite3_value_text(cur_value);
      cur_py_value = PyUnicode_FromString(val_str);
      /* TODO: have a way to show errors here */
      if (!cur_py_value) {
             PyErr_Clear();
             Py_INCREF(Py_None);
             cur_py_value = Py_None;
      }
      break;

As you can see we call PyUnicode_FromString instead of text_factory.

I started making a patch to fix this by passing the text_factory to _pysqlite_build_py_params function but I currently have a problem with setting the result to the sqlite. User text_factory may return any type of object and I can't see how to handle that...

----------
nosy: +palaviv

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29021>
_______________________________________

From report at bugs.python.org  Wed Jan  4 17:20:29 2017
From: report at bugs.python.org (Davin Potts)
Date: Wed, 04 Jan 2017 22:20:29 +0000
Subject: [issue29158] Possible glitch in the interaction of a thread and a
 multiprocessing manager
In-Reply-To: <1483557543.41.0.26156873729.issue29158@psf.upfronthosting.co.za>
Message-ID: <1483568429.93.0.672176364752.issue29158@psf.upfronthosting.co.za>


Davin Potts added the comment:

There are too many things going on in this example -- it would be far easier to digest if the example could be simplified.

The general programming rule of thumb (completely unrelated to but still just as relevant to Python) that I think David might have been invoking is:  create processes first then create threads inside of them.  Otherwise, if you fork a process that has multiple threads going inside it, you should expect problems.  Assuming you're on a unix platform, it looks like you're creating threads then forking a process as well as doing it the other way around in another part of your code.

Different topic:  you mention killing the main process for server.py... which would likely kill the manager process referred to by shared_objects_manager... but you're creating a different manager process in bridge.py that is told to listen on the same port...


Without pulling apart your code further, I suspect confusion over how to use a Manager to share objects / data / information across processes.  If it helps, generally one process creates a manager instance (which itself results in the creation of a new process) and then other processes / threads created are created by that first process and given a handle on the manager instance or the objects managed by that manager.  I am a bit confused by your example but I hope that explanation helps provide some clarity?

----------
nosy: +davin

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29158>
_______________________________________

From report at bugs.python.org  Wed Jan  4 17:34:57 2017
From: report at bugs.python.org (Pavel Chuvakhov)
Date: Wed, 04 Jan 2017 22:34:57 +0000
Subject: [issue29160] `pow` with three int arguments works like it had two
 arguments
Message-ID: <1483569297.56.0.246813655497.issue29160@psf.upfronthosting.co.za>


New submission from Pavel Chuvakhov:

Standard `pow` function of three integer arguments should result in a reminder `(x**y) % z`. It seems that `pow(x,y,z)` ignores `%z` operation if type(z) is not `int`. This happens also in the cases when `z` has type numpy.int32, numpy.int64, etc. I consider such a behavior of `pow` as a bug to be fixed.

Thank you for your attantion, guys!
Best wishes, Pavel.

----------
components: Distutils
files: pow-bug_report.py
messages: 284675
nosy: dstufft, eric.araujo, pch
priority: normal
severity: normal
status: open
title: `pow` with three int arguments works like it had two arguments
type: behavior
versions: Python 2.7
Added file: http://bugs.python.org/file46146/pow-bug_report.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29160>
_______________________________________

From report at bugs.python.org  Wed Jan  4 18:16:34 2017
From: report at bugs.python.org (Benjamin Pollak)
Date: Wed, 04 Jan 2017 23:16:34 +0000
Subject: [issue29160] `pow` with three int arguments works like it had two
 arguments
In-Reply-To: <1483569297.56.0.246813655497.issue29160@psf.upfronthosting.co.za>
Message-ID: <1483571794.09.0.17156991754.issue29160@psf.upfronthosting.co.za>


Benjamin Pollak added the comment:

I tried running your script, but it crashes on the last two lines (variables n2 and n1 appear to be undefined). Could you please update that script so we can have a better idea of the behavior of your bug?

Thanks,
BP

----------
nosy: +Benjamin Pollak

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29160>
_______________________________________

From report at bugs.python.org  Wed Jan  4 18:28:50 2017
From: report at bugs.python.org (Jim Fasarakis-Hilliard)
Date: Wed, 04 Jan 2017 23:28:50 +0000
Subject: [issue29160] pow with three int arguments works like it had two
 arguments
In-Reply-To: <1483569297.56.0.246813655497.issue29160@psf.upfronthosting.co.za>
Message-ID: <1483572530.76.0.0627787923688.issue29160@psf.upfronthosting.co.za>


Changes by Jim Fasarakis-Hilliard <d.f.hilliard at gmail.com>:


----------
components: +Interpreter Core -Distutils
title: `pow` with three int arguments works like it had two arguments -> pow with three int arguments works like it had two arguments

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29160>
_______________________________________

From report at bugs.python.org  Wed Jan  4 18:40:45 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Wed, 04 Jan 2017 23:40:45 +0000
Subject: [issue29023] Results of random.seed() call with integer argument
 should be claimed deterministic.
In-Reply-To: <1482237974.75.0.612413216492.issue29023@psf.upfronthosting.co.za>
Message-ID: <1483573245.3.0.882368752233.issue29023@psf.upfronthosting.co.za>


Raymond Hettinger added the comment:

The wording can perhaps be made more precise.  However, this needs to be the end of this series of tracker items which are turning into time wasters.

----------
assignee: docs at python -> rhettinger
nosy: +rhettinger
priority: normal -> low

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29023>
_______________________________________

From report at bugs.python.org  Wed Jan  4 18:44:44 2017
From: report at bugs.python.org (Gary E. Miller)
Date: Wed, 04 Jan 2017 23:44:44 +0000
Subject: [issue29161] random.SystemRandom(seed) documentation issue
Message-ID: <1483573484.55.0.222683171285.issue29161@psf.upfronthosting.co.za>


New submission from Gary E. Miller:

The man page for random.SystemRandom([seed]]) fails to mention that the parameter 'seed' is never used.  This should be prominent in the documentation.  I have found several cases where a seed was provided to SystemRandom().

https://docs.python.org/2.7/library/random.html?highlight=systemrandom

Present in all versions of Python docs that I could find.

----------
components: Library (Lib)
messages: 284678
nosy: gem
priority: normal
severity: normal
status: open
title: random.SystemRandom(seed) documentation issue
type: behavior
versions: Python 2.7, Python 3.3, Python 3.4, Python 3.5, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29161>
_______________________________________

From report at bugs.python.org  Wed Jan  4 19:04:38 2017
From: report at bugs.python.org (Case Van Horsen)
Date: Thu, 05 Jan 2017 00:04:38 +0000
Subject: [issue29160] pow with three int arguments works like it had two
 arguments
In-Reply-To: <1483569297.56.0.246813655497.issue29160@psf.upfronthosting.co.za>
Message-ID: <1483574678.72.0.862350819468.issue29160@psf.upfronthosting.co.za>


Case Van Horsen added the comment:

This is a bug/limitation in numpy. If x and y are Python integers, and type(z) is of another numeric type, pow calls the nb_power slot of the underlying type. Here a quick example using numpy.int32 and gmpy2.mpz:

>>> import numpy, gmpy2
>>> pow(11,13,7)
4
>>> pow(11,13,numpy.int32(7))
34522712143931
>>> pow(11,13,gmpy2.mpz(7))
mpz(4)


>>> (11).__pow__(13,7)
4
>>> numpy.int32(11).__pow__(13,7)
34522712143931
>>> gmpy2.mpz(11).__pow__(13,7)
mpz(4)


casevh

----------
nosy: +casevh

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29160>
_______________________________________

From report at bugs.python.org  Wed Jan  4 19:13:35 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Thu, 05 Jan 2017 00:13:35 +0000
Subject: [issue29161] random.SystemRandom(seed) documentation issue
In-Reply-To: <1483573484.55.0.222683171285.issue29161@psf.upfronthosting.co.za>
Message-ID: <1483575215.59.0.315071721836.issue29161@psf.upfronthosting.co.za>


Changes by Raymond Hettinger <raymond.hettinger at gmail.com>:


----------
assignee:  -> rhettinger
nosy: +rhettinger

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29161>
_______________________________________

From report at bugs.python.org  Wed Jan  4 19:15:46 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Thu, 05 Jan 2017 00:15:46 +0000
Subject: [issue29161] random.SystemRandom(seed) documentation issue
In-Reply-To: <1483573484.55.0.222683171285.issue29161@psf.upfronthosting.co.za>
Message-ID: <1483575346.56.0.348395816328.issue29161@psf.upfronthosting.co.za>


Raymond Hettinger added the comment:

>From the docs at https://docs.python.org/3/library/random.html#alternative-generator :

"""
class random.SystemRandom([seed])
Class that uses the os.urandom() function for generating random numbers from sources provided by the operating system. Not available on all systems. Does not rely on software state, and sequences are not reproducible. Accordingly, the seed() method has no effect and is ignored. The getstate() and setstate() methods raise NotImplementedError if called.
"""

What is it that you think needs to be added or changed?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29161>
_______________________________________

From report at bugs.python.org  Wed Jan  4 19:39:25 2017
From: report at bugs.python.org (INADA Naoki)
Date: Thu, 05 Jan 2017 00:39:25 +0000
Subject: [issue29159] Regression in bytes constructor
In-Reply-To: <1483559394.32.0.335371111214.issue29159@psf.upfronthosting.co.za>
Message-ID: <1483576765.64.0.773256774774.issue29159@psf.upfronthosting.co.za>


Changes by INADA Naoki <songofacandy at gmail.com>:


----------
keywords: +patch
Added file: http://bugs.python.org/file46147/29159-index-fallback.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29159>
_______________________________________

From report at bugs.python.org  Wed Jan  4 19:53:40 2017
From: report at bugs.python.org (Gary E. Miller)
Date: Thu, 05 Jan 2017 00:53:40 +0000
Subject: [issue29161] random.SystemRandom(seed) documentation issue
In-Reply-To: <1483573484.55.0.222683171285.issue29161@psf.upfronthosting.co.za>
Message-ID: <1483577620.02.0.841512293532.issue29161@psf.upfronthosting.co.za>


Gary E. Miller added the comment:

I would change:

"Accordingly, the seed() method has no effect and is ignored."

To:

"Accordingly, the optional seed parameter and the seed() method have no effect and are ignored."

It was not obvious to me that the seed paramrter got passed to the seed() method and was then ignored.  After all, why have an ignored parameter that is not plainly documented as ignored.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29161>
_______________________________________

From report at bugs.python.org  Wed Jan  4 20:06:31 2017
From: report at bugs.python.org (Berker Peksag)
Date: Thu, 05 Jan 2017 01:06:31 +0000
Subject: [issue29148] Inheritance behaviour ambiguos
In-Reply-To: <1483506887.2.0.751213470204.issue29148@psf.upfronthosting.co.za>
Message-ID: <1483578391.17.0.49697729812.issue29148@psf.upfronthosting.co.za>


Berker Peksag added the comment:

You may find Raymond Hettinger's "super() considered super" article helpful to understand how Python's super() works: https://rhettinger.wordpress.com/2011/05/26/super-considered-super/

----------
nosy: +berker.peksag

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29148>
_______________________________________

From report at bugs.python.org  Wed Jan  4 20:18:31 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Thu, 05 Jan 2017 01:18:31 +0000
Subject: [issue29161] random.SystemRandom(seed) documentation issue
In-Reply-To: <1483573484.55.0.222683171285.issue29161@psf.upfronthosting.co.za>
Message-ID: <1483579111.48.0.0250364541677.issue29161@psf.upfronthosting.co.za>


Raymond Hettinger added the comment:

> why have an ignored parameter that is not plainly documented as ignored.

Because it improves the substitutability of one RNG for another (i.e. the same reason that we even have a seed() method).

----------
priority: normal -> low
resolution:  -> not a bug

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29161>
_______________________________________

From report at bugs.python.org  Wed Jan  4 20:20:08 2017
From: report at bugs.python.org (OO O)
Date: Thu, 05 Jan 2017 01:20:08 +0000
Subject: [issue29162] pyshell.py: name 'sys' is not defined
Message-ID: <1483579208.82.0.533923327462.issue29162@psf.upfronthosting.co.za>


New submission from OO O:

Sorry for my bad English.

line 7, 13 at pyshell.py.
Call sys befor import sys.
Cause name 'sys' is not defined

----------
assignee: terry.reedy
components: IDLE
messages: 284684
nosy: OO O, terry.reedy
priority: normal
severity: normal
status: open
title: pyshell.py: name 'sys' is not defined
type: crash
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29162>
_______________________________________

From report at bugs.python.org  Wed Jan  4 20:27:12 2017
From: report at bugs.python.org (Michal Ostrowski)
Date: Thu, 05 Jan 2017 01:27:12 +0000
Subject: [issue29163] shlex error in posix mode and empty quotes
Message-ID: <1483579631.92.0.342698548142.issue29163@psf.upfronthosting.co.za>


New submission from Michal Ostrowski:

Three examples below.  I believe the second is wrong, because it reorders
the "echo" and "," tokens.

>>> list(shlex.shlex('echo b="a",echo bar', posix=True))
['echo', 'b', '=', 'a', ',', 'echo', 'bar']
>>> list(shlex.shlex('echo b="",echo bar', posix=True))
['echo', 'b', '=', 'echo', ',', 'bar']
>>> list(shlex.shlex('echo b="",echo bar'))
['echo', 'b', '=', '""', ',', 'echo', 'bar']

----------
messages: 284685
nosy: Michal Ostrowski
priority: normal
severity: normal
status: open
title: shlex error in posix mode and empty quotes

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29163>
_______________________________________

From report at bugs.python.org  Wed Jan  4 20:27:37 2017
From: report at bugs.python.org (Michal Ostrowski)
Date: Thu, 05 Jan 2017 01:27:37 +0000
Subject: [issue29163] shlex error in posix mode and empty quotes
In-Reply-To: <1483579631.92.0.342698548142.issue29163@psf.upfronthosting.co.za>
Message-ID: <1483579657.91.0.0185588157271.issue29163@psf.upfronthosting.co.za>


Changes by Michal Ostrowski <mostrows at gmail.com>:


----------
components: +Library (Lib)
type:  -> behavior
versions: +Python 2.7, Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29163>
_______________________________________

From report at bugs.python.org  Wed Jan  4 20:36:21 2017
From: report at bugs.python.org (Nick Coghlan)
Date: Thu, 05 Jan 2017 01:36:21 +0000
Subject: [issue29157] random.c: Prefer getrandom() over getentropy(),
 handle ENOSYS in py_getentropy()
In-Reply-To: <1483551181.22.0.980001779001.issue29157@psf.upfronthosting.co.za>
Message-ID: <1483580181.16.0.155456818375.issue29157@psf.upfronthosting.co.za>


Changes by Nick Coghlan <ncoghlan at gmail.com>:


----------
nosy: +ncoghlan

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29157>
_______________________________________

From report at bugs.python.org  Wed Jan  4 20:43:53 2017
From: report at bugs.python.org (Gary E. Miller)
Date: Thu, 05 Jan 2017 01:43:53 +0000
Subject: [issue29161] random.SystemRandom(seed) documentation issue
In-Reply-To: <1483573484.55.0.222683171285.issue29161@psf.upfronthosting.co.za>
Message-ID: <1483580633.79.0.18437653468.issue29161@psf.upfronthosting.co.za>


Gary E. Miller added the comment:

> > why have an ignored parameter that is not plainly documented as ignored.

> Because it improves the substitutability of one RNG for another (i.e. the same reason that we even have a seed() method).

I understand why it the parameter it there.  I think the parameter should stay there.  I am all for orthogonal functions.  It just needs to be documented that the parameter is ignored by SystemRandom().

Programmers are being misled by the online documentation and adding a seed parameter when it does nothing.

I should not have to read the library code to find out that a parameter is ignored.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29161>
_______________________________________

From report at bugs.python.org  Wed Jan  4 20:44:27 2017
From: report at bugs.python.org (Nick Coghlan)
Date: Thu, 05 Jan 2017 01:44:27 +0000
Subject: [issue29157] random.c: Prefer getrandom() over getentropy(),
 handle ENOSYS in py_getentropy()
In-Reply-To: <1483551181.22.0.980001779001.issue29157@psf.upfronthosting.co.za>
Message-ID: <1483580667.8.0.61691152647.issue29157@psf.upfronthosting.co.za>


Nick Coghlan added the comment:

Aside from a couple of outdated comments and the EPERM question, the attached patch looks good to me.

Regarding EPERM, I think it would make sense to make py_getrandom and py_getentropy handle that consistently, otherwise I can see future maintainers readily getting confused by the discrepancy.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29157>
_______________________________________

From report at bugs.python.org  Wed Jan  4 20:45:43 2017
From: report at bugs.python.org (Gary E. Miller)
Date: Thu, 05 Jan 2017 01:45:43 +0000
Subject: [issue29161] random.SystemRandom(seed) documentation issue
In-Reply-To: <1483573484.55.0.222683171285.issue29161@psf.upfronthosting.co.za>
Message-ID: <1483580743.93.0.454519124762.issue29161@psf.upfronthosting.co.za>


Gary E. Miller added the comment:

Is there a better place to submit documentation problems to?  After my programming team spends a lot of valuable time figuring what the Python doc failed to mention I would like this knowledge to be put to good use by others.  Paying it forward if you will.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29161>
_______________________________________

From report at bugs.python.org  Wed Jan  4 21:00:24 2017
From: report at bugs.python.org (Alexander Belopolsky)
Date: Thu, 05 Jan 2017 02:00:24 +0000
Subject: [issue29159] Regression in bytes constructor
In-Reply-To: <1483559394.32.0.335371111214.issue29159@psf.upfronthosting.co.za>
Message-ID: <1483581624.54.0.143520952542.issue29159@psf.upfronthosting.co.za>


Alexander Belopolsky added the comment:

Inada,

Can you explain what your patch does?  I don't understand why you single out the OverflowError.  When is __index__ expected to raise it?

>>> (2**300).__index__()
2037035976334486086268445688409378161051468393665936250636140449354381299763336706183397376

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29159>
_______________________________________

From report at bugs.python.org  Wed Jan  4 21:00:55 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Thu, 05 Jan 2017 02:00:55 +0000
Subject: [issue29161] random.SystemRandom(seed) documentation issue
In-Reply-To: <1483573484.55.0.222683171285.issue29161@psf.upfronthosting.co.za>
Message-ID: <1483581655.96.0.945643738853.issue29161@psf.upfronthosting.co.za>


Raymond Hettinger added the comment:

Thank you for the submission.  I respect what you're trying to do, but disagree that there is any issue here.   The docs are clear that System Random uses os.urandom() for creating random numbers, that there is not state, that sequences aren't reproducible, and that seed method has no effect and is ignored.  Also, the square brackets on the seed argument is our traditional way of saying that an argument is optional.

Apologies, but I'm going to close this.  IMO, the proposed wording is less clear and doesn't really help if someone is already misunderstanding what SystemRandom is all about and is imagining that the "seed" argument has some effect when the seed method is documented as being ignored.

----------
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29161>
_______________________________________

From report at bugs.python.org  Wed Jan  4 21:05:50 2017
From: report at bugs.python.org (Berker Peksag)
Date: Thu, 05 Jan 2017 02:05:50 +0000
Subject: [issue29152] unittest subTest does not call addFailure
In-Reply-To: <1483534655.28.0.0626841658388.issue29152@psf.upfronthosting.co.za>
Message-ID: <1483581950.16.0.0329182740166.issue29152@psf.upfronthosting.co.za>


Berker Peksag added the comment:

Thanks for the report. That's because TestResult.addSubTest() updates TestResult.errors and TestResult.failures itself instead of calling the addError() and addFailure() methods respectively:

    if issubclass(err[0], test.failureException):
        errors = self.failures
    else:
        errors = self.errors
    errors.append((subtest, self._exc_info_to_string(err, test)))

I don't know whether it was intentional or not so I'm adding Antoine to nosy list.

(I removed 3.4 from the versions field because it's in security-fix-only mode.)

----------
components: +Library (Lib) -Tests
nosy: +berker.peksag, pitrou
stage:  -> test needed
type:  -> behavior
versions: +Python 3.6, Python 3.7 -Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29152>
_______________________________________

From report at bugs.python.org  Wed Jan  4 21:21:05 2017
From: report at bugs.python.org (OO O)
Date: Thu, 05 Jan 2017 02:21:05 +0000
Subject: [issue29164] make test always fail at 218/405 ( AssertionError: ',
 ' not found in '1234.5' )
Message-ID: <1483582865.63.0.483903091008.issue29164@psf.upfronthosting.co.za>


New submission from OO O:

make test always fail at 218/405
at test_format.py line 426

message:

[218/405/1] test test_format failed -- Traceback (most recent call last):
  File "/home/vm00/Downloads/Python-3.6.0/Lib/test/test_format.py", line 426, in test_locale
    self.assertIn(sep, text)
AssertionError: ',' not found in '1234.5'

----------
components: Installation
messages: 284692
nosy: OO O
priority: normal
severity: normal
status: open
title: make test always fail at 218/405 ( AssertionError: ',' not found in '1234.5' )
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29164>
_______________________________________

From report at bugs.python.org  Wed Jan  4 21:43:15 2017
From: report at bugs.python.org (Gary E. Miller)
Date: Thu, 05 Jan 2017 02:43:15 +0000
Subject: [issue29161] random.SystemRandom(seed) documentation issue
In-Reply-To: <1483573484.55.0.222683171285.issue29161@psf.upfronthosting.co.za>
Message-ID: <1483584195.97.0.716870606347.issue29161@psf.upfronthosting.co.za>


Gary E. Miller added the comment:

> The docs are clear that System Random uses os.urandom() for creating random numbers, that there is not state, that sequences aren't reproducible, and that seed method has no effect and is ignored.  

Agreed, but not relevant. I have anecdotal proof that the docs are not clear to a number of talented Python programmers that the seed parameter is not used.  Or that the seed parameter is passed to the seed() method (and thus unused).

Disappointing...

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29161>
_______________________________________

From report at bugs.python.org  Wed Jan  4 22:00:30 2017
From: report at bugs.python.org (Pavel Chuvakhov)
Date: Thu, 05 Jan 2017 03:00:30 +0000
Subject: [issue29160] pow with three int arguments works like it had two
 arguments
In-Reply-To: <1483569297.56.0.246813655497.issue29160@psf.upfronthosting.co.za>
Message-ID: <1483585230.63.0.730096012104.issue29160@psf.upfronthosting.co.za>


Pavel Chuvakhov added the comment:

Sorry about script, n1 should be n, and n2 should be m. Updated script is attached.

Ofc mpz is a way out. One also could cast int( np.int32 ) explicitly. I just wanted to underline that the best way is to hide all this stuff from a user and not make the user think what kind of int subtypes he should use. Python is a pretty language, isn't it? =)

Ones this is bug/limitation of numpy, shall we make a report for 'numpy guys' directly?

----------
components: +Distutils -Interpreter Core
Added file: http://bugs.python.org/file46148/pow-bug_repor-upd.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29160>
_______________________________________

From report at bugs.python.org  Wed Jan  4 22:18:24 2017
From: report at bugs.python.org (Milan Oberkirch)
Date: Thu, 05 Jan 2017 03:18:24 +0000
Subject: [issue29162] pyshell.py: name 'sys' is not defined
In-Reply-To: <1483579208.82.0.533923327462.issue29162@psf.upfronthosting.co.za>
Message-ID: <1483586304.19.0.483101482073.issue29162@psf.upfronthosting.co.za>


Milan Oberkirch added the comment:

pyshell.py is not part of CPython, this is the wrong place to report a bug like this.

----------
nosy: +zvyn

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29162>
_______________________________________

From report at bugs.python.org  Wed Jan  4 22:27:18 2017
From: report at bugs.python.org (Berker Peksag)
Date: Thu, 05 Jan 2017 03:27:18 +0000
Subject: [issue29162] pyshell.py: name 'sys' is not defined
In-Reply-To: <1483579208.82.0.533923327462.issue29162@psf.upfronthosting.co.za>
Message-ID: <1483586838.03.0.296547434114.issue29162@psf.upfronthosting.co.za>


Berker Peksag added the comment:

I think the OP is talking about Lib/idlelib/pyshell.py. The relevant changeset is 76f831e4b806. "from tkinter import *" imports the sys module as a side-effect. OO O, did you notice this by reading the code or by using a tool like flake8?

----------
nosy: +berker.peksag
type: crash -> behavior

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29162>
_______________________________________

From report at bugs.python.org  Wed Jan  4 22:32:16 2017
From: report at bugs.python.org (INADA Naoki)
Date: Thu, 05 Jan 2017 03:32:16 +0000
Subject: [issue28180] sys.getfilesystemencoding() should default to utf-8
In-Reply-To: <1474024622.65.0.838685461995.issue28180@psf.upfronthosting.co.za>
Message-ID: <1483587136.15.0.871659882246.issue28180@psf.upfronthosting.co.za>


INADA Naoki added the comment:

On Linux, I think most people wants UTF-8:surrogateescape by default, without fighting against locale and environment variables.

There are already `#if defined(__APPLE__) || defined(__ANDROID__)` path for it.
How about adding configure option to use same logic? (say `--with-encoding=(locale|utf-8)`, preferred encoding is changed in same way).

It may help many people building Python themselves without having root privilege for generating C.UTF-8 locale.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28180>
_______________________________________

From report at bugs.python.org  Wed Jan  4 22:35:25 2017
From: report at bugs.python.org (OO O)
Date: Thu, 05 Jan 2017 03:35:25 +0000
Subject: [issue29162] pyshell.py: name 'sys' is not defined
In-Reply-To: <1483579208.82.0.533923327462.issue29162@psf.upfronthosting.co.za>
Message-ID: <1483587325.04.0.889615529002.issue29162@psf.upfronthosting.co.za>


OO O added the comment:

to msg284696 - (view):

OK~ 
Sorry I found i forget to install tkinter before I run make.
so the "from tkinter import *" failed. ( sys not import ).

----------
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29162>
_______________________________________

From report at bugs.python.org  Wed Jan  4 22:35:58 2017
From: report at bugs.python.org (INADA Naoki)
Date: Thu, 05 Jan 2017 03:35:58 +0000
Subject: [issue29159] Regression in bytes constructor
In-Reply-To: <1483559394.32.0.335371111214.issue29159@psf.upfronthosting.co.za>
Message-ID: <1483587358.47.0.208650921608.issue29159@psf.upfronthosting.co.za>


INADA Naoki added the comment:

@belopolsky
It is backported behavior from Python 3.5, to fix regression.

See code review discussion in #27704.
The first patch kept the behavior, but we simplified it in second patch.
That cause this regression.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29159>
_______________________________________

From report at bugs.python.org  Wed Jan  4 22:53:04 2017
From: report at bugs.python.org (Berker Peksag)
Date: Thu, 05 Jan 2017 03:53:04 +0000
Subject: [issue29021] Custom functions in sqlite receive None on invalid UTF-8
In-Reply-To: <1482225891.27.0.11020777474.issue29021@psf.upfronthosting.co.za>
Message-ID: <1483588384.0.0.369648757261.issue29021@psf.upfronthosting.co.za>


Berker Peksag added the comment:

We can just fallback to the current behavior (i.e. return None) if the return type of text_factory(val_str) is not str, bytes or bytearray. I think this is also somewhat similar to what we do for the return values of sqlite3_column_text().

----------
nosy: +berker.peksag
stage:  -> needs patch
versions: +Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29021>
_______________________________________

From report at bugs.python.org  Wed Jan  4 22:56:20 2017
From: report at bugs.python.org (Berker Peksag)
Date: Thu, 05 Jan 2017 03:56:20 +0000
Subject: [issue29162] pyshell.py: name 'sys' is not defined
In-Reply-To: <1483579208.82.0.533923327462.issue29162@psf.upfronthosting.co.za>
Message-ID: <1483588580.58.0.977378871487.issue29162@psf.upfronthosting.co.za>


Berker Peksag added the comment:

Great, thanks for following up with us.

----------
resolution:  -> not a bug
stage:  -> resolved

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29162>
_______________________________________

From report at bugs.python.org  Wed Jan  4 22:58:33 2017
From: report at bugs.python.org (Milan Oberkirch)
Date: Thu, 05 Jan 2017 03:58:33 +0000
Subject: [issue29142] urllib: no_proxy variable values with leading dot not
 properly handled
In-Reply-To: <1483452589.12.0.177615671636.issue29142@psf.upfronthosting.co.za>
Message-ID: <1483588713.54.0.690197459069.issue29142@psf.upfronthosting.co.za>


Milan Oberkirch added the comment:

I added a line stripping dots from name, I think it makes sense to strip at the end as well (`example.com` is the same domain as `example.com.`).

----------
nosy: +zvyn
pull_requests: +8

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29142>
_______________________________________

From report at bugs.python.org  Wed Jan  4 22:59:53 2017
From: report at bugs.python.org (Berker Peksag)
Date: Thu, 05 Jan 2017 03:59:53 +0000
Subject: [issue29162] pyshell.py: name 'sys' is not defined
In-Reply-To: <1483579208.82.0.533923327462.issue29162@psf.upfronthosting.co.za>
Message-ID: <1483588793.12.0.614332650894.issue29162@psf.upfronthosting.co.za>


Berker Peksag added the comment:

I misread your comment. I think we need to add "import sys" to that file so I'm reopening this.

----------
keywords: +easy
resolution: not a bug -> 
stage: resolved -> needs patch
status: closed -> open
versions: +Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29162>
_______________________________________

From report at bugs.python.org  Wed Jan  4 23:18:30 2017
From: report at bugs.python.org (Roundup Robot)
Date: Thu, 05 Jan 2017 04:18:30 +0000
Subject: [issue29162] pyshell.py: name 'sys' is not defined
In-Reply-To: <1483579208.82.0.533923327462.issue29162@psf.upfronthosting.co.za>
Message-ID: <20170105041816.2957.59930.5E3950E6@psf.io>


Roundup Robot added the comment:

New changeset 52d671684342 by Terry Jan Reedy in branch '3.6':
Issue #29162: Don't depend on 'from tkinter import *' importing sys.
https://hg.python.org/cpython/rev/52d671684342

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29162>
_______________________________________

From report at bugs.python.org  Wed Jan  4 23:21:15 2017
From: report at bugs.python.org (Berker Peksag)
Date: Thu, 05 Jan 2017 04:21:15 +0000
Subject: [issue29142] urllib: no_proxy variable values with leading dot not
 properly handled
In-Reply-To: <1483452589.12.0.177615671636.issue29142@psf.upfronthosting.co.za>
Message-ID: <1483590075.87.0.754225141538.issue29142@psf.upfronthosting.co.za>


Berker Peksag added the comment:

Note: We don't officially use pull requests yet. Current way of contributing to Python is documented at https://docs.python.org/devguide/patch.html

----------
nosy: +berker.peksag

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29142>
_______________________________________

From report at bugs.python.org  Wed Jan  4 23:28:20 2017
From: report at bugs.python.org (Terry J. Reedy)
Date: Thu, 05 Jan 2017 04:28:20 +0000
Subject: [issue29162] pyshell.py: name 'sys' is not defined
In-Reply-To: <1483579208.82.0.533923327462.issue29162@psf.upfronthosting.co.za>
Message-ID: <1483590500.65.0.283323290851.issue29162@psf.upfronthosting.co.za>


Terry J. Reedy added the comment:

I agree that depending on the incidental import is a bug.  Tkinter could change, or I might change 'from tkinter import *' to 'from tkinter import Tk, ...' or 'import tkinter as tk'.

Serhiy, tkinter/__init__.py imports enum, sys, and re under their own names, so that * imports incidentally import them into other programs.  I believe it is customary for library modules to underscore stdlib imports to prevent this: "import enum as _enum", etc.  But I don't remember is this is in the devguide.  Do you think we should make the change to tkinter?

----------
nosy: +serhiy.storchaka
resolution:  -> fixed
stage: needs patch -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29162>
_______________________________________

From report at bugs.python.org  Wed Jan  4 23:44:25 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Thu, 05 Jan 2017 04:44:25 +0000
Subject: [issue29096] unicode_concatenate() optimization is not signal-safe
 (not atomic)
In-Reply-To: <1482957515.7.0.441178310139.issue29096@psf.upfronthosting.co.za>
Message-ID: <1483591465.99.0.868555001999.issue29096@psf.upfronthosting.co.za>


Changes by Raymond Hettinger <raymond.hettinger at gmail.com>:


----------
nosy: +arigo

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29096>
_______________________________________

From report at bugs.python.org  Wed Jan  4 23:48:07 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Thu, 05 Jan 2017 04:48:07 +0000
Subject: [issue29164] make test always fail at 218/405 ( AssertionError: ',
 ' not found in '1234.5' )
In-Reply-To: <1483582865.63.0.483903091008.issue29164@psf.upfronthosting.co.za>
Message-ID: <1483591687.3.0.0274900168862.issue29164@psf.upfronthosting.co.za>


Xiang Zhang added the comment:

Could you provide your default locale setting, the result of `locale.setlocale(locale.LC_ALL, '')`?

----------
nosy: +xiang.zhang

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29164>
_______________________________________

From report at bugs.python.org  Thu Jan  5 01:06:51 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Thu, 05 Jan 2017 06:06:51 +0000
Subject: [issue29145] failing overflow checks in replace_*
In-Reply-To: <1483468602.77.0.231507148809.issue29145@psf.upfronthosting.co.za>
Message-ID: <1483596411.22.0.705616444886.issue29145@psf.upfronthosting.co.za>


Xiang Zhang added the comment:

> It would be better to write the code in the same form as in 3.x. This could help backporting other patches if needed.

Yes, it is. :-) I only had a fast glance at at 3.x codes and thought they didn't share much. :-( But apparently I was wrong. The new patch backports 3.x code.

----------
Added file: http://bugs.python.org/file46149/replace-overflow-check-v2.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29145>
_______________________________________

From report at bugs.python.org  Thu Jan  5 01:47:59 2017
From: report at bugs.python.org (INADA Naoki)
Date: Thu, 05 Jan 2017 06:47:59 +0000
Subject: [issue29165] Use forward compatible macro in example code for
 creating new type
Message-ID: <1483598879.57.0.52341008423.issue29165@psf.upfronthosting.co.za>


New submission from INADA Naoki:

https://docs.python.org/2.7/extending/newtypes.html#the-basics uses PyObject_HEAD_INIT for type object header.

static PyTypeObject noddy_NoddyType = {
    PyObject_HEAD_INIT(NULL)
    0,                         /*ob_size*/

This code isn't compatible with Python 3.  In Python 3, PyVarObject_HEAD_INIT is used instead.
https://docs.python.org/3.6/extending/newtypes.html#the-basics

static PyTypeObject noddy_NoddyType = {
    PyVarObject_HEAD_INIT(NULL, 0)

This code is compatible with Python 2.


This example code can be copy and pasted when creating new extension.
If people start writing Python 2 extension, and forward port it to Python 3,
this small incompatibility cause compile error.

Let's use more forward compatible and short code for example.

----------
assignee: docs at python
components: Documentation
messages: 284709
nosy: docs at python, inada.naoki
priority: normal
severity: normal
status: open
title: Use forward compatible macro in example code for creating new type
type: enhancement
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29165>
_______________________________________

From report at bugs.python.org  Thu Jan  5 02:03:36 2017
From: report at bugs.python.org (Armin Rigo)
Date: Thu, 05 Jan 2017 07:03:36 +0000
Subject: [issue29096] unicode_concatenate() optimization is not signal-safe
 (not atomic)
In-Reply-To: <1482957515.7.0.441178310139.issue29096@psf.upfronthosting.co.za>
Message-ID: <1483599816.61.0.675169723151.issue29096@psf.upfronthosting.co.za>


Armin Rigo added the comment:

The signal handler is called between the INPLACE_ADD and the following STORE_FAST opcode, never from string_concatenate() itself.  A fix would be to make sure signal handlers are not called between these two opcodes.  See the minimal, proof-of-concept patch #1.  A possibly better fix, which at least should match the SETUP_FINALLY handling in the ticker handler, is attached as patch #2.  (Patches against 2.7.13)

----------
keywords: +patch
Added file: http://bugs.python.org/file46150/patch1.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29096>
_______________________________________

From report at bugs.python.org  Thu Jan  5 02:04:31 2017
From: report at bugs.python.org (Armin Rigo)
Date: Thu, 05 Jan 2017 07:04:31 +0000
Subject: [issue29096] unicode_concatenate() optimization is not signal-safe
 (not atomic)
In-Reply-To: <1482957515.7.0.441178310139.issue29096@psf.upfronthosting.co.za>
Message-ID: <1483599871.43.0.953821657324.issue29096@psf.upfronthosting.co.za>


Changes by Armin Rigo <arigo at users.sourceforge.net>:


Added file: http://bugs.python.org/file46151/patch2.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29096>
_______________________________________

From report at bugs.python.org  Thu Jan  5 02:05:32 2017
From: report at bugs.python.org (Armin Rigo)
Date: Thu, 05 Jan 2017 07:05:32 +0000
Subject: [issue29096] unicode_concatenate() optimization is not signal-safe
 (not atomic)
In-Reply-To: <1482957515.7.0.441178310139.issue29096@psf.upfronthosting.co.za>
Message-ID: <1483599932.28.0.0374376387867.issue29096@psf.upfronthosting.co.za>


Changes by Armin Rigo <arigo at users.sourceforge.net>:


Removed file: http://bugs.python.org/file46150/patch1.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29096>
_______________________________________

From report at bugs.python.org  Thu Jan  5 02:06:51 2017
From: report at bugs.python.org (Mama Mama)
Date: Thu, 05 Jan 2017 07:06:51 +0000
Subject: [issue29166] AT&T tech email support number <1 855 338 0710<AT&T tech
 email support phone number
Message-ID: <1483600011.89.0.656486652082.issue29166@psf.upfronthosting.co.za>


New submission from Mama Mama:

AT&T tech email support number <1 855 338 0710<AT&T tech email support phone number
AT&T email support number 1 855-338-0710 AT&T tech email support number  at&t email technical support phone number
AT&T email support number  1 (855 338 0710)AT&T tech email support number AT&T tech email support phone number 

<HUSSSTY<AT&T tech email support number>$$$>>>1 855 338 0710 <<$$$$$<$_)_)AT&T tech email support phone number AT&T email support number<LHHSSDF>
&<AT&T tech email support number>$$$>>>1 855 338 0710 <<$$$$$<$_)_)AT&T tech email support phone number AT&T email support number$$$$$$$$
1 855 338 0710  moto khama wla AT&T tech email support number AT&T email support number  AT&T tech email support phone number at&t email technical support number

----------
components: asyncio
files: 1 855 338 0710 AT&T tech email support phone number AT&T email support number.jpg
messages: 284711
nosy: Mama Mama, gvanrossum, yselivanov
priority: normal
severity: normal
status: open
title: AT&T tech email support number <1 855 338 0710<AT&T tech email support phone number
versions: Python 3.6
Added file: http://bugs.python.org/file46152/1 855 338 0710 AT&T tech email support phone number AT&T email support number.jpg

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29166>
_______________________________________

From report at bugs.python.org  Thu Jan  5 02:06:58 2017
From: report at bugs.python.org (Armin Rigo)
Date: Thu, 05 Jan 2017 07:06:58 +0000
Subject: [issue29096] unicode_concatenate() optimization is not signal-safe
 (not atomic)
In-Reply-To: <1482957515.7.0.441178310139.issue29096@psf.upfronthosting.co.za>
Message-ID: <1483600018.1.0.336550731081.issue29096@psf.upfronthosting.co.za>


Changes by Armin Rigo <arigo at users.sourceforge.net>:


Added file: http://bugs.python.org/file46153/patch1.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29096>
_______________________________________

From report at bugs.python.org  Thu Jan  5 02:12:17 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Thu, 05 Jan 2017 07:12:17 +0000
Subject: [issue29166] AT&T tech email support number <1 855 338 0710<AT&T tech
 email support phone number
Message-ID: <1483600337.39.0.772282755297.issue29166@psf.upfronthosting.co.za>


Changes by Xiang Zhang <angwerzx at 126.com>:


----------
Removed message: http://bugs.python.org/msg284711

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29166>
_______________________________________

From report at bugs.python.org  Thu Jan  5 02:12:29 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Thu, 05 Jan 2017 07:12:29 +0000
Subject: [issue29166] AT&T tech email support number <1 855 338 0710<AT&T tech
 email support phone number
Message-ID: <1483600349.27.0.521616330719.issue29166@psf.upfronthosting.co.za>


Changes by Xiang Zhang <angwerzx at 126.com>:


Removed file: http://bugs.python.org/file46152/1 855 338 0710 AT&T tech email support phone number AT&T email support number.jpg

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29166>
_______________________________________

From report at bugs.python.org  Thu Jan  5 02:12:46 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Thu, 05 Jan 2017 07:12:46 +0000
Subject: [issue29166] AT&T tech email support number <1 855 338 0710<AT&T tech
 email support phone number
Message-ID: <1483600366.06.0.809937860579.issue29166@psf.upfronthosting.co.za>


Changes by Xiang Zhang <angwerzx at 126.com>:


----------
nosy:  -gvanrossum, yselivanov
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29166>
_______________________________________

From report at bugs.python.org  Thu Jan  5 02:14:24 2017
From: report at bugs.python.org (Zachary Ware)
Date: Thu, 05 Jan 2017 07:14:24 +0000
Subject: [issue29166] Spam
Message-ID: <1483600464.66.0.640948786194.issue29166@psf.upfronthosting.co.za>


Changes by Zachary Ware <zachary.ware at gmail.com>:


----------
components:  -asyncio
nosy:  -Mama Mama
title: AT&T tech email support number <1 855 338 0710<AT&T tech email support phone number -> Spam
versions:  -Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29166>
_______________________________________

From report at bugs.python.org  Thu Jan  5 02:14:28 2017
From: report at bugs.python.org (jha.amit6666)
Date: Thu, 05 Jan 2017 07:14:28 +0000
Subject: [issue29147] registry value to be cleared when python is uninstalled
In-Reply-To: <1483505292.36.0.945723017008.issue29147@psf.upfronthosting.co.za>
Message-ID: <1483600468.39.0.675152934845.issue29147@psf.upfronthosting.co.za>


jha.amit6666 added the comment:

Before I read your comment, I ran disk clean up which deleted everything.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29147>
_______________________________________

From report at bugs.python.org  Thu Jan  5 02:43:16 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Thu, 05 Jan 2017 07:43:16 +0000
Subject: [issue26867] test_ssl test_options fails on ubuntu 16.04
In-Reply-To: <1461728238.71.0.110276038105.issue26867@psf.upfronthosting.co.za>
Message-ID: <1483602196.86.0.530522343928.issue26867@psf.upfronthosting.co.za>


Xiang Zhang added the comment:

This test still fails with lastest Py2.7 on Ubuntu 16.10. Could we backport the patch to silence the failure?

./python -m test.regrtest test_ssl
[1/1] test_ssl
test test_ssl failed -- Traceback (most recent call last):
  File "/home/angwer/py2.7/Lib/test/test_ssl.py", line 780, in test_options
    self.assertEqual(0, ctx.options)
AssertionError: 0 != 33554432L

1 test failed:
    test_ssl

----------
nosy: +benjamin.peterson

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue26867>
_______________________________________

From report at bugs.python.org  Thu Jan  5 02:44:39 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Thu, 05 Jan 2017 07:44:39 +0000
Subject: [issue28806] Improve the netrc library
In-Reply-To: <1480158810.39.0.736994382472.issue28806@psf.upfronthosting.co.za>
Message-ID: <1483602279.14.0.542021790427.issue28806@psf.upfronthosting.co.za>


Changes by Xiang Zhang <angwerzx at 126.com>:


----------
nosy: +r.david.murray -haypo

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28806>
_______________________________________

From report at bugs.python.org  Thu Jan  5 03:12:15 2017
From: report at bugs.python.org (Milan Oberkirch)
Date: Thu, 05 Jan 2017 08:12:15 +0000
Subject: [issue29142] urllib: no_proxy variable values with leading dot not
 properly handled
In-Reply-To: <1483452589.12.0.177615671636.issue29142@psf.upfronthosting.co.za>
Message-ID: <1483603935.41.0.896399291617.issue29142@psf.upfronthosting.co.za>


Milan Oberkirch added the comment:

Too bad, I was all exited when I saw that I can paste a PR link here :)

I downloaded the patch from the commit using github, I hope that works?

----------
keywords: +patch
Added file: http://bugs.python.org/file46154/ignore_dots-v1.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29142>
_______________________________________

From report at bugs.python.org  Thu Jan  5 04:01:24 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Thu, 05 Jan 2017 09:01:24 +0000
Subject: [issue29023] Results of random.seed() call with integer argument
 should be claimed deterministic.
In-Reply-To: <1482237974.75.0.612413216492.issue29023@psf.upfronthosting.co.za>
Message-ID: <1483606884.83.0.848403325466.issue29023@psf.upfronthosting.co.za>


Changes by Raymond Hettinger <raymond.hettinger at gmail.com>:


----------
Removed message: http://bugs.python.org/msg284677

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29023>
_______________________________________

From report at bugs.python.org  Thu Jan  5 04:02:25 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Thu, 05 Jan 2017 09:02:25 +0000
Subject: [issue29023] Results of random.seed() call with integer argument
 should be claimed deterministic.
In-Reply-To: <1482237974.75.0.612413216492.issue29023@psf.upfronthosting.co.za>
Message-ID: <1483606945.45.0.582754797698.issue29023@psf.upfronthosting.co.za>


Changes by Raymond Hettinger <raymond.hettinger at gmail.com>:


----------
keywords: +patch
Added file: http://bugs.python.org/file46155/random_docs.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29023>
_______________________________________

From report at bugs.python.org  Thu Jan  5 04:03:06 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Thu, 05 Jan 2017 09:03:06 +0000
Subject: [issue29023] Results of random.seed() call with integer argument
 should be claimed deterministic.
In-Reply-To: <1482237974.75.0.612413216492.issue29023@psf.upfronthosting.co.za>
Message-ID: <1483606986.67.0.885581141612.issue29023@psf.upfronthosting.co.za>


Raymond Hettinger added the comment:

Attached patch simplifies, improves, and syncs the main docs with the docstrings.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29023>
_______________________________________

From report at bugs.python.org  Thu Jan  5 04:26:22 2017
From: report at bugs.python.org (Nick Coghlan)
Date: Thu, 05 Jan 2017 09:26:22 +0000
Subject: [issue28180] sys.getfilesystemencoding() should default to utf-8
In-Reply-To: <1474024622.65.0.838685461995.issue28180@psf.upfronthosting.co.za>
Message-ID: <1483608382.57.0.914601332406.issue28180@psf.upfronthosting.co.za>


Nick Coghlan added the comment:

Anything purely on the Python side of things doesn't work in a traditional C environment - CPython relies on the C lib to do conversions during startup, so we need the C locale to be set correctly. We can do things differently on Mac OS X and iOS because Apple ensure that *C* behaves differently on Mac OS X and iOS (and apparently Google do something similar for Android, so I'll update the PEP to mention that as well).

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28180>
_______________________________________

From report at bugs.python.org  Thu Jan  5 04:36:28 2017
From: report at bugs.python.org (Berker Peksag)
Date: Thu, 05 Jan 2017 09:36:28 +0000
Subject: [issue29142] urllib: no_proxy variable values with leading dot not
 properly handled
In-Reply-To: <1483452589.12.0.177615671636.issue29142@psf.upfronthosting.co.za>
Message-ID: <1483608988.25.0.700140776938.issue29142@psf.upfronthosting.co.za>


Berker Peksag added the comment:

Yes, thanks!

----------
nosy: +martin.panter, xiang.zhang
stage:  -> patch review
type:  -> behavior
versions: +Python 3.5, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29142>
_______________________________________

From report at bugs.python.org  Thu Jan  5 04:42:56 2017
From: report at bugs.python.org (INADA Naoki)
Date: Thu, 05 Jan 2017 09:42:56 +0000
Subject: [issue28180] sys.getfilesystemencoding() should default to utf-8
In-Reply-To: <1474024622.65.0.838685461995.issue28180@psf.upfronthosting.co.za>
Message-ID: <1483609376.86.0.0256194647372.issue28180@psf.upfronthosting.co.za>


INADA Naoki added the comment:

> Anything purely on the Python side of things doesn't work in a traditional C environment - CPython relies on the C lib to do conversions during startup, so we need the C locale to be set correctly. 

What I propose is non't use mbstowcs, like __ANDROID__

wchar_t*
Py_DecodeLocale(const char* arg, size_t *size)
{
#if defined(__APPLE__) || defined(__ANDROID__)
    wchar_t *wstr;
    wstr = _Py_DecodeUTF8_surrogateescape(arg, strlen(arg));


On Linux, command line arguments and filepath is just a byte sequence.
So using UTF-8:surrogateescape from during startup should works fine.

Am I wrong?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28180>
_______________________________________

From report at bugs.python.org  Thu Jan  5 04:51:21 2017
From: report at bugs.python.org (Marc-Andre Lemburg)
Date: Thu, 05 Jan 2017 09:51:21 +0000
Subject: [issue28180] sys.getfilesystemencoding() should default to utf-8
In-Reply-To: <1483608382.57.0.914601332406.issue28180@psf.upfronthosting.co.za>
Message-ID: <d81609e1-2c27-5845-2581-5c6c204b798d@egenix.com>


Marc-Andre Lemburg added the comment:

On 05.01.2017 10:26, Nick Coghlan wrote:
> 
> Anything purely on the Python side of things doesn't work in a traditional C environment - CPython relies on the C lib to do conversions during startup, so we need the C locale to be set correctly. We can do things differently on Mac OS X and iOS because Apple ensure that *C* behaves differently on Mac OS X and iOS (and apparently Google do something similar for Android, so I'll update the PEP to mention that as well).

I believe IANADA-san (hope that's the right way to address him)
raised a good point though: what if a system doesn't come with
the C.UTF-8 local setup ?

The C lib would then error out when trying to use setlocale()
on such an environment.

Now, Python's main() function doesn't look at any such errors
(and neither do the other places which use it such as frozenmain.c
and readline.c), so it wouldn't even notice.

The setlocal() man-page doesn't mention how such a failure would
affect the current locale settings. My guess is that the locale
remains set to what it was before, which in case of a fresh C
application start is the "C" locale.

So in the implementation of the PEP, there should be a test
to see whether "C.UTF-8" does result in a successful
call to setlocale(). If it doesn't, there would have to be
some work-around to still make Python's FS encoding happy
while leaving the C lib locale set at "C".

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28180>
_______________________________________

From report at bugs.python.org  Thu Jan  5 05:10:46 2017
From: report at bugs.python.org (INADA Naoki)
Date: Thu, 05 Jan 2017 10:10:46 +0000
Subject: [issue28180] sys.getfilesystemencoding() should default to utf-8
In-Reply-To: <1474024622.65.0.838685461995.issue28180@psf.upfronthosting.co.za>
Message-ID: <1483611046.86.0.395331620974.issue28180@psf.upfronthosting.co.za>


INADA Naoki added the comment:

Why I want to add configure option to ignore locale is:


1. C.UTF-8 is not supported by RHEL7 (https://bugzilla.redhat.com/show_bug.cgi?id=1361965)

RHEL7 will be used for a long time.
And many people uses new Python instead of distro's Python, via pyenv or pythonz.
I feel deprecating C locale from Python 3.7 is bit aggressive.


2. Many admins like C locale.

locale setting will cause unintended side effects. So many admins dislike xx_XX.UTF-8 locale.
For example (from https://fumiyas.github.io/2016/12/25/dislike.sh-advent-calendar.html ):

$ mkdir tmp
$ cd tmp
$ touch a b c x y z A B C X Y Z
$ LC_ALL=C /bin/bash --noprofile --norc -c 'echo [A-Z]'
A B C X Y Z
$ LC_ALL=en_US.UTF-8 /bin/bash --noprofile --norc -c 'echo [A-Z]'
A b B c C x X y Y z Z


3. Many other languages can use UTF-8 even when C locale

node.js, Ruby, Rust, Go can use UTF-8 on Linux
People don't want to learn how to configure locale properly only for Python.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28180>
_______________________________________

From report at bugs.python.org  Thu Jan  5 05:31:55 2017
From: report at bugs.python.org (jan matejek)
Date: Thu, 05 Jan 2017 10:31:55 +0000
Subject: [issue29145] failing overflow checks in replace_*
In-Reply-To: <1483468602.77.0.231507148809.issue29145@psf.upfronthosting.co.za>
Message-ID: <1483612315.52.0.518233573367.issue29145@psf.upfronthosting.co.za>


jan matejek added the comment:

some instances are present in unicodeobject.c too

----------
Added file: http://bugs.python.org/file46156/unicode-overflow.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29145>
_______________________________________

From report at bugs.python.org  Thu Jan  5 05:50:23 2017
From: report at bugs.python.org (Evan)
Date: Thu, 05 Jan 2017 10:50:23 +0000
Subject: [issue29163] shlex error in posix mode and empty quotes
In-Reply-To: <1483579631.92.0.342698548142.issue29163@psf.upfronthosting.co.za>
Message-ID: <1483613423.0.0.248704159455.issue29163@psf.upfronthosting.co.za>


Changes by Evan <evanunderscore at gmail.com>:


----------
nosy: +evan_

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29163>
_______________________________________

From report at bugs.python.org  Thu Jan  5 05:50:28 2017
From: report at bugs.python.org (Nick Coghlan)
Date: Thu, 05 Jan 2017 10:50:28 +0000
Subject: [issue28180] sys.getfilesystemencoding() should default to utf-8
In-Reply-To: <1474024622.65.0.838685461995.issue28180@psf.upfronthosting.co.za>
Message-ID: <1483613428.37.0.466599990107.issue28180@psf.upfronthosting.co.za>


Nick Coghlan added the comment:

No, requesting a locale that doesn't exist doesn't error out, because we don't check the return code - it just keeps working the same way it does now (i.e. falling back to the legacy C locale).

However, it would be entirely reasonable to put together a competing PEP proposing to eliminate the reliance on the problematic libc APIs, and instead use locale independent replacements. I'm simply not offering to implement or champion such a PEP myself, as I think ignoring the locale settings rather than coercing them to something more sensible will break integration with C/C++ GUI toolkits like Tcl/Tk, Gtk, and Qt, and it's reasonable for us to expect OS providers to offer at least one of C.UTF-8 or en_US.UTF-8 (see https://github.com/python/peps/issues/171 for more on that).

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28180>
_______________________________________

From report at bugs.python.org  Thu Jan  5 05:52:00 2017
From: report at bugs.python.org (STINNER Victor)
Date: Thu, 05 Jan 2017 10:52:00 +0000
Subject: [issue29157] random.c: Prefer getrandom() over getentropy(),
 handle ENOSYS in py_getentropy()
In-Reply-To: <1483551181.22.0.980001779001.issue29157@psf.upfronthosting.co.za>
Message-ID: <1483613520.21.0.0781858278922.issue29157@psf.upfronthosting.co.za>


STINNER Victor added the comment:

New patch (version 2), much larger: it refactors the code, not only fix this specific issue (prefer getrandom() over getentropy()). Changes since getentropy.patch:

* Add a lot of comments to explain in depth how each function is implemented, which errors are handled, etc. It should help to audit the code: this code is very critical for security and so should be, IMHO, well documented.

* handle also EPERM and EINTR errors in getentropy(): retry on EINTR, fallback on /dev/urandom on EPERM -- sadly, I don't have access to a system with getentropy() to test this part of the code.

* call py_getrandom() and py_getentropy() in pyurandom() to make dev_urandom() simpler and so easy to review: dev_urandom() looses its blocking parameter

* Document the cached file descriptor, and cached st_dev+st_ino in dev_urandom().

* Document explicitly that functions are retried on EINTR error. Document that only getrandom() supports non-blocking mode. Document why we prefer an entropy source over others.

I'm not sure that getentropy() can fail with EPERM or EINTR in practice, but it shouldn't harm to handle correctly these errors :-) At least, getentropy() can fail with these errors on Linux since the glibc implements the getentropy() function using the getrandom() syscall (and it's known that getrandom() can fail with these errors). But on Linux, the code now prefers getrandom() over getentropy().

Should we use the new shiny code on all Python versions? Or only fix the reported issue on all Python issues, and use the refactored code in Python default?

Note: Python 2.7 still supports VMS. VMS is unsupported in Python 3.3 and the VMS code was removed in Python 3.4 (issue 16136): see the PEP 11.

I suggest to use the same code on all maintained Python versions to ease maintenance.

----------
Added file: http://bugs.python.org/file46157/random-2.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29157>
_______________________________________

From report at bugs.python.org  Thu Jan  5 05:54:26 2017
From: report at bugs.python.org (Simon Percivall)
Date: Thu, 05 Jan 2017 10:54:26 +0000
Subject: [issue29167] Race condition in enum.py:_decompose()
Message-ID: <1483613666.47.0.301459916652.issue29167@psf.upfronthosting.co.za>


New submission from Simon Percivall:

When called by `_create_pseudo_member_()`, the dictionary iteration of `_value2member_map` in `_decompose()` in enum.py may lead to a "RuntimeError: dictionary changed size during iteration". For me, it happened in `re.compile`.

```
Traceback (most recent call last):
  ...
  File "/usr/local/Cellar/python3/3.6.0/Frameworks/Python.framework/Versions/3.6/lib/python3.6/re.py", line 233, in compile
    return _compile(pattern, flags)
  File "/usr/local/Cellar/python3/3.6.0/Frameworks/Python.framework/Versions/3.6/lib/python3.6/re.py", line 301, in _compile
    p = sre_compile.compile(pattern, flags)
  File "/usr/local/Cellar/python3/3.6.0/Frameworks/Python.framework/Versions/3.6/lib/python3.6/sre_compile.py", line 562, in compile
    p = sre_parse.parse(p, flags)
  File "/usr/local/Cellar/python3/3.6.0/Frameworks/Python.framework/Versions/3.6/lib/python3.6/sre_parse.py", line 866, in parse
    p.pattern.flags = fix_flags(str, p.pattern.flags)
  File "/usr/local/Cellar/python3/3.6.0/Frameworks/Python.framework/Versions/3.6/lib/python3.6/sre_parse.py", line 835, in fix_flags
    flags |= SRE_FLAG_UNICODE
  File "/usr/local/Cellar/python3/3.6.0/Frameworks/Python.framework/Versions/3.6/lib/python3.6/enum.py", line 794, in __or__
    result = self.__class__(self._value_ | self.__class__(other)._value_)
  File "/usr/local/Cellar/python3/3.6.0/Frameworks/Python.framework/Versions/3.6/lib/python3.6/enum.py", line 291, in __call__
    return cls.__new__(cls, value)
  File "/usr/local/Cellar/python3/3.6.0/Frameworks/Python.framework/Versions/3.6/lib/python3.6/enum.py", line 533, in __new__
    return cls._missing_(value)
  File "/usr/local/Cellar/python3/3.6.0/Frameworks/Python.framework/Versions/3.6/lib/python3.6/enum.py", line 760, in _missing_
    new_member = cls._create_pseudo_member_(value)
  File "/usr/local/Cellar/python3/3.6.0/Frameworks/Python.framework/Versions/3.6/lib/python3.6/enum.py", line 769, in _create_pseudo_member_
    _, extra_flags = _decompose(cls, value)
  File "/usr/local/Cellar/python3/3.6.0/Frameworks/Python.framework/Versions/3.6/lib/python3.6/enum.py", line 849, in _decompose
    for v, m in flag._value2member_map_.items()
  File "/usr/local/Cellar/python3/3.6.0/Frameworks/Python.framework/Versions/3.6/lib/python3.6/enum.py", line 848, in <listcomp>
    (m, v)
RuntimeError: dictionary changed size during iteration
```

----------
messages: 284724
nosy: simon.percivall
priority: normal
severity: normal
status: open
title: Race condition in enum.py:_decompose()
type: crash
versions: Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29167>
_______________________________________

From report at bugs.python.org  Thu Jan  5 05:54:48 2017
From: report at bugs.python.org (Nick Coghlan)
Date: Thu, 05 Jan 2017 10:54:48 +0000
Subject: [issue28180] sys.getfilesystemencoding() should default to utf-8
In-Reply-To: <1474024622.65.0.838685461995.issue28180@psf.upfronthosting.co.za>
Message-ID: <1483613688.04.0.668880312544.issue28180@psf.upfronthosting.co.za>


Nick Coghlan added the comment:

The PEP already explains how other runtimes achieve UTF-8 and UTF-18-LE everywhere: by ignoring the C/C++ locale entirely. While this breaks integration with other C/C++ components, the developers of those languages and runtimes simply don't care, as they never supported integrating with those components in the first place.

CPython doesn't have that luxury, since it is used extensively in locale aware desktop applications.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28180>
_______________________________________

From report at bugs.python.org  Thu Jan  5 05:56:45 2017
From: report at bugs.python.org (Evan)
Date: Thu, 05 Jan 2017 10:56:45 +0000
Subject: [issue29163] shlex error in posix mode and empty quotes
In-Reply-To: <1483579631.92.0.342698548142.issue29163@psf.upfronthosting.co.za>
Message-ID: <1483613805.37.0.248446908567.issue29163@psf.upfronthosting.co.za>


Evan added the comment:

This is also broken in 3.5.2, but not in 3.6.0:

>>> list(shlex.shlex('echo b="",echo bar', posix=True))
['echo', 'b', '=', '', ',', 'echo', 'bar']

I'm guessing it was fixed incidentally by the changes in issue1521950.

----------
versions: +Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29163>
_______________________________________

From report at bugs.python.org  Thu Jan  5 05:57:17 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Thu, 05 Jan 2017 10:57:17 +0000
Subject: [issue29167] Race condition in enum.py:_decompose()
In-Reply-To: <1483613666.47.0.301459916652.issue29167@psf.upfronthosting.co.za>
Message-ID: <1483613837.61.0.483129133199.issue29167@psf.upfronthosting.co.za>


Changes by Serhiy Storchaka <storchaka+cpython at gmail.com>:


----------
components: +IO
nosy: +ethan.furman
type: crash -> behavior

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29167>
_______________________________________

From report at bugs.python.org  Thu Jan  5 06:09:54 2017
From: report at bugs.python.org (Nick Coghlan)
Date: Thu, 05 Jan 2017 11:09:54 +0000
Subject: [issue29157] random.c: Prefer getrandom() over getentropy(),
 handle ENOSYS in py_getentropy()
In-Reply-To: <1483551181.22.0.980001779001.issue29157@psf.upfronthosting.co.za>
Message-ID: <1483614594.96.0.867868287857.issue29157@psf.upfronthosting.co.za>


Nick Coghlan added the comment:

New patch looks good to me, and +1 on applying the refactoring to all supported branches.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29157>
_______________________________________

From report at bugs.python.org  Thu Jan  5 06:10:18 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Thu, 05 Jan 2017 11:10:18 +0000
Subject: [issue29145] failing overflow checks in replace_*
In-Reply-To: <1483468602.77.0.231507148809.issue29145@psf.upfronthosting.co.za>
Message-ID: <1483614618.31.0.135013683072.issue29145@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

replace-overflow-check-v2.patch LGTM.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29145>
_______________________________________

From report at bugs.python.org  Thu Jan  5 06:11:53 2017
From: report at bugs.python.org (STINNER Victor)
Date: Thu, 05 Jan 2017 11:11:53 +0000
Subject: [issue28180] sys.getfilesystemencoding() should default to utf-8
In-Reply-To: <1474024622.65.0.838685461995.issue28180@psf.upfronthosting.co.za>
Message-ID: <1483614713.43.0.706506222214.issue28180@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Sorry, I still didn't have enough time to read carefully the PEP 538. But since the discussion already started on this issue, I will add my comments:

* I'm sure that many Linux, UNIX and BSD systems don't have the "C.UTF-8" locale. For example, HP-UX has "C.utf8" which is not exactly "C.UTF-8".

* Setting the locale has an impact on all libraries running in the Python process. At this point, I'm not sure that it is what we want.

* I'm not sure that it's ok in 2017 to always force the UTF-8 encoding if the user locale uses a different encoding. I had the same concern with the PEP 528 (Change Windows console encoding to UTF-8) and PEP 529 (Change Windows filesystem encoding to UTF-8) on Windows, but these PEPs were approved and merged into Python 3.6. My fear is obviously mojibake with the other applications using the other encoding, the locale encoding. Other applications are not impacted by setlocale() in the Python process.

* I proposed an opt-in option to force UTF-8: -X utf8 command line option and PYTHONUTF8=1 env var. Opt-in will obviously reduce the risk of backward compatibility issues. With an opt-in option, users are better prepared for mojibake issues.

* I dislike "Backporting to earlier Python 3 releases". In my experience, changes on how Python handles text (encodings, codecs, etc.) always have subtle issues, and users dislike getting backward incompatible changes in minor releases. *Maybe* if the option is an opt-in, the risk is lower and acceptable?

* I dislike that Fedora has such downstream change. I would prefer to decide upstream how to convert UTF-8 slowly as a first-class citizen in Python. Otherwise, Fedora would behave differently than other Linux distributions and it can be painful to write applications having the same behaviour on all Linux distributions. But I also understand that Fedora has sometimes to move faster than the slow CPython project :-) Fedora can also seen as a toy to experiment changes quickly which helps to provide a wide feedback upstream to take better decision.

* Using strict or surrogateescape error handler is a very important choice which has a wide impact. If we use utf8 by default (PEP 538), people will problably complain less if Python magically pass undecoded bytes thanks to the surrogateescape. If the option is an opt-in, strict may make sense. But surrogateescape is maybe still more "convenient". I don't know at this point.

Nick: it seems like you have a well defined plan. But I dislike on multiple points. I don't know if it's better to try to convince you to change your PEP, or write a different PEP.

I planned to write such "UTF-8" PEP since 2015, but I never started because the scope is so large that I fear all tiny but annoying corner cases...

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28180>
_______________________________________

From report at bugs.python.org  Thu Jan  5 06:13:29 2017
From: report at bugs.python.org (STINNER Victor)
Date: Thu, 05 Jan 2017 11:13:29 +0000
Subject: [issue29157] random.c: Prefer getrandom() over getentropy(),
 handle ENOSYS in py_getentropy()
In-Reply-To: <1483551181.22.0.980001779001.issue29157@psf.upfronthosting.co.za>
Message-ID: <1483614809.97.0.891512657927.issue29157@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Nick: Thanks for the review!

Since random.c is critical for security, I would prefer to have at least a review from another core developer. I added Serhiy and Xiang in the nosy list. I'm also looking at you, Christian! ;-) (Christian reused random.c code in the pycrytography project.)

----------
nosy: +serhiy.storchaka, xiang.zhang

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29157>
_______________________________________

From report at bugs.python.org  Thu Jan  5 06:14:51 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Thu, 05 Jan 2017 11:14:51 +0000
Subject: [issue29145] failing overflow checks in replace_*
In-Reply-To: <1483468602.77.0.231507148809.issue29145@psf.upfronthosting.co.za>
Message-ID: <1483614891.77.0.621479138817.issue29145@psf.upfronthosting.co.za>


Xiang Zhang added the comment:

> some instances are present in unicodeobject.c too

Thanks for your notification.

v3 adds the some changes needed I could find in unicodeobject.c. Nothing in v2 is changed.

----------
Added file: http://bugs.python.org/file46158/replace-overflow-check-v3.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29145>
_______________________________________

From report at bugs.python.org  Thu Jan  5 06:15:05 2017
From: report at bugs.python.org (Evan)
Date: Thu, 05 Jan 2017 11:15:05 +0000
Subject: [issue29163] shlex error in posix mode and empty quotes
In-Reply-To: <1483579631.92.0.342698548142.issue29163@psf.upfronthosting.co.za>
Message-ID: <1483614905.16.0.695351542029.issue29163@psf.upfronthosting.co.za>


Evan added the comment:

My mistake, it was addressed directly in issue21999. This is already fixed in 2.7.13 and will be fixed in 3.5.3.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29163>
_______________________________________

From report at bugs.python.org  Thu Jan  5 06:19:44 2017
From: report at bugs.python.org (Christian Heimes)
Date: Thu, 05 Jan 2017 11:19:44 +0000
Subject: [issue29157] random.c: Prefer getrandom() over getentropy(),
 handle ENOSYS in py_getentropy()
In-Reply-To: <1483551181.22.0.980001779001.issue29157@psf.upfronthosting.co.za>
Message-ID: <1483615184.89.0.234504931753.issue29157@psf.upfronthosting.co.za>


Christian Heimes added the comment:

I'm doing a review now.

By the way I did not copy random.c for cryptography. I took bits and pieces out of it. https://github.com/pyca/cryptography/blob/master/src/_cffi_src/openssl/src/osrandom_engine.c and https://github.com/pyca/cryptography/blob/master/src/_cffi_src/openssl/src/osrandom_engine.h

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29157>
_______________________________________

From report at bugs.python.org  Thu Jan  5 06:21:29 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Thu, 05 Jan 2017 11:21:29 +0000
Subject: [issue29145] failing overflow checks in replace_*
In-Reply-To: <1483468602.77.0.231507148809.issue29145@psf.upfronthosting.co.za>
Message-ID: <1483615289.93.0.709017877594.issue29145@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

It seems to me that unicodeobject.c should be changed in 3.x too.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29145>
_______________________________________

From report at bugs.python.org  Thu Jan  5 06:30:57 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Thu, 05 Jan 2017 11:30:57 +0000
Subject: [issue29145] failing overflow checks in replace_*
In-Reply-To: <1483468602.77.0.231507148809.issue29145@psf.upfronthosting.co.za>
Message-ID: <1483615857.49.0.247510700292.issue29145@psf.upfronthosting.co.za>


Xiang Zhang added the comment:

> It seems to me that unicodeobject.c should be changed in 3.x too.

I don't understand. Would you mind tell me which part? The only suspicious part to me is `PY_SSIZE_T_MAX >> (rkind - 1)`. It seems should be `PY_SSIZE_T_MAX / rkind`.

And I missed jan's patch so wrote mine. :-( But they are almost the same. :-)

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29145>
_______________________________________

From report at bugs.python.org  Thu Jan  5 06:36:20 2017
From: report at bugs.python.org (Marc-Andre Lemburg)
Date: Thu, 05 Jan 2017 11:36:20 +0000
Subject: [issue28180] sys.getfilesystemencoding() should default to utf-8
In-Reply-To: <1483613688.04.0.668880312544.issue28180@psf.upfronthosting.co.za>
Message-ID: <5aea9f92-d3a0-35c0-8081-13255c354620@egenix.com>


Marc-Andre Lemburg added the comment:

While going for the full locale setting may be a good option,
perhaps just focusing on the FS encoding for now is a better
way forward (and also more in line with the ticket title).

So essentially go for the PEP 529 approach on Unix as well
(except that we use 'ascii' as fallback in legacy mode):

https://www.python.org/dev/peps/pep-0529/

The PEP also includes a section on affected modules, which we
could double check (even though the term "FS encoding" implies
that only file system relevant APIs are touched by such a change,
the encoding is used in several other places as well):

https://www.python.org/dev/peps/pep-0529/#id14

For Windows, a couple of modules such as pwd and nis are not
used, so those may need some extra attention.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28180>
_______________________________________

From report at bugs.python.org  Thu Jan  5 06:50:51 2017
From: report at bugs.python.org (Mark Dickinson)
Date: Thu, 05 Jan 2017 11:50:51 +0000
Subject: [issue29160] pow with three int arguments works like it had two
 arguments
In-Reply-To: <1483569297.56.0.246813655497.issue29160@psf.upfronthosting.co.za>
Message-ID: <1483617051.76.0.699731333536.issue29160@psf.upfronthosting.co.za>


Mark Dickinson added the comment:

> shall we make a report for 'numpy guys' directly?

Yes, please! I'll close here, since this isn't a core Python issue: what 3rd party libraries choose to do in their `__pow__` methods is out of the control of the core language.

----------
nosy: +mark.dickinson
resolution:  -> not a bug
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29160>
_______________________________________

From report at bugs.python.org  Thu Jan  5 07:12:49 2017
From: report at bugs.python.org (Simon Schuler)
Date: Thu, 05 Jan 2017 12:12:49 +0000
Subject: [issue29168] multiprocessing pickle error
Message-ID: <1483618368.97.0.107401413332.issue29168@psf.upfronthosting.co.za>


New submission from Simon Schuler:

Hello,

the following code doesn't work any longer in the new Python version 3.6.

import sys
import os
import subprocess
from multiprocessing import Pool, Value, Queue
import multiprocessing
import logging
import logging.handlers
import pickle

queue = multiprocessing.Manager().Queue(-1)
qh = logging.handlers.QueueHandler(queue)
pickle.dumps(qh)

It raises the following exception.
>>> TypeError: can't pickle _thread.RLock objects


Furthermore, also for customized logging handler classes it doesn't work anymore.

class CustomHandler(logging.Handler):
    def __init__(self, queue):
        logging.Handler.__init__(self)
        self.queue = queue
        
    def emit(self, record):
        try:
            ei = record.exc_info
            if ei:
                dummy = self.format(record)
                record.exc_info = None  
        except (KeyboardInterrupt, SystemExit):
            raise
        except:
            self.handleError(record)

For a centralized logging facility in a multiprocess environment this is a big problem. How can I handle this in the 3.6 version?

----------
messages: 284738
nosy: cxss
priority: normal
severity: normal
status: open
title: multiprocessing pickle error
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29168>
_______________________________________

From report at bugs.python.org  Thu Jan  5 07:17:20 2017
From: report at bugs.python.org (Berker Peksag)
Date: Thu, 05 Jan 2017 12:17:20 +0000
Subject: [issue29163] shlex error in posix mode and empty quotes
In-Reply-To: <1483579631.92.0.342698548142.issue29163@psf.upfronthosting.co.za>
Message-ID: <1483618640.4.0.150702824878.issue29163@psf.upfronthosting.co.za>


Changes by Berker Peksag <berker.peksag at gmail.com>:


----------
resolution:  -> out of date
stage:  -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29163>
_______________________________________

From report at bugs.python.org  Thu Jan  5 07:18:37 2017
From: report at bugs.python.org (Xavier de Gaye)
Date: Thu, 05 Jan 2017 12:18:37 +0000
Subject: [issue26851] android compilation and link flags
In-Reply-To: <1461673211.54.0.945034926098.issue26851@psf.upfronthosting.co.za>
Message-ID: <1483618717.79.0.970551178029.issue26851@psf.upfronthosting.co.za>


Xavier de Gaye added the comment:

In the code review Victor asked the following question:

> I'm a little bit surprised that you need to pass so much options which are specific to the platform. GCC doesn't have a generic option "hello, please compile for my architecture?"

ARM defines two incompatible floating point ABIS that specify the functions calling conventions, whether to use integer registers (soft floating point ABI) or the floating point registers (hard floating point ABI).  The compiler must be told which one to use and whether the hardware FPU may be used if present when the soft floating point ABI has been chosen. This last case is set with the command line option '-mfloat-abi=softfp'. The following two documents describe this feature in details:
  https://wiki.debian.org/ArmHardFloatPort/VfpComparison
  https://wiki.debian.org/ArmHardFloatPort

The following command prints the command line options, including implicitly predefined ones of the clang compiler in android-ndk-r13b and shows that the default is '-mfloat-abi=soft' for armv7:

$ ./clang -x c < /dev/null -dM -E - -target armv7-none-linux-androideabi -###
Android clang version 3.8.256229  (based on LLVM 3.8.256229)
Target: armv7-none-linux-android
Thread model: posix
InstalledDir: /opt/android-ndk/toolchains/llvm/prebuilt/linux-x86_64/bin/.
 "/home/opt.symlink/android-ndk-r13b/toolchains/llvm/prebuilt/linux-x86_64/bin/clang" "-cc1" "-triple" "armv7-none-linux-android" "-E" "-disable-free" "-disable-llvm-verifier" "-main-file-name" "-" "-mrelocation-model" "pic" "-pic-level" "1" "-mthread-model" "posix" "-mdisable-fp-elim" "-fmath-errno" "-masm-verbose" "-mconstructor-aliases" "-fuse-init-array" "-target-cpu" "cortex-a8" "-target-feature" "+soft-float-abi" "-target-abi" "aapcs-linux" "-mfloat-abi" "soft" "-target-linker-version" "2.24" "-dwarf-column-info" "-debugger-tuning=gdb" "-resource-dir" "/home/opt.symlink/android-ndk-r13b/toolchains/llvm/prebuilt/linux-x86_64/bin/../lib64/clang/3.8.256229" "-internal-isystem" "/usr/local/include" "-internal-isystem" "/home/opt.symlink/android-ndk-r13b/toolchains/llvm/prebuilt/linux-x86_64/bin/../lib64/clang/3.8.256229/include" "-internal-externc-isystem" "/include" "-internal-externc-isystem" "/usr/include" "-fdebug-compilation-dir" "/opt/android-ndk/toolchains/llvm/prebuilt/linux-x86_64/bin" "-ferror-limit" "19" "-fmessage-length" "100" "-femulated-tls" "-fallow-half-arguments-and-returns" "-fno-signed-char" "-fobjc-runtime=gcc" "-fdiagnostics-show-option" "-fcolor-diagnostics" "-dM" "-o" "-" "-x" "c" "-"

Note that the system libffi must also be compiled with these same flags. The bundled libffi is deprecated in 3.6 by issue 27976 and has been removed in 3.7 by issue 27979.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue26851>
_______________________________________

From report at bugs.python.org  Thu Jan  5 07:24:21 2017
From: report at bugs.python.org (luke_16)
Date: Thu, 05 Jan 2017 12:24:21 +0000
Subject: [issue29158] Possible glitch in the interaction of a thread and a
 multiprocessing manager
In-Reply-To: <1483557543.41.0.26156873729.issue29158@psf.upfronthosting.co.za>
Message-ID: <1483619061.8.0.0944762616143.issue29158@psf.upfronthosting.co.za>


luke_16 added the comment:

Relating to the idea that it is not recommended to spawn a process whenever there are already spawned threads running, which would be the case of the server side of my example, I have to disagree. If a process is supposed to be completely independent of the current one (no shared memory), than this should not be a problem. Anyway, I changed the server.py module just to clear this possible error source.

Trying to simplify the example, I moved everything to the client.py module and eliminated the "thread spawns process" part in order to see what happens. Actually, I don't think that that was the case because I could not see any process being spawned when trying to connect to the server. I think that this happens in the current MainThread.

In the new client, I'm doing exactly what I was trying to avoid, which is to wait for the connect() method to return. Surprisingly, the problem is still there, in a way. Let's say you start the server and then the client. The first attempt to send something fails and the client tries to connect to the server. All goes well with the connection, and the client starts sending stuff to the server. If you stop the server, wait a while (or not), and restart it again, the client can no longer send anything, and tries to reconnect again. The connection is then successful, but immediately after that, it cannot send anything because of a BrokenPipeError. Only after the exception is raised and the next loop of the "while" begins, the client can send something to the server again. If you insert a time.sleep(x) of any number of seconds inside the scope of the first "else" (between line 26 and 38) and right after a "remote_buffer.put('anything')", it will raise a BrokenPipeError. Only when a new loop in "while" begins, then it is possible again to send something to the server.

This makes no sense to me. There must be something wrong with it.

----------
Added file: http://bugs.python.org/file46159/case2.zip

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29158>
_______________________________________

From report at bugs.python.org  Thu Jan  5 07:29:43 2017
From: report at bugs.python.org (Charles Brossollet)
Date: Thu, 05 Jan 2017 12:29:43 +0000
Subject: [issue4431] Distutils MSVC doesn't create manifest file
In-Reply-To: <1227644194.32.0.230924221438.issue4431@psf.upfronthosting.co.za>
Message-ID: <1483619383.08.0.464779655012.issue4431@psf.upfronthosting.co.za>


Charles Brossollet added the comment:

Here with python 2.7.13 and Visual Studio 2013, I get hit by the problem and it looks like link.exe needs the /MANIFEST keyword. Voting to get it fixed!

----------
nosy: +Charles Brossollet

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue4431>
_______________________________________

From report at bugs.python.org  Thu Jan  5 07:41:50 2017
From: report at bugs.python.org (Nick Coghlan)
Date: Thu, 05 Jan 2017 12:41:50 +0000
Subject: [issue28180] sys.getfilesystemencoding() should default to utf-8
In-Reply-To: <1474024622.65.0.838685461995.issue28180@psf.upfronthosting.co.za>
Message-ID: <1483620110.56.0.870517187115.issue28180@psf.upfronthosting.co.za>


Nick Coghlan added the comment:

The trade-offs here are incredibly complex (and are mainly a matter of deciding whose code and configurations we want to break in 3.7+), so I think competing PEPs are going to be better than attempting to create a combined PEP that tries to cover all the options.

That way each PEP can argue as strongly as it can for the respective authors preferred approach to tackling the default C locale problem, even if they point to a common background section in one of the PEPs (similar to the way PEPs 522 and 524 shared a common problem definition, even though they proposed different ways of handling it).

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28180>
_______________________________________

From report at bugs.python.org  Thu Jan  5 07:48:12 2017
From: report at bugs.python.org (Mark Lawrence)
Date: Thu, 05 Jan 2017 12:48:12 +0000
Subject: [issue4431] Distutils MSVC doesn't create manifest file
In-Reply-To: <1227644194.32.0.230924221438.issue4431@psf.upfronthosting.co.za>
Message-ID: <1483620492.5.0.244852942489.issue4431@psf.upfronthosting.co.za>


Changes by Mark Lawrence <breamoreboy at gmail.com>:


----------
nosy:  -BreamoreBoy

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue4431>
_______________________________________

From report at bugs.python.org  Thu Jan  5 07:55:24 2017
From: report at bugs.python.org (Xavier de Gaye)
Date: Thu, 05 Jan 2017 12:55:24 +0000
Subject: [issue26865] Meta-issue: support of the android platform
In-Reply-To: <1461685011.99.0.617135447086.issue26865@psf.upfronthosting.co.za>
Message-ID: <1483620924.14.0.218211235383.issue26865@psf.upfronthosting.co.za>


Xavier de Gaye added the comment:

issue #28833: cross compilation of third-party extension modules

----------
dependencies: +byte-compile fails for cross-builds, cross compilation of third-party extension modules

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue26865>
_______________________________________

From report at bugs.python.org  Thu Jan  5 08:39:54 2017
From: report at bugs.python.org (Simon Schuler)
Date: Thu, 05 Jan 2017 13:39:54 +0000
Subject: [issue29168] multiprocessing pickle error
In-Reply-To: <1483618368.97.0.107401413332.issue29168@psf.upfronthosting.co.za>
Message-ID: <1483623594.83.0.210473079196.issue29168@psf.upfronthosting.co.za>


Simon Schuler added the comment:

Attached is a sample program to illustrate the problem. When I use a multiprocessing pool the exception is raised.

----------
Added file: http://bugs.python.org/file46160/sample.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29168>
_______________________________________

From report at bugs.python.org  Thu Jan  5 08:45:03 2017
From: report at bugs.python.org (Chi Hsuan Yen)
Date: Thu, 05 Jan 2017 13:45:03 +0000
Subject: [issue26851] android compilation and link flags
In-Reply-To: <1461673211.54.0.945034926098.issue26851@psf.upfronthosting.co.za>
Message-ID: <1483623903.63.0.286921572957.issue26851@psf.upfronthosting.co.za>


Chi Hsuan Yen added the comment:

> Note that the system libffi must also be compiled with these same flags

Just tried. With my packaging scripts, CPython on ARM is compiled with -mfloat-abi=softfp -mfpu=vfpv3-d16 while libffi not. test_ctypes pass as usual. Maybe ctypes test suite is not complete?

And by the way, if all packages on a system should be compiled with the same set of flags, shouldn't those flags specified in build/package scripts rather than Makefile of individual packages?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue26851>
_______________________________________

From report at bugs.python.org  Thu Jan  5 08:47:56 2017
From: report at bugs.python.org (Davin Potts)
Date: Thu, 05 Jan 2017 13:47:56 +0000
Subject: [issue29168] multiprocessing pickle error
In-Reply-To: <1483618368.97.0.107401413332.issue29168@psf.upfronthosting.co.za>
Message-ID: <1483624076.89.0.966224767417.issue29168@psf.upfronthosting.co.za>


Davin Potts added the comment:

In your first example (not sample.py), if I make the following change:

#queue = multiprocessing.Manager().Queue(-1)
queue = 42


I am still able to reproduce the exception you observe.  This suggests the issue has nothing to do with the use of multiprocessing but does have to do with the logging handlers.

I can also reproduce that this is a change in behavior between 3.5 and 3.6.

----------
nosy: +davin

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29168>
_______________________________________

From report at bugs.python.org  Thu Jan  5 08:50:51 2017
From: report at bugs.python.org (Davin Potts)
Date: Thu, 05 Jan 2017 13:50:51 +0000
Subject: [issue29168] multiprocessing pickle error
In-Reply-To: <1483618368.97.0.107401413332.issue29168@psf.upfronthosting.co.za>
Message-ID: <1483624251.13.0.62498739697.issue29168@psf.upfronthosting.co.za>


Changes by Davin Potts <python at discontinuity.net>:


----------
nosy: +vinay.sajip

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29168>
_______________________________________

From report at bugs.python.org  Thu Jan  5 09:44:31 2017
From: report at bugs.python.org (Barry A. Warsaw)
Date: Thu, 05 Jan 2017 14:44:31 +0000
Subject: [issue28180] sys.getfilesystemencoding() should default to utf-8
In-Reply-To: <1483614713.43.0.706506222214.issue28180@psf.upfronthosting.co.za>
Message-ID: <20170105094428.25652cb9@subdivisions.wooz.org>


Barry A. Warsaw added the comment:

On Jan 05, 2017, at 11:11 AM, STINNER Victor wrote:

>I'm sure that many Linux, UNIX and BSD systems don't have the "C.UTF-8"
>locale. For example, HP-UX has "C.utf8" which is not exactly "C.UTF-8".
>
>I'm not sure that it's ok in 2017 to always force the UTF-8 encoding if the
>user locale uses a different encoding.

It's not just any different encoding, it's specifically C (implicitly,
C.ASCII).

>I proposed an opt-in option to force UTF-8: -X utf8 command line option and
>PYTHONUTF8=1 env var. Opt-in will obviously reduce the risk of backward
>compatibility issues. With an opt-in option, users are better prepared for
>mojibake issues.

If this is true, then I would like a configuration option to default this on.
As mentioned, Debian and Ubuntu already have C.UTF-8 and most environments
(although not all, see my sbuild/schroot comment earlier) will at least be
C.UTF-8.  Perhaps it doesn't matter then, but what I really want is that for
those few odd outliers (e.g. schroot), Python would act the same inside and
out those environments.  I really don't want people to have to add that envar
or switch (or even export LC_ALL) to get proper build behavior.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28180>
_______________________________________

From report at bugs.python.org  Thu Jan  5 09:45:10 2017
From: report at bugs.python.org (Eric V. Smith)
Date: Thu, 05 Jan 2017 14:45:10 +0000
Subject: [issue29164] make test always fail at 218/405 ( AssertionError: ',
 ' not found in '1234.5' )
In-Reply-To: <1483582865.63.0.483903091008.issue29164@psf.upfronthosting.co.za>
Message-ID: <1483627510.06.0.264274652385.issue29164@psf.upfronthosting.co.za>


Changes by Eric V. Smith <eric at trueblade.com>:


----------
nosy: +eric.smith

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29164>
_______________________________________

From report at bugs.python.org  Thu Jan  5 09:59:43 2017
From: report at bugs.python.org (Vinay Sajip)
Date: Thu, 05 Jan 2017 14:59:43 +0000
Subject: [issue29168] multiprocessing pickle error
In-Reply-To: <1483618368.97.0.107401413332.issue29168@psf.upfronthosting.co.za>
Message-ID: <1483628383.8.0.305720288341.issue29168@psf.upfronthosting.co.za>


Vinay Sajip added the comment:

It may be that _thread.RLock objects were pickleable before 3.6 but aren't now, and if so, then this is unrelated to logging except by coincidence.

Why do you want to pickle the *handler*? It's not especially designed for sending over the wire, and that's not supported - so it's not a bug. If that code worked in Python < 3.6, that's just by luck.

Pickling of LogRecords should work.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29168>
_______________________________________

From report at bugs.python.org  Thu Jan  5 10:11:48 2017
From: report at bugs.python.org (Matthias Klose)
Date: Thu, 05 Jan 2017 15:11:48 +0000
Subject: [issue29169] update zlib to 1.2.10
Message-ID: <1483629107.49.0.982374684244.issue29169@psf.upfronthosting.co.za>


New submission from Matthias Klose:

These are the changes updating zlib from 1.2.8 to 1.2.10. It is only used when building without a system zlib.  The new release includes fixes for security issues CVE-2016-9840, CVE-2016-9841, CVE-2016-9842, CVE-2016-9843.

Intending to update all active branches. Larry, is it ok to add this before the upcoming 3.4 and 3.5 releases, or should it wait?

Changes in 1.2.10 (2 Jan 2017)
- Avoid warnings on snprintf() return value
- Fix bug in deflate_stored() for zero-length input
- Fix bug in gzwrite.c that produced corrupt gzip files
- Remove files to be installed before copying them in Makefile.in
- Add warnings when compiling with assembler code

Changes in 1.2.9 (31 Dec 2016)
- Fix contrib/minizip to permit unzipping with desktop API [Zouzou]
- Improve contrib/blast to return unused bytes
- Assure that gzoffset() is correct when appending
- Improve compress() and uncompress() to support large lengths
- Fix bug in test/example.c where error code not saved
- Remedy Coverity warning [Randers-Pehrson]
- Improve speed of gzprintf() in transparent mode
- Fix inflateInit2() bug when windowBits is 16 or 32
- Change DEBUG macro to ZLIB_DEBUG
- Avoid uninitialized access by gzclose_w()
- Allow building zlib outside of the source directory
- Fix bug that accepted invalid zlib header when windowBits is zero
- Fix gzseek() problem on MinGW due to buggy _lseeki64 there
- Loop on write() calls in gzwrite.c in case of non-blocking I/O
- Add --warn (-w) option to ./configure for more compiler warnings
- Reject a window size of 256 bytes if not using the zlib wrapper
- Fix bug when level 0 used with Z_HUFFMAN or Z_RLE
- Add --debug (-d) option to ./configure to define ZLIB_DEBUG
- Fix bugs in creating a very large gzip header
- Add uncompress2() function, which returns the input size used
- Assure that deflateParams() will not switch functions mid-block
- Dramatically speed up deflation for level 0 (storing)
- Add gzfread(), duplicating the interface of fread()
- Add gzfwrite(), duplicating the interface of fwrite()
- Add deflateGetDictionary() function
- Use snprintf() for later versions of Microsoft C
- Fix *Init macros to use z_ prefix when requested
- Replace as400 with os400 for OS/400 support [Monnerat]
- Add crc32_z() and adler32_z() functions with size_t lengths
- Update Visual Studio project files [AraHaan]

----------
assignee: doko
components: Extension Modules
files: zlib-1.2.10.diff
keywords: patch
messages: 284749
nosy: doko, larry
priority: normal
severity: normal
status: open
title: update zlib to 1.2.10
versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7
Added file: http://bugs.python.org/file46161/zlib-1.2.10.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29169>
_______________________________________

From report at bugs.python.org  Thu Jan  5 10:23:30 2017
From: report at bugs.python.org (Simon Schuler)
Date: Thu, 05 Jan 2017 15:23:30 +0000
Subject: [issue29168] multiprocessing pickle error
In-Reply-To: <1483618368.97.0.107401413332.issue29168@psf.upfronthosting.co.za>
Message-ID: <1483629810.88.0.318788504591.issue29168@psf.upfronthosting.co.za>


Simon Schuler added the comment:

I want to handle the logging of the main and all my started processes. They should all log to the same Queue. 

Have a look at the sample.py program. In addition there is a inconsistency in using a multiprocessing pool or just the process class directly. The sample program illustrated this.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29168>
_______________________________________

From report at bugs.python.org  Thu Jan  5 10:33:58 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Thu, 05 Jan 2017 15:33:58 +0000
Subject: [issue29168] multiprocessing pickle error
In-Reply-To: <1483618368.97.0.107401413332.issue29168@psf.upfronthosting.co.za>
Message-ID: <1483630438.95.0.747183196732.issue29168@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

_thread.RLock objects were wrongly pickleable before 3.6.

>>> import pickle, pickletools, _thread
>>> lock = _thread.RLock()
>>> lock.acquire()
True
>>> lock
<locked _thread.RLock object owner=-1219508480 count=1 at 0xb713acb0>
>>> pickle.loads(pickle.dumps(lock))
<unlocked _thread.RLock object owner=0 count=0 at 0xb6facb90>
>>> pickletools.dis(pickletools.optimize(pickle.dumps(lock)))
    0: \x80 PROTO      3
    2: c    GLOBAL     '_thread RLock'
   17: )    EMPTY_TUPLE
   18: \x81 NEWOBJ
   19: .    STOP
highest protocol among opcodes = 2

Actually only an information about the type was saved. The state of the lock was lost. Unpickled lock was not a copy of the original lock, but a newly created object with default initial state.

There were errors in programs that pickled _thread.RLock objects. Just these errors were unnoticed. But programs likely didn't work as expected. Now an exception raised on early stage allows you to rewrite not working code. I don't know whether QueueHandler was designed to be pickleable. As a workaround try to set the lock attribute to None before pickling and set it to threading.RLock() (or call the createLock() method) after unpickling.

This change was made in issue22995.

----------
nosy: +serhiy.storchaka

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29168>
_______________________________________

From report at bugs.python.org  Thu Jan  5 10:40:29 2017
From: report at bugs.python.org (Xavier de Gaye)
Date: Thu, 05 Jan 2017 15:40:29 +0000
Subject: [issue26851] android compilation and link flags
In-Reply-To: <1461673211.54.0.945034926098.issue26851@psf.upfronthosting.co.za>
Message-ID: <1483630829.05.0.126736142631.issue26851@psf.upfronthosting.co.za>


Xavier de Gaye added the comment:

> Just tried. With my packaging scripts, CPython on ARM is compiled with -mfloat-abi=softfp -mfpu=vfpv3-d16 while libffi not. test_ctypes pass as usual.

This works as expected, '-mfloat-abi=softfp' and the default '-mfloat-abi=soft' use the same ABI [1]: "Function calls are generated to pass FP arguments (float, double) in integer registers (one for float, a pair of registers for double)".

[1] https://wiki.debian.org/ArmHardFloatPort/VfpComparison

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue26851>
_______________________________________

From report at bugs.python.org  Thu Jan  5 10:46:33 2017
From: report at bugs.python.org (Simon Schuler)
Date: Thu, 05 Jan 2017 15:46:33 +0000
Subject: [issue29168] multiprocessing pickle error
In-Reply-To: <1483618368.97.0.107401413332.issue29168@psf.upfronthosting.co.za>
Message-ID: <1483631193.55.0.026048020259.issue29168@psf.upfronthosting.co.za>


Simon Schuler added the comment:

I don't have any lock object. I just use the multiprocessing pool and a QueueHandler in order to be able to log from all processes.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29168>
_______________________________________

From report at bugs.python.org  Thu Jan  5 10:51:50 2017
From: report at bugs.python.org (Jeremy Freeman)
Date: Thu, 05 Jan 2017 15:51:50 +0000
Subject: [issue28231] zipfile does not support pathlib
In-Reply-To: <1474445503.11.0.219964117103.issue28231@psf.upfronthosting.co.za>
Message-ID: <1483631510.62.0.706531189302.issue28231@psf.upfronthosting.co.za>


Jeremy Freeman added the comment:

This also affects python 3.5 on Windows and OSX.

Python 3.5.2 (default, Sep 21 2016, 15:07:18)
[GCC 4.2.1 Compatible Apple LLVM 7.3.0 (clang-703.0.31)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from pathlib import Path
>>> import zipfile
>>> f = Path('zipfile.zip')
>>> with zipfile.ZipFile(f) as zf:
... 	pass
...
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/jeremy/.pyenv/versions/3.5.2/lib/python3.5/zipfile.py", line 1026, in __init__
    self._RealGetContents()
  File "/Users/jeremy/.pyenv/versions/3.5.2/lib/python3.5/zipfile.py", line 1089, in _RealGetContents
    endrec = _EndRecData(fp)
  File "/Users/jeremy/.pyenv/versions/3.5.2/lib/python3.5/zipfile.py", line 241, in _EndRecData
    fpin.seek(0, 2)
AttributeError: 'PosixPath' object has no attribute 'seek'

----------
nosy: +jtf621

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28231>
_______________________________________

From report at bugs.python.org  Thu Jan  5 10:54:43 2017
From: report at bugs.python.org (Vinay Sajip)
Date: Thu, 05 Jan 2017 15:54:43 +0000
Subject: [issue29168] multiprocessing pickle error
In-Reply-To: <1483618368.97.0.107401413332.issue29168@psf.upfronthosting.co.za>
Message-ID: <1483631683.1.0.523588728803.issue29168@psf.upfronthosting.co.za>


Vinay Sajip added the comment:

> Have a look at the sample.py program.

It fails on Python 3.4.3 on my Linux system (just renamed to mp_sample.py):

python3 mp_sample.py 
starting (without multiprocessing pool)...
worker process 0
worker process 3
worker process 4
worker process 6
worker process 5
worker process 9
worker process 8
worker process 7
worker process 1
worker process 2
starting (with multiprocessing pool)...
Traceback (most recent call last):
  File "mp_sample.py", line 82, in <module>
    op.start()
  File "mp_sample.py", line 43, in start
    result.get()
  File "/usr/lib/python3.4/multiprocessing/pool.py", line 599, in get
    raise self._value
  File "/usr/lib/python3.4/multiprocessing/pool.py", line 383, in _handle_tasks
    put(task)
  File "/usr/lib/python3.4/multiprocessing/connection.py", line 206, in send
    self._send_bytes(ForkingPickler.dumps(obj))
  File "/usr/lib/python3.4/multiprocessing/reduction.py", line 50, in dumps
    cls(buf, protocol).dump(obj)
  File "/usr/lib/python3.4/multiprocessing/queues.py", line 57, in __getstate__
    context.assert_spawning(self)
  File "/usr/lib/python3.4/multiprocessing/context.py", line 347, in assert_spawning
    ' through inheritance' % type(obj).__name__
RuntimeError: Queue objects should only be shared between processes through inheritance

Also:

QueueHandler isn't designed to be pickleable, nor is that necessary for use with multiprocessing. The logging cookbook contains a working example with QueueHandler and multiple processes using multiprocessing, and I've just run it on Python 3.6 (and 3.7) with no problems. See

https://docs.python.org/3/howto/logging-cookbook.html#logging-to-a-single-file-from-multiple-processes

No idea why the sample program isn't working, and I'm sorry I haven't the time to look at it in detail. Please confirm if the linked sample works on your system, and if it does, see if you can adapt it to your needs. Note that it doesn't pickle any handlers.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29168>
_______________________________________

From report at bugs.python.org  Thu Jan  5 10:56:08 2017
From: report at bugs.python.org (Davin Potts)
Date: Thu, 05 Jan 2017 15:56:08 +0000
Subject: [issue29168] multiprocessing pickle error
In-Reply-To: <1483618368.97.0.107401413332.issue29168@psf.upfronthosting.co.za>
Message-ID: <1483631768.71.0.14461677427.issue29168@psf.upfronthosting.co.za>


Davin Potts added the comment:

Simon:  You do have a lock object inside the logging handler object.  That is what prevents your pickling of the logging handler object.

The background information Serhiy shared is focused on why pickling a _thread.RLock object is problematic and unsupportable.

I do not know enough to understand how you implemented your code that worked under 3.5 but one of the points made by Serhiy's comments is that your code worked _in spite_ of the fact that you did not have a proper RLock inside your logging handler.  I optimistically believe you can implement what you need to work without needing to pickle a logging handler object.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29168>
_______________________________________

From report at bugs.python.org  Thu Jan  5 11:03:13 2017
From: report at bugs.python.org (Xavier de Gaye)
Date: Thu, 05 Jan 2017 16:03:13 +0000
Subject: [issue26865] Meta-issue: support of the android platform
In-Reply-To: <1461685011.99.0.617135447086.issue26865@psf.upfronthosting.co.za>
Message-ID: <1483632193.17.0.475583533803.issue26865@psf.upfronthosting.co.za>


Xavier de Gaye added the comment:

Removing issues from the dependencies list:
    issue #22747: Interpreter fails in initialize on systems where HAVE_LANGINFO_H is undefined
      This issue is fixed for Android.
    issue #26859: unittest fails with "Start directory is not importable" when trying to run sourceless tests
      This issue is out of scope.

----------
dependencies:  -Interpreter fails in initialize on systems where HAVE_LANGINFO_H is undefined, unittest fails with "Start directory is not importable" when trying to run sourceless tests

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue26865>
_______________________________________

From report at bugs.python.org  Thu Jan  5 11:07:17 2017
From: report at bugs.python.org (R. David Murray)
Date: Thu, 05 Jan 2017 16:07:17 +0000
Subject: [issue28231] zipfile does not support pathlib
In-Reply-To: <1474445503.11.0.219964117103.issue28231@psf.upfronthosting.co.za>
Message-ID: <1483632437.65.0.775759112058.issue28231@psf.upfronthosting.co.za>


R. David Murray added the comment:

In 3.5 the stdlib is not supporting PathLib.  So this issue only affects 3.6 and 3.7.

----------
nosy: +r.david.murray

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28231>
_______________________________________

From report at bugs.python.org  Thu Jan  5 11:10:44 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Thu, 05 Jan 2017 16:10:44 +0000
Subject: [issue29034] Fix memory leak and use-after-free in path_converter
In-Reply-To: <1482316498.7.0.932585385435.issue29034@psf.upfronthosting.co.za>
Message-ID: <1483632644.1.0.420859840811.issue29034@psf.upfronthosting.co.za>


Xiang Zhang added the comment:

The new patch is a try to solve the problems mentioned in this thread.

> It can be solved by making path->object referring an original argument (but some code checks the type of path->object), or making path->object owning a reference.

I prefer the later approach. There are codes relying on this behaviour (readlink).

----------
title: Fix memory leak in path_converter -> Fix memory leak and use-after-free in path_converter
type:  -> resource usage
Added file: http://bugs.python.org/file46162/path_converter-new.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29034>
_______________________________________

From report at bugs.python.org  Thu Jan  5 11:26:47 2017
From: report at bugs.python.org (Roundup Robot)
Date: Thu, 05 Jan 2017 16:26:47 +0000
Subject: [issue29169] update zlib to 1.2.10
In-Reply-To: <1483629107.49.0.982374684244.issue29169@psf.upfronthosting.co.za>
Message-ID: <20170105162643.9948.95622.5553CDEB@psf.io>


Roundup Robot added the comment:

New changeset ed172054a812 by doko in branch '2.7':
- Issue #29169: Update zlib to 1.2.10.
https://hg.python.org/cpython/rev/ed172054a812

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29169>
_______________________________________

From report at bugs.python.org  Thu Jan  5 11:59:53 2017
From: report at bugs.python.org (Ethan Furman)
Date: Thu, 05 Jan 2017 16:59:53 +0000
Subject: [issue29167] Race condition in enum.py:_decompose()
In-Reply-To: <1483613666.47.0.301459916652.issue29167@psf.upfronthosting.co.za>
Message-ID: <1483635593.97.0.594700510252.issue29167@psf.upfronthosting.co.za>


Ethan Furman added the comment:

Simon, can you post the exact line of code that causes the error?  It would be useful for creating a test case and the couple things I have tried to duplicate the error have worked fine.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29167>
_______________________________________

From report at bugs.python.org  Thu Jan  5 12:00:19 2017
From: report at bugs.python.org (Ethan Furman)
Date: Thu, 05 Jan 2017 17:00:19 +0000
Subject: [issue29167] Race condition in enum.py:_decompose()
In-Reply-To: <1483613666.47.0.301459916652.issue29167@psf.upfronthosting.co.za>
Message-ID: <1483635619.95.0.587091512239.issue29167@psf.upfronthosting.co.za>


Changes by Ethan Furman <ethan at stoneleaf.us>:


----------
assignee:  -> ethan.furman
stage:  -> test needed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29167>
_______________________________________

From report at bugs.python.org  Thu Jan  5 12:01:29 2017
From: report at bugs.python.org (Jim Nasby)
Date: Thu, 05 Jan 2017 17:01:29 +0000
Subject: [issue29057] Compiler failure on Mac OS X - sys/random.h
In-Reply-To: <1482514649.72.0.460227366509.issue29057@psf.upfronthosting.co.za>
Message-ID: <1483635689.96.0.459099095538.issue29057@psf.upfronthosting.co.za>


Jim Nasby added the comment:

This is still affecting me on 2.7, even with the new changes. 3.7 seems to be OK (I'm getting a different error on 3.7, but it seems unrelated to this.)

OS X 10.11.6.

----------
nosy: +Jim Nasby

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29057>
_______________________________________

From report at bugs.python.org  Thu Jan  5 12:15:02 2017
From: report at bugs.python.org (Vinay Sajip)
Date: Thu, 05 Jan 2017 17:15:02 +0000
Subject: [issue29132] shlex.shlex with punctuation_chars and posix doesn't
 handle punctuation next to quotes
In-Reply-To: <1483361783.89.0.00336686739256.issue29132@psf.upfronthosting.co.za>
Message-ID: <1483636502.56.0.968883297255.issue29132@psf.upfronthosting.co.za>


Vinay Sajip added the comment:

Evan, thanks for looking at this. Would you like to submit a completed contributor form?

https://www.python.org/psf/contrib/contrib-form/

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29132>
_______________________________________

From report at bugs.python.org  Thu Jan  5 12:18:25 2017
From: report at bugs.python.org (STINNER Victor)
Date: Thu, 05 Jan 2017 17:18:25 +0000
Subject: [issue28180] sys.getfilesystemencoding() should default to utf-8
In-Reply-To: <1474024622.65.0.838685461995.issue28180@psf.upfronthosting.co.za>
Message-ID: <1483636705.92.0.477331632272.issue28180@psf.upfronthosting.co.za>


STINNER Victor added the comment:

> That way each PEP can argue as strongly as it can for the respective authors preferred approach to tackling the default C locale problem, even if they point to a common background section in one of the PEPs (similar to the way PEPs 522 and 524 shared a common problem definition, even though they proposed different ways of handling it).

Ok, same players play again: as PEP 522/524 with Nick and me, I just wrote the PEP 540 "Add a new UTF-8 mode" and Nick wrote the PEP 538 :-D

I started a thread to discuss the PEP on python-ideas:
https://mail.python.org/pipermail/python-ideas/2017-January/044089.html

IMHO the PEP 538 should discuss the usage of the surrogateescape error handler: see my second mail in the thread for the details.

I proposed a change in my 3rd mail which would move my PEP closer to Nick's PEP 538: enable "automatically" the UTF-8 mode when the locale is POSIX.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28180>
_______________________________________

From report at bugs.python.org  Thu Jan  5 14:00:30 2017
From: report at bugs.python.org (Jim Nasby)
Date: Thu, 05 Jan 2017 19:00:30 +0000
Subject: [issue29057] Compiler failure on Mac OS X - sys/random.h
In-Reply-To: <1482514649.72.0.460227366509.issue29057@psf.upfronthosting.co.za>
Message-ID: <1483642830.33.0.74121213651.issue29057@psf.upfronthosting.co.za>


Jim Nasby added the comment:

I take it back... turns out I was building with the unsupported macports gcc.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29057>
_______________________________________

From report at bugs.python.org  Thu Jan  5 14:04:41 2017
From: report at bugs.python.org (Larry Hastings)
Date: Thu, 05 Jan 2017 19:04:41 +0000
Subject: [issue29057] Compiler failure on Mac OS X - sys/random.h
In-Reply-To: <1482514649.72.0.460227366509.issue29057@psf.upfronthosting.co.za>
Message-ID: <1483643081.67.0.281508387538.issue29057@psf.upfronthosting.co.za>


Larry Hastings added the comment:

Mr. Nasby, as long as you're in a test-reproducing mood, would you mind downloading the source to 3.5.3rc1 and confirming that it builds correctly for you?  I'd appreciate it!  (Not that I don't trust Ned et al, but independent confirmation always helps soothe these jangled nerves.)

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29057>
_______________________________________

From report at bugs.python.org  Thu Jan  5 14:17:35 2017
From: report at bugs.python.org (Alexander Belopolsky)
Date: Thu, 05 Jan 2017 19:17:35 +0000
Subject: [issue29159] Regression in bytes constructor
In-Reply-To: <1483559394.32.0.335371111214.issue29159@psf.upfronthosting.co.za>
Message-ID: <1483643855.26.0.735993882341.issue29159@psf.upfronthosting.co.za>


Alexander Belopolsky added the comment:

@inada.naoki

Sorry, I still don't understand the role of OverflowError.

With respect to my earlier suggestion that buffer protocol should have priority over __index__, note that the documentation implies the same order:

>>> help(bytes)

class bytes(object)
 |  bytes(iterable_of_ints) -> bytes
 |  bytes(string, encoding[, errors]) -> bytes
 |  bytes(bytes_or_buffer) -> immutable copy of bytes_or_buffer
 |  bytes(int) -> bytes object of size given by the parameter initialized with null bytes
 |  bytes() -> empty bytes object
 |
 |  Construct an immutable array of bytes from:
 |    - an iterable yielding integers in range(256)
 |    - a text string encoded using the specified encoding
 |    - any object implementing the buffer API.
 |    - an integer
..

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29159>
_______________________________________

From report at bugs.python.org  Thu Jan  5 14:36:40 2017
From: report at bugs.python.org (Alexander Belopolsky)
Date: Thu, 05 Jan 2017 19:36:40 +0000
Subject: [issue29159] Regression in bytes constructor
In-Reply-To: <1483559394.32.0.335371111214.issue29159@psf.upfronthosting.co.za>
Message-ID: <1483645000.39.0.512520992092.issue29159@psf.upfronthosting.co.za>


Alexander Belopolsky added the comment:

On the other hand, the documentation for the bytearray constructor [1] lists integer argument first. 

[1]: https://docs.python.org/3/library/functions.html#bytearray

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29159>
_______________________________________

From report at bugs.python.org  Thu Jan  5 14:53:43 2017
From: report at bugs.python.org (Larry Hastings)
Date: Thu, 05 Jan 2017 19:53:43 +0000
Subject: [issue29169] update zlib to 1.2.10
In-Reply-To: <1483629107.49.0.982374684244.issue29169@psf.upfronthosting.co.za>
Message-ID: <1483646023.23.0.192306594715.issue29169@psf.upfronthosting.co.za>


Larry Hastings added the comment:

I cut 3.4.6rc1 and 3.5.3rc1 a couple of days ago.  Do you think the CVEs are bad enough to warrant cherry-picking this?  A quick google suggests they were all low severity:

http://www.openwall.com/lists/oss-security/2016/12/05/21

I'm inclined to not cherry-pick this, which means it'd ship in 3.5.4 and 3.4.7, probably in six months.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29169>
_______________________________________

From report at bugs.python.org  Thu Jan  5 16:10:02 2017
From: report at bugs.python.org (wim glenn)
Date: Thu, 05 Jan 2017 21:10:02 +0000
Subject: [issue28848] Add CopyingMock to mock.py
In-Reply-To: <1480571018.23.0.434543002887.issue28848@psf.upfronthosting.co.za>
Message-ID: <1483650602.83.0.366304569746.issue28848@psf.upfronthosting.co.za>


wim glenn added the comment:

Guess there was no interest in this.   I've released it on pypi instead (pip install copyingmock).  

https://github.com/wimglenn/copyingmock

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28848>
_______________________________________

From report at bugs.python.org  Thu Jan  5 16:57:30 2017
From: report at bugs.python.org (Roy Marples)
Date: Thu, 05 Jan 2017 21:57:30 +0000
Subject: [issue29170] Curses Module should check for is_keypad and not window
 flags
Message-ID: <1483653450.75.0.3170058096.issue29170@psf.upfronthosting.co.za>


New submission from Roy Marples:

checking whether WINDOW has _flags... no

The ncurses library can be built with an opaque window structure.
As such, it's not always possible to query it's flags.
Luckily there is the is_keypad function which works regardless of how ncurses was built.

If this test is changed, then it will help support NetBSD curses as well.

----------
components: Extension Modules
messages: 284771
nosy: Roy Marples
priority: normal
severity: normal
status: open
title: Curses Module should check for is_keypad and not window flags
type: compile error
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29170>
_______________________________________

From report at bugs.python.org  Thu Jan  5 16:59:18 2017
From: report at bugs.python.org (Roundup Robot)
Date: Thu, 05 Jan 2017 21:59:18 +0000
Subject: [issue27961] remove support for platforms without "long long"
In-Reply-To: <1473119429.61.0.683650779093.issue27961@psf.upfronthosting.co.za>
Message-ID: <20170105215915.5864.45835.CDE5D8E9@psf.io>


Roundup Robot added the comment:

New changeset fad67c66885f by Victor Stinner in branch '3.6':
Issue #27961: Define HAVE_LONG_LONG as 1.
https://hg.python.org/cpython/rev/fad67c66885f

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue27961>
_______________________________________

From report at bugs.python.org  Thu Jan  5 17:00:23 2017
From: report at bugs.python.org (STINNER Victor)
Date: Thu, 05 Jan 2017 22:00:23 +0000
Subject: [issue27961] remove support for platforms without "long long"
In-Reply-To: <1473119429.61.0.683650779093.issue27961@psf.upfronthosting.co.za>
Message-ID: <1483653623.09.0.499230569898.issue27961@psf.upfronthosting.co.za>


STINNER Victor added the comment:

> The backwards compatibility is not as strong as it could be: previously, HAVE_LONG_LONG was defined to 1; now it's defined but empty.

Oh, right. This issue should now be fixed as well.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue27961>
_______________________________________

From report at bugs.python.org  Thu Jan  5 17:04:54 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Thu, 05 Jan 2017 22:04:54 +0000
Subject: [issue29159] Regression in bytes constructor
In-Reply-To: <1483559394.32.0.335371111214.issue29159@psf.upfronthosting.co.za>
Message-ID: <1483653894.58.0.639450208838.issue29159@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

The code restored by 29159-index-fallback.patch is slightly different from the code before issue27704, but the patch LGTM except that arguments of assertEqual are in reversed order.

> Can you explain what your patch does?  I don't understand why you single out the OverflowError.  When is __index__ expected to raise it?

PyNumber_AsSsize_t(arg, PyExc_OverflowError) raises an OverflowError when the result of __index__ don't fit in Py_ssize_t.

> I wonder if we could check for buffer protocol support before detecting an integer argument?

This would fix an issue with NumPy arrays, but it is much harder to implement.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29159>
_______________________________________

From report at bugs.python.org  Thu Jan  5 17:15:42 2017
From: report at bugs.python.org (Roy Marples)
Date: Thu, 05 Jan 2017 22:15:42 +0000
Subject: [issue29170] Curses Module should check for is_keypad and not window
 flags
In-Reply-To: <1483653450.75.0.3170058096.issue29170@psf.upfronthosting.co.za>
Message-ID: <1483654542.38.0.843765738798.issue29170@psf.upfronthosting.co.za>


Roy Marples added the comment:

Remove WINDOW test
Add is_keypad test
Change #define in _cursesmodule.c

----------
keywords: +patch
Added file: http://bugs.python.org/file46163/cursesmodule.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29170>
_______________________________________

From report at bugs.python.org  Thu Jan  5 17:22:16 2017
From: report at bugs.python.org (Aviv Palivoda)
Date: Thu, 05 Jan 2017 22:22:16 +0000
Subject: [issue29021] Custom functions in sqlite receive None on invalid UTF-8
In-Reply-To: <1482225891.27.0.11020777474.issue29021@psf.upfronthosting.co.za>
Message-ID: <1483654936.62.0.197040317678.issue29021@psf.upfronthosting.co.za>


Aviv Palivoda added the comment:

I actually was wrong and there is no problem with the type that is returned from the text_factory as it passes directly as argument for the create_function function.

Attached is a patch that change create_function and create_aggregate to use the text_factory when a TEXT data type is received. I am now using sqlite3_create_function_v2 but this is fine because we use sqlite3_stmt_readonly and it is from a newer version.

As for Ingo example code it still don't work with this current fix but this is due to a different problem. Now the _pysqlite_set_result function fail when we do PyUnicode_AsUTF8 on the result from py_identity. As this is a different problem I will fix this in a different patch.

----------
keywords: +patch
Added file: http://bugs.python.org/file46164/29021.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29021>
_______________________________________

From report at bugs.python.org  Thu Jan  5 17:26:44 2017
From: report at bugs.python.org (Masayuki Yamamoto)
Date: Thu, 05 Jan 2017 22:26:44 +0000
Subject: [issue29170] Curses Module should check for is_keypad and not window
 flags
In-Reply-To: <1483653450.75.0.3170058096.issue29170@psf.upfronthosting.co.za>
Message-ID: <1483655204.32.0.77074254322.issue29170@psf.upfronthosting.co.za>


Masayuki Yamamoto added the comment:

#25720 is a similar issue that is originated by opaque type WINDOW.

----------
nosy: +masamoto

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29170>
_______________________________________

From report at bugs.python.org  Thu Jan  5 17:28:58 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Thu, 05 Jan 2017 22:28:58 +0000
Subject: [issue29034] Fix memory leak and use-after-free in path_converter
In-Reply-To: <1482316498.7.0.932585385435.issue29034@psf.upfronthosting.co.za>
Message-ID: <1483655338.07.0.392222128561.issue29034@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

path->length and path->object are set in each way to success_exit (except error_exit). I think these assignment can be moved after success_exit.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29034>
_______________________________________

From report at bugs.python.org  Thu Jan  5 17:36:57 2017
From: report at bugs.python.org (Roy Marples)
Date: Thu, 05 Jan 2017 22:36:57 +0000
Subject: [issue29170] Curses Module should check for is_keypad and not window
 flags
In-Reply-To: <1483653450.75.0.3170058096.issue29170@psf.upfronthosting.co.za>
Message-ID: <1483655817.98.0.70517075582.issue29170@psf.upfronthosting.co.za>


Roy Marples added the comment:

It is, sorry for noise.

----------
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29170>
_______________________________________

From report at bugs.python.org  Thu Jan  5 17:37:59 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Thu, 05 Jan 2017 22:37:59 +0000
Subject: [issue29169] update zlib to 1.2.10
In-Reply-To: <1483629107.49.0.982374684244.issue29169@psf.upfronthosting.co.za>
Message-ID: <1483655879.83.0.792329390832.issue29169@psf.upfronthosting.co.za>


Raymond Hettinger added the comment:

> I'm inclined to not cherry-pick this, which means it'd 
> ship in 3.5.4 and 3.4.7, probably in six months.

I concur.  Looking at the CVEs, these all seem minor and not exploitable through the Python interface.

----------
nosy: +rhettinger

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29169>
_______________________________________

From report at bugs.python.org  Thu Jan  5 17:38:51 2017
From: report at bugs.python.org (Eric N. Vander Weele)
Date: Thu, 05 Jan 2017 22:38:51 +0000
Subject: [issue29171] Remove unused blake2 function to avoid undefined
 references
Message-ID: <1483655931.95.0.807787917048.issue29171@psf.upfronthosting.co.za>


New submission from Eric N. Vander Weele:

Compilers are not required to elide static functions which are unused.

Some compilers, such as Solaris Studio, always emits the function, even
if the function does not get called within the translation unit.  This
becomes problematic when a static inline function calls a non-existent
function; thus, resulting in (dynamic or static) link time errors.

Given that 'blake2' is never referenced nor called, remove the
definition of this function to increase portability for non-Linux
toolchains.

https://blogs.oracle.com/d/entry/inline_functions_in_c also indicates that this is case for Solaris Studio as well.

----------
components: Build
files: blake2-remove-unused-function-1.patch
keywords: patch
messages: 284781
nosy: christian.heimes, ericvw
priority: normal
severity: normal
status: open
title: Remove unused blake2 function to avoid undefined references
type: behavior
versions: Python 3.6
Added file: http://bugs.python.org/file46165/blake2-remove-unused-function-1.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29171>
_______________________________________

From report at bugs.python.org  Thu Jan  5 17:44:13 2017
From: report at bugs.python.org (STINNER Victor)
Date: Thu, 05 Jan 2017 22:44:13 +0000
Subject: [issue28180] sys.getfilesystemencoding() should default to utf-8
In-Reply-To: <1474024622.65.0.838685461995.issue28180@psf.upfronthosting.co.za>
Message-ID: <1483656253.84.0.155103547911.issue28180@psf.upfronthosting.co.za>


STINNER Victor added the comment:

> Working with Docker I often end up with an environment where the locale isn't correctly set.

The locale encoding is controlled by 3 environment variables: LC_ALL, LC_CTYPE and LANG.
https://www.python.org/dev/peps/pep-0540/#the-posix-locale-and-its-encoding

Can you please tell me if these variables are set and if yes, give me their value?

I would like to know if it would be possible to change the behaviour of Python when the (LC_CTYPE) locale is POSIX (aka the famous "C" locale).

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28180>
_______________________________________

From report at bugs.python.org  Thu Jan  5 17:44:42 2017
From: report at bugs.python.org (Eric N. Vander Weele)
Date: Thu, 05 Jan 2017 22:44:42 +0000
Subject: [issue29171] blake2: Remove unused function to avoid undefined
 references
In-Reply-To: <1483655931.95.0.807787917048.issue29171@psf.upfronthosting.co.za>
Message-ID: <1483656282.35.0.710697885844.issue29171@psf.upfronthosting.co.za>


Changes by Eric N. Vander Weele <ericvw at gmail.com>:


----------
title: Remove unused blake2 function to avoid undefined references -> blake2: Remove unused function to avoid undefined references

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29171>
_______________________________________

From report at bugs.python.org  Thu Jan  5 17:51:39 2017
From: report at bugs.python.org (Aviv Palivoda)
Date: Thu, 05 Jan 2017 22:51:39 +0000
Subject: [issue29021] Custom functions in sqlite receive None on invalid UTF-8
In-Reply-To: <1482225891.27.0.11020777474.issue29021@psf.upfronthosting.co.za>
Message-ID: <1483656699.14.0.537052052094.issue29021@psf.upfronthosting.co.za>


Aviv Palivoda added the comment:

Actually had a small mistake in the patch I uploaded. Uploading a fixed one.

----------
Added file: http://bugs.python.org/file46166/29021-fixed.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29021>
_______________________________________

From report at bugs.python.org  Thu Jan  5 17:55:18 2017
From: report at bugs.python.org (Eric N. Vander Weele)
Date: Thu, 05 Jan 2017 22:55:18 +0000
Subject: [issue29172] blake2: Use lowest-common denominator signature of
 #pragma pack
Message-ID: <1483656918.9.0.359858354352.issue29172@psf.upfronthosting.co.za>


New submission from Eric N. Vander Weele:

Solaris Studio emits the following during compilation:

    "/tmp/Python-3.6.0/Modules/_blake2/impl/blake2.h", line 89: warning: ignoring malformed #pragma pack(n)
    "/tmp/Python-3.6.0/Modules/_blake2/impl/blake2.h", line 119: warning: ignoring malformed #pragma pack(n)

To make the usage of '#pragma pack' more portable, change to the
optional, single argument form.

----------
components: Build
files: blake2-pragma-pack-1.patch
keywords: patch
messages: 284784
nosy: christian.heimes, ericvw
priority: normal
severity: normal
status: open
title: blake2: Use lowest-common denominator signature of #pragma pack
type: compile error
versions: Python 3.6
Added file: http://bugs.python.org/file46167/blake2-pragma-pack-1.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29172>
_______________________________________

From report at bugs.python.org  Thu Jan  5 18:05:22 2017
From: report at bugs.python.org (Ezio Melotti)
Date: Thu, 05 Jan 2017 23:05:22 +0000
Subject: [issue2771] Test issue
In-Reply-To: <1210005645.74.0.283923986194.issue2771@psf.upfronthosting.co.za>
Message-ID: <1483657522.03.0.251997162965.issue2771@psf.upfronthosting.co.za>


Ezio Melotti added the comment:

testing github integration

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue2771>
_______________________________________

From report at bugs.python.org  Thu Jan  5 18:06:50 2017
From: report at bugs.python.org (Ezio Melotti)
Date: Thu, 05 Jan 2017 23:06:50 +0000
Subject: [issue2771] Test issue
In-Reply-To: <1483657522.03.0.251997162965.issue2771@psf.upfronthosting.co.za>
Message-ID: <CACBhJdE0f4s-mB3jYYisEvd-HphaNv0DBOp5VErKGK8e7WDJnw@mail.gmail.com>


Ezio Melotti added the comment:

another test

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue2771>
_______________________________________

From report at bugs.python.org  Thu Jan  5 18:07:47 2017
From: report at bugs.python.org (Milan Oberkirch)
Date: Thu, 05 Jan 2017 23:07:47 +0000
Subject: [issue29142] urllib: no_proxy variable values with leading dot not
 properly handled
In-Reply-To: <1483452589.12.0.177615671636.issue29142@psf.upfronthosting.co.za>
Message-ID: <1483657667.35.0.255301691194.issue29142@psf.upfronthosting.co.za>


Changes by Milan Oberkirch <milan.py at oberkirch.org>:


----------
type: behavior -> 
versions:  -Python 3.5, Python 3.6, Python 3.7
Added file: http://bugs.python.org/file46168/ignore_dots-v2.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29142>
_______________________________________

From report at bugs.python.org  Thu Jan  5 18:28:45 2017
From: report at bugs.python.org (Prahlad Yeri)
Date: Thu, 05 Jan 2017 23:28:45 +0000
Subject: [issue29173] Python 3.6 on Windows wastes a lot of CPU cycles in a
 while loop
Message-ID: <1483658925.81.0.717567578019.issue29173@psf.upfronthosting.co.za>


New submission from Prahlad Yeri:

I'm running Python 3.6 on Windows 7. It wastes a lot of cpu cycles when inside a loop. In attached screenshot, you'll find that ~25% cpu is used by Python, while all I'm doing is running a pomodoro script that waits for a specific timeslot to complete. Here is the code for pomodoro.py that runs the while loop inside start_tracking() function:

https://github.com/prahladyeri/PyPomodoro/blob/master/pomodoro.py



def start_tracking(task):
	global last_beep
	print("Working on %s:%s (%d minutes)." % (task['category'], task['name'], task['duration']))
	print("Started tracking at %s." % (datetime.datetime.now().strftime("%H:%M")))
	print("Beep interval is set to %d minutes." % config.slot_interval)
	session_start_time = datetime.datetime.now()
	task_end_time = session_start_time + datetime.timedelta(minutes=task['duration'])
	last_beep = datetime.datetime.now()
	notified = False
	while(True):
		now = datetime.datetime.now()
		diff = (now - last_beep).total_seconds() / 60.0 #in minutes
		minutes_worked = round(diff)
		reminder_text = "%s:%s" % (task['category'], task['name'])
		#diff = diff.total_seconds() / (60.0) 
		if (diff >= config.slot_interval): #30
			#notify

----------
components: Windows
files: Python_high_CPU_Windows.png
messages: 284787
nosy: paul.moore, prahladyeri, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Python 3.6 on Windows wastes a lot of CPU cycles in a while loop
type: resource usage
versions: Python 3.6
Added file: http://bugs.python.org/file46169/Python_high_CPU_Windows.png

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29173>
_______________________________________

From report at bugs.python.org  Thu Jan  5 18:32:50 2017
From: report at bugs.python.org (STINNER Victor)
Date: Thu, 05 Jan 2017 23:32:50 +0000
Subject: [issue29173] Python 3.6 on Windows wastes a lot of CPU cycles in a
 while loop
In-Reply-To: <1483658925.81.0.717567578019.issue29173@psf.upfronthosting.co.za>
Message-ID: <1483659170.76.0.181871934426.issue29173@psf.upfronthosting.co.za>


STINNER Victor added the comment:

The code doesn't seem like a bug in Python, but more a classic inefficient busy loop pattern. Your loop doesn't sleep and so eats all the CPU.

I suggest to close the issue and ask Python questions on a place to ask questions, not on the Python *bug tracker*.

----------
nosy: +haypo

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29173>
_______________________________________

From report at bugs.python.org  Thu Jan  5 18:43:52 2017
From: report at bugs.python.org (Prahlad Yeri)
Date: Thu, 05 Jan 2017 23:43:52 +0000
Subject: [issue29173] Python 3.6 on Windows wastes a lot of CPU cycles in a
 while loop
In-Reply-To: <1483658925.81.0.717567578019.issue29173@psf.upfronthosting.co.za>
Message-ID: <1483659832.17.0.460186987046.issue29173@psf.upfronthosting.co.za>


Prahlad Yeri added the comment:

Hi STINNER Victor,

Thanks a lot, after adding the sleep function, it has stopped wasting the CPU cycles! A long while ago, I remember coding a similar loop in linux and not faced such issue, so I thought maybe it was a bug in 3.6.

Anyway, I'm closing this.

----------
resolution:  -> not a bug
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29173>
_______________________________________

From report at bugs.python.org  Thu Jan  5 19:16:04 2017
From: report at bugs.python.org (Eryk Sun)
Date: Fri, 06 Jan 2017 00:16:04 +0000
Subject: [issue29173] Python 3.6 on Windows wastes a lot of CPU cycles in a
 while loop
In-Reply-To: <1483658925.81.0.717567578019.issue29173@psf.upfronthosting.co.za>
Message-ID: <1483661764.7.0.323614104874.issue29173@psf.upfronthosting.co.za>


Changes by Eryk Sun <eryksun at gmail.com>:


----------
stage:  -> resolved

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29173>
_______________________________________

From report at bugs.python.org  Thu Jan  5 19:17:12 2017
From: report at bugs.python.org (Berker Peksag)
Date: Fri, 06 Jan 2017 00:17:12 +0000
Subject: [issue29142] urllib: no_proxy variable values with leading dot not
 properly handled
In-Reply-To: <1483452589.12.0.177615671636.issue29142@psf.upfronthosting.co.za>
Message-ID: <1483661832.8.0.9579198801.issue29142@psf.upfronthosting.co.za>


Changes by Berker Peksag <berker.peksag at gmail.com>:


----------
type:  -> behavior
versions: +Python 3.5, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29142>
_______________________________________

From report at bugs.python.org  Thu Jan  5 19:24:30 2017
From: report at bugs.python.org (Berker Peksag)
Date: Fri, 06 Jan 2017 00:24:30 +0000
Subject: [issue28848] Add CopyingMock to mock.py
In-Reply-To: <1480571018.23.0.434543002887.issue28848@psf.upfronthosting.co.za>
Message-ID: <1483662270.12.0.957051655749.issue28848@psf.upfronthosting.co.za>


Berker Peksag added the comment:

Thanks for releasing it on PyPI! It's probably better to keep it as an example in the documentation for now. We can reopen this if there's a demand for CopyingMock in the future.

----------
nosy: +berker.peksag
resolution:  -> rejected
stage:  -> resolved
status: open -> closed
type:  -> enhancement

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28848>
_______________________________________

From report at bugs.python.org  Thu Jan  5 19:56:07 2017
From: report at bugs.python.org (Jeremy Freeman)
Date: Fri, 06 Jan 2017 00:56:07 +0000
Subject: [issue28231] zipfile does not support pathlib
In-Reply-To: <1474445503.11.0.219964117103.issue28231@psf.upfronthosting.co.za>
Message-ID: <1483664167.72.0.217143112829.issue28231@psf.upfronthosting.co.za>


Jeremy Freeman added the comment:

OK, I understand.  How can I help get this issue fixed?

1) review the patch?
2) update the docs to reflect the patch?
3) find the other uses of pathlib in the zipfile module?
4) something else ...

I am a longtime user of python but a first time contributor but happy to help.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28231>
_______________________________________

From report at bugs.python.org  Thu Jan  5 20:06:24 2017
From: report at bugs.python.org (Berker Peksag)
Date: Fri, 06 Jan 2017 01:06:24 +0000
Subject: [issue28231] zipfile does not support pathlib
In-Reply-To: <1474445503.11.0.219964117103.issue28231@psf.upfronthosting.co.za>
Message-ID: <1483664784.23.0.578389134728.issue28231@psf.upfronthosting.co.za>


Berker Peksag added the comment:

I think the next steps are:

1. Add tests for the cases Serhiy has mentioned in msg277109:

   > Shouldn't the ZipFile.filename attribute be converted to str?

   and

   > The file name of ZipFile is only a part of the issue. There are other uses of file paths: paths for added files, path for extracted directory.

2. Update the docs to reflect the pathlib support.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28231>
_______________________________________

From report at bugs.python.org  Thu Jan  5 21:03:07 2017
From: report at bugs.python.org (Martin Panter)
Date: Fri, 06 Jan 2017 02:03:07 +0000
Subject: [issue29172] blake2: Use lowest-common denominator signature of
 #pragma pack
In-Reply-To: <1483656918.9.0.359858354352.issue29172@psf.upfronthosting.co.za>
Message-ID: <1483668187.08.0.430688176734.issue29172@psf.upfronthosting.co.za>


Martin Panter added the comment:

FWIW Issue 28290 was also opened about this pragma directive being not recognized by a compiler called xlC on AIX.

----------
nosy: +martin.panter

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29172>
_______________________________________

From report at bugs.python.org  Thu Jan  5 21:17:02 2017
From: report at bugs.python.org (Berker Peksag)
Date: Fri, 06 Jan 2017 02:17:02 +0000
Subject: [issue29170] Curses Module should check for is_keypad and not window
 flags
In-Reply-To: <1483653450.75.0.3170058096.issue29170@psf.upfronthosting.co.za>
Message-ID: <1483669022.8.0.859135420604.issue29170@psf.upfronthosting.co.za>


Changes by Berker Peksag <berker.peksag at gmail.com>:


----------
resolution:  -> duplicate
stage:  -> resolved
superseder:  -> Fix curses module compilation with ncurses6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29170>
_______________________________________

From report at bugs.python.org  Thu Jan  5 21:50:53 2017
From: report at bugs.python.org (Nick Coghlan)
Date: Fri, 06 Jan 2017 02:50:53 +0000
Subject: [issue28180] sys.getfilesystemencoding() should default to utf-8
In-Reply-To: <1474024622.65.0.838685461995.issue28180@psf.upfronthosting.co.za>
Message-ID: <1483671053.45.0.732754313997.issue28180@psf.upfronthosting.co.za>


Nick Coghlan added the comment:

Docker containers don't have a locale set by default - the approach proposed in PEP 528 actually comes from the way I configure Docker images (which in turn comes from Armin Ronacher's recommendations in click for Python 3 locale handling).

In the Dockerfile for Fedora based containers I add:

    ENV LC_ALL=C.UTF-8
    ENV LANG=C.UTF-8

while in CentOS 7 based containers I add:

    ENV LC_ALL=en_US.UTF-8
    ENV LANG=en_US.UTF-8

And with those settings, Python 3 based containers just work (my laptop is running en_AU.UTF-8 locally)

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28180>
_______________________________________

From report at bugs.python.org  Thu Jan  5 22:07:56 2017
From: report at bugs.python.org (Nick Coghlan)
Date: Fri, 06 Jan 2017 03:07:56 +0000
Subject: [issue28180] sys.getfilesystemencoding() should default to utf-8
In-Reply-To: <1474024622.65.0.838685461995.issue28180@psf.upfronthosting.co.za>
Message-ID: <1483672076.51.0.316078181198.issue28180@psf.upfronthosting.co.za>


Nick Coghlan added the comment:

And by PEP 528, I actually mean PEP 538 :)

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28180>
_______________________________________

From report at bugs.python.org  Fri Jan  6 00:33:03 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Fri, 06 Jan 2017 05:33:03 +0000
Subject: [issue29034] Fix memory leak and use-after-free in path_converter
In-Reply-To: <1482316498.7.0.932585385435.issue29034@psf.upfronthosting.co.za>
Message-ID: <1483680783.47.0.778572094471.issue29034@psf.upfronthosting.co.za>


Xiang Zhang added the comment:

path_converter-new-2.patch addresses Serhiy's comments. :-)

----------
Added file: http://bugs.python.org/file46170/path_converter-new-2.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29034>
_______________________________________

From report at bugs.python.org  Fri Jan  6 01:23:36 2017
From: report at bugs.python.org (Matthias Klose)
Date: Fri, 06 Jan 2017 06:23:36 +0000
Subject: [issue29169] update zlib to 1.2.10
In-Reply-To: <1483629107.49.0.982374684244.issue29169@psf.upfronthosting.co.za>
Message-ID: <1483683816.34.0.4339139208.issue29169@psf.upfronthosting.co.za>


Matthias Klose added the comment:

ok, will wait with the commits until after the releases.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29169>
_______________________________________

From report at bugs.python.org  Fri Jan  6 02:48:58 2017
From: report at bugs.python.org (ita1024)
Date: Fri, 06 Jan 2017 07:48:58 +0000
Subject: [issue29174] 'NoneType' object is not callable in subprocess.py
Message-ID: <1483688938.47.0.507597836624.issue29174@psf.upfronthosting.co.za>


New submission from ita1024:

Please try the attached testcase with `python3.6 test.py`; Python 3.6 displays unnecessary warnings of the following form:

$ ../test.py
Exception ignored in: <bound method Popen.__del__ of <subprocess.Popen object at 0x7fe6744fc5c0>>
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/subprocess.py", line 761, in __del__
TypeError: 'NoneType' object is not callable
Exception ignored in: <bound method Popen.__del__ of <subprocess.Popen object at 0x7fe6744fc550>>
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/subprocess.py", line 761, in __del__

These warnings appear because of the line "warnings.warn" recently added in subprocess.Popen.__del__:
1. The call to warnings.warn is not usable during interpreter shutdown (and running `python -W ignore test.py` has no effect)
2. Calling "process.terminate()" or "process.kill()" at in the testcase or in an atexit handler would not get rid of the warning, one must set the return code on the Popen object
3. The warning can show up in existing code that has absolutely no zombie problems.

I suggest to revert the recently added warning from subprocess.py:

"""
@@ -754,11 +995,6 @@
         if not self._child_created:
             # We didn't get to successfully create a child process.
             return
-        if self.returncode is None:
-            # Not reading subprocess exit status creates a zombi process which
-            # is only destroyed at the parent python process exit
-            warnings.warn("subprocess %s is still running" % self.pid,
-                          ResourceWarning, source=self)
         # In case the child hasn't been waited on, check if it's done.
         self._internal_poll(_deadstate=_maxsize)
         if self.returncode is None and _active is not None:
"""

----------
components: Library (Lib)
files: test.py
messages: 284798
nosy: ita1024
priority: normal
severity: normal
status: open
title: 'NoneType' object is not callable in subprocess.py
type: behavior
versions: Python 3.6
Added file: http://bugs.python.org/file46171/test.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29174>
_______________________________________

From report at bugs.python.org  Fri Jan  6 02:51:04 2017
From: report at bugs.python.org (Jan Niklas Hasse)
Date: Fri, 06 Jan 2017 07:51:04 +0000
Subject: [issue28180] sys.getfilesystemencoding() should default to utf-8
In-Reply-To: <1474024622.65.0.838685461995.issue28180@psf.upfronthosting.co.za>
Message-ID: <1483689064.15.0.0306186676208.issue28180@psf.upfronthosting.co.za>


Jan Niklas Hasse added the comment:

> Can you please tell me if these variables are set and if yes, give me their value?

None of these variables are set (with `docker run -it fedora:25 /bin/bash`).

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28180>
_______________________________________

From report at bugs.python.org  Fri Jan  6 02:53:01 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Fri, 06 Jan 2017 07:53:01 +0000
Subject: [issue29174] 'NoneType' object is not callable in subprocess.py
In-Reply-To: <1483688938.47.0.507597836624.issue29174@psf.upfronthosting.co.za>
Message-ID: <1483689181.65.0.230076040391.issue29174@psf.upfronthosting.co.za>


Changes by Xiang Zhang <angwerzx at 126.com>:


----------
nosy: +haypo

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29174>
_______________________________________

From report at bugs.python.org  Fri Jan  6 03:28:54 2017
From: report at bugs.python.org (Michael Felt)
Date: Fri, 06 Jan 2017 08:28:54 +0000
Subject: [issue27632] build on AIX fails when builddir != srcdir,
 more than bad path to ld_so_aix
In-Reply-To: <1469608112.86.0.803958819971.issue27632@psf.upfronthosting.co.za>
Message-ID: <1483691334.78.0.229006067941.issue27632@psf.upfronthosting.co.za>


Michael Felt added the comment:

My - bad - this was largely a duplicate of three now closed issues: issue10656, issue16189, and issue25825

as far as the build process goes the issue is fixed. However, issue10656 is not yet "fully-finished" because "make distclean" does not complete successfully. Among other things it leaves Makefile and config.status behind.

Question: re: issue10656 - shall I open a new issue, or reply (reopen) the original?

issue16189 and issue25825 are largely duplicates, as far as I can tell. And unfortunately - "neither" is repaired.

There are three (environment) variables that are used: BLDSHARED (during build - seems fixed), LDSHARED and LDCXXSHARED.

Short summary:
BLDSHARED is fixed per issue10656
LDSHARED is changed (issue16189 and issue25825), but still broken
LDCXXSHARED - was, and is - correct in _sysconfigdata.py

Here is what I have in the 'installed' state:

PATCHED by hand!
root at x064:[/data/prj/python/python-2.7.13]grep LDS /opt/lib/python2.7/_sysconfigdata.py
 'BLDSHARED': './Modules/ld_so_aix xlc_r -bI:./Modules/python.exp -L/opt/lib',
 'LDSHARED': '/opt/lib/python2.7/config/ld_so_aix xlc_r -bI:/opt/lib/python2.7/config/python.exp',

What comes with 2.7.13 by default:
root at x064:[/data/prj/python/python-2.7.13]grep LDS /opt/lib/python2.7/_sysconfigdata.py
 'BLDSHARED': 'Modules/ld_so_aix xlc_r -bI:Modules/python.exp -L/opt/lib',
 'LDSHARED': 'Modules/ld_so_aix xlc_r -bI:Modules/python.exp -L/opt/lib',

This is where ld_so_aix is  'installed':
 /opt/lib/python2.7/config/ld_so_aix         aixtools.python.rte   File

As python does now about where everything else in /opt/lib/pythonX.Y is,
I think this still needs a small change - Modules/* is wrong in any case.

FYI: In both cases this parameter is correct!
 'LDCXXSHARED': '/opt/lib/python2.7/config/ld_so_aix xlc_r -bI:/opt/lib/python2.7/config/python.exp',

Maybe this helps identify what needs fixing!

Moving forward:
a) close this issue - most of it is a duplicate - I will open a new issue for the issue (read my question) re: the cause/importance of all the 'Skipping' (an 'issue' because I see it once, or not at all, and then it goes away)
b) comment on whether I should open a new issue, or comment to the closed issues.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue27632>
_______________________________________

From report at bugs.python.org  Fri Jan  6 03:45:05 2017
From: report at bugs.python.org (Roundup Robot)
Date: Fri, 06 Jan 2017 08:45:05 +0000
Subject: [issue29159] Regression in bytes constructor
In-Reply-To: <1483559394.32.0.335371111214.issue29159@psf.upfronthosting.co.za>
Message-ID: <20170106084502.6251.8430.F205976D@psf.io>


Roundup Robot added the comment:

New changeset 505cc50ddc82 by INADA Naoki in branch '3.6':
Issue #29159: Fix regression in bytes(x) when x.__index__() raises Exception.
https://hg.python.org/cpython/rev/505cc50ddc82

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29159>
_______________________________________

From report at bugs.python.org  Fri Jan  6 03:46:25 2017
From: report at bugs.python.org (INADA Naoki)
Date: Fri, 06 Jan 2017 08:46:25 +0000
Subject: [issue29159] Regression in bytes constructor
In-Reply-To: <1483559394.32.0.335371111214.issue29159@psf.upfronthosting.co.za>
Message-ID: <1483692385.29.0.40813704751.issue29159@psf.upfronthosting.co.za>


INADA Naoki added the comment:

> patch LGTM except that arguments of assertEqual are in reversed order.

fixed.

----------
resolution:  -> fixed
stage:  -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29159>
_______________________________________

From report at bugs.python.org  Fri Jan  6 04:50:28 2017
From: report at bugs.python.org (Roundup Robot)
Date: Fri, 06 Jan 2017 09:50:28 +0000
Subject: [issue29174] 'NoneType' object is not callable in subprocess.py
In-Reply-To: <1483688938.47.0.507597836624.issue29174@psf.upfronthosting.co.za>
Message-ID: <20170106095025.24017.92558.F9F186E5@psf.io>


Roundup Robot added the comment:

New changeset b14a1e81c34a by Victor Stinner in branch '3.6':
Fix subprocess.Popen.__del__() fox Python shutdown
https://hg.python.org/cpython/rev/b14a1e81c34a

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29174>
_______________________________________

From report at bugs.python.org  Fri Jan  6 04:50:28 2017
From: report at bugs.python.org (Roundup Robot)
Date: Fri, 06 Jan 2017 09:50:28 +0000
Subject: [issue26741] subprocess.Popen should emit a ResourceWarning in
 destructor if the process is still running
In-Reply-To: <1460501347.41.0.70782898945.issue26741@psf.upfronthosting.co.za>
Message-ID: <20170106095025.24017.91007.46C9FE97@psf.io>


Roundup Robot added the comment:

New changeset b14a1e81c34a by Victor Stinner in branch '3.6':
Fix subprocess.Popen.__del__() fox Python shutdown
https://hg.python.org/cpython/rev/b14a1e81c34a

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue26741>
_______________________________________

From report at bugs.python.org  Fri Jan  6 04:53:55 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Fri, 06 Jan 2017 09:53:55 +0000
Subject: [issue29157] random.c: Prefer getrandom() over getentropy(),
 handle ENOSYS in py_getentropy()
In-Reply-To: <1483551181.22.0.980001779001.issue29157@psf.upfronthosting.co.za>
Message-ID: <1483696435.91.0.261375703975.issue29157@psf.upfronthosting.co.za>


Xiang Zhang added the comment:

LGTM.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29157>
_______________________________________

From report at bugs.python.org  Fri Jan  6 04:58:43 2017
From: report at bugs.python.org (Jonathan Roach)
Date: Fri, 06 Jan 2017 09:58:43 +0000
Subject: [issue29175] Documentation for File objects missing
Message-ID: <1483696723.95.0.735667565924.issue29175@psf.upfronthosting.co.za>


New submission from Jonathan Roach:

The file documentation (as seen in python.org) went missing in 3.3. In 2.7.13 it can be found in 'The Python Standard Library' as section '5.9 File Objects', between '5.8 Mapping Types' and '5.10 memory view type'. In 3.3 and onwards this section is no longer present, and, as far as I can tell, is not present elsewhere in the documentation.
[related: in 'The Python Tutorial' section '7.2.1 Methods of File Objects' the inline references to the file object methods are shown as references, but without a target]

----------
assignee: docs at python
components: Documentation
messages: 284806
nosy: docs at python, eric.araujo, ezio.melotti, georg.brandl, jonroach at icloud.com
priority: normal
severity: normal
status: open
title: Documentation for File objects missing
type: behavior
versions: Python 3.3, Python 3.4, Python 3.5, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29175>
_______________________________________

From report at bugs.python.org  Fri Jan  6 05:09:09 2017
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 06 Jan 2017 10:09:09 +0000
Subject: [issue29174] 'NoneType' object is not callable in subprocess.py
In-Reply-To: <1483688938.47.0.507597836624.issue29174@psf.upfronthosting.co.za>
Message-ID: <1483697349.86.0.917196654815.issue29174@psf.upfronthosting.co.za>


STINNER Victor added the comment:

> 1. The call to warnings.warn is not usable during interpreter shutdown (and running `python -W ignore test.py` has no effect)

Oops right. I just fixed this issue.


> 2. Calling "process.terminate()" or "process.kill()" at in the testcase or in an atexit handler would not get rid of the warning, one must set the return code on the Popen object

Hum. I should document somehow how to fix such bug: you must read the exit status of the child process, not set manually the returncode attribute. You have to call the wait() method of each process after calling terminate().


> 3. The warning can show up in existing code that has absolutely no zombie problems.

I modified your example to list zombi processes: try test2.py.

Output:

$ ./python test2.py 
0  1000 25520 25120  20   0 140940 11828 wait   S+   pts/0      0:00 ./python test2.py
0  1000 25521 25520  20   0      0     0 -      Z+   pts/0      0:00 [python] <defunct>
0  1000 25522 25520  20   0      0     0 -      Z+   pts/0      0:00 [python] <defunct>
0  1000 25523 25520  20   0      0     0 -      Z+   pts/0      0:00 [python] <defunct>
0  1000 25524 25520  20   0      0     0 -      Z+   pts/0      0:00 [python] <defunct>
0  1000 25525 25520  20   0      0     0 -      Z+   pts/0      0:00 [python] <defunct>
0  1000 25526 25520  20   0      0     0 -      Z+   pts/0      0:00 [python] <defunct>
0  1000 25527 25520  20   0      0     0 -      Z+   pts/0      0:00 [python] <defunct>
0  1000 25528 25520  20   0      0     0 -      Z+   pts/0      0:00 [python] <defunct>
0  1000 25529 25520  20   0 119032  3008 wait   S+   pts/0      0:00 sh -c ps l|grep 25520
0  1000 25531 25529  20   0 118540   880 -      S+   pts/0      0:00 grep 25520
Lib/subprocess.py:761: ResourceWarning: subprocess 25528 is still running
sys:1: ResourceWarning: unclosed file <_io.FileIO name=18 mode='wb' closefd=True>
sys:1: ResourceWarning: unclosed file <_io.FileIO name=19 mode='rb' closefd=True>
(...)

The long list of <defunct> are the zombi processes: it means that the kernel is unable to remove completely child processes because the parent didn't read the exit status yet. Process identifiers and memory are wasted.

The warning can also help to detect when an application forgot to check the exit status. At least, if you add a call to process.wait(), it becomes explicit that ignoring the exit status is deliberate.

----------
nosy: +martin.panter
Added file: http://bugs.python.org/file46172/test2.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29174>
_______________________________________

From report at bugs.python.org  Fri Jan  6 05:40:39 2017
From: report at bugs.python.org (Roundup Robot)
Date: Fri, 06 Jan 2017 10:40:39 +0000
Subject: [issue29157] random.c: Prefer getrandom() over getentropy(),
 handle ENOSYS in py_getentropy()
In-Reply-To: <1483551181.22.0.980001779001.issue29157@psf.upfronthosting.co.za>
Message-ID: <20170106104036.6251.42636.73C14B2D@psf.io>


Roundup Robot added the comment:

New changeset 140f0459fe80 by Victor Stinner in branch 'default':
Issue #29157: dev_urandom() now calls py_getentropy()
https://hg.python.org/cpython/rev/140f0459fe80

New changeset 69b23952d122 by Victor Stinner in branch 'default':
Issue #29157: Simplify dev_urandom()
https://hg.python.org/cpython/rev/69b23952d122

New changeset 46ca697c6f26 by Victor Stinner in branch 'default':
Issue #29157: getrandom() is now preferred over getentropy()
https://hg.python.org/cpython/rev/46ca697c6f26

New changeset 4c11a01fa881 by Victor Stinner in branch 'default':
py_getentropy() now supports ENOSYS, EPERM & EINTR
https://hg.python.org/cpython/rev/4c11a01fa881

New changeset 4edd6cbf9abf by Victor Stinner in branch 'default':
Issue #29157: enhance py_getrandom() documentation
https://hg.python.org/cpython/rev/4edd6cbf9abf

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29157>
_______________________________________

From report at bugs.python.org  Fri Jan  6 05:43:23 2017
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 06 Jan 2017 10:43:23 +0000
Subject: [issue29157] random.c: Prefer getrandom() over getentropy(),
 handle ENOSYS in py_getentropy()
In-Reply-To: <1483551181.22.0.980001779001.issue29157@psf.upfronthosting.co.za>
Message-ID: <1483699403.03.0.511837306742.issue29157@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Thanks Nick and Xiang for the review. I splitted again the giant change into small atomic changes, easier to review and understand.

Right now, I only applied the change to default (Python 3.7). I will now wait for buildbots before considering backporting the change.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29157>
_______________________________________

From report at bugs.python.org  Fri Jan  6 05:47:00 2017
From: report at bugs.python.org (Xavier de Gaye)
Date: Fri, 06 Jan 2017 10:47:00 +0000
Subject: [issue29176] /tmp does not exist on Android and is used by
 curses.window.putwin()
Message-ID: <1483699620.77.0.827616599041.issue29176@psf.upfronthosting.co.za>


New submission from Xavier de Gaye:

======================================================================
ERROR: test_module_funcs (test.test_curses.TestCurses)
Test module-level functions
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/test/test_curses.py", line 219, in test_module_fun
cs
    self.stdscr.putwin(f)
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/py.curses.putwin.nsIZYY'

----------------------------------------------------------------------

----------
assignee: xdegaye
components: Extension Modules
keywords: patch
messages: 284810
nosy: twouters, xdegaye
priority: normal
severity: normal
stage: needs patch
status: open
title: /tmp does not exist on Android and is used by curses.window.putwin()
type: behavior
versions: Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29176>
_______________________________________

From report at bugs.python.org  Fri Jan  6 06:05:30 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Fri, 06 Jan 2017 11:05:30 +0000
Subject: [issue29142] urllib: no_proxy variable values with leading dot not
 properly handled
In-Reply-To: <1483452589.12.0.177615671636.issue29142@psf.upfronthosting.co.za>
Message-ID: <1483700730.69.0.0967168062635.issue29142@psf.upfronthosting.co.za>


Xiang Zhang added the comment:

> I think it makes sense to strip at the end as well (`example.com` is the same domain as `example.com.`).

Are your sure the host checked against the list is FQDN? With and without the trailing dot are different.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29142>
_______________________________________

From report at bugs.python.org  Fri Jan  6 06:14:22 2017
From: report at bugs.python.org (Xavier de Gaye)
Date: Fri, 06 Jan 2017 11:14:22 +0000
Subject: [issue29177] skip tests using socketserver.UnixStreamServer when
 bind() raises PermissionError
Message-ID: <1483701262.25.0.0295401910206.issue29177@psf.upfronthosting.co.za>


New submission from Xavier de Gaye:

This happens on Android for a non-root user.  One test in test_logging fails. Multiple tests fail in test_socketserver with identical backtraces, only the first one is listed here.

======================================================================                   [1955/2616]
ERROR: test_noserver (test.test_logging.UnixSocketHandlerTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/test/test_logging.py", line 1527, in setUp
    SocketHandlerTest.setUp(self)
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/test/test_logging.py", line 1444, in setUp
    self.handle_socket, 0.01)
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/test/test_logging.py", line 885, in __init__
    bind_and_activate)
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/socketserver.py", line 452, in __init__
    self.server_bind()
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/test/test_logging.py", line 889, in server_bind
    super(TestTCPServer, self).server_bind()
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/socketserver.py", line 466, in server_bind
    self.socket.bind(self.server_address)
PermissionError: [Errno 13] Permission denied

======================================================================                    [905/2616]
ERROR: test_ForkingUnixDatagramServer (test.test_socketserver.SocketServerTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/test/test_socketserver.py", line 243, in test_Fork
ingUnixDatagramServer
    self.dgram_examine)
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/test/support/__init__.py", line 2040, in decorator
    return func(*args)
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/test/test_socketserver.py", line 121, in run_serve
r
    svrcls, hdlrbase)
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/test/test_socketserver.py", line 114, in make_serv
er
    server = MyServer(addr, MyHandler)
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/socketserver.py", line 452, in __init__
    self.server_bind()
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/socketserver.py", line 466, in server_bind
    self.socket.bind(self.server_address)
PermissionError: [Errno 13] Permission denied

----------
assignee: xdegaye
components: Tests
messages: 284812
nosy: xdegaye
priority: normal
severity: normal
stage: needs patch
status: open
title: skip tests using socketserver.UnixStreamServer when bind() raises PermissionError
type: behavior
versions: Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29177>
_______________________________________

From report at bugs.python.org  Fri Jan  6 06:16:17 2017
From: report at bugs.python.org (INADA Naoki)
Date: Fri, 06 Jan 2017 11:16:17 +0000
Subject: [issue29178] Adding bytes.frombuffer(byteslike) constructor
Message-ID: <1483701377.3.0.893514886442.issue29178@psf.upfronthosting.co.za>


New submission from INADA Naoki:

# Summary

## 1. Making bytes from slice of bytearray easy and efficient.

bs = bytes(memoryview(bytelike)[start:end]) works fine on CPython,
but it will cause issue on PyPy.

Since memoryview is not closed explicitly, exception like
"BufferError: Existing exports of data: object cannot be re-sized"
will be raised after it.
Where exception raised can be far from where unclosed memoryview is leaked.


## 2. bytes(x) constructor is too overloaded.

It has undocumented corner cases. See PEP 467 and #29159


# ML threads

https://mail.python.org/pipermail/python-dev/2016-October/146668.html
https://mail.python.org/pipermail/python-dev/2017-January/147109.html

+1 from: Nathaniel Smith, Alexander Belopolsky, Yury Selivanov
-1 from: Nick Coghlan

Nick proposed put it on separated module, instead of adding it as builtin method.


# First draft patch

bytes-frombuffer.patch is first draft patch.  It implements frombuffer to only bytes,
with signature proposed first. Only C-contiguous buffer is supported for now.

  frombuffer(byteslike, length=-1, offset=0) method of builtins.type instance
    Create a bytes object from bytes-like object.

    Examples:
        bytes.frombuffer(b'abcd') -> b'abcd'
        bytes.frombuffer(b'abcd', 2) -> b'ab'
        bytes.frombuffer(b'abcd', 8) -> b'abcd'
        bytes.frombuffer(b'abcd', offset=2) -> b'cd'
        bytes.frombuffer(b'abcd', 1, 2) -> b'c'

----------
components: Interpreter Core
files: bytes-frombuffer.patch
keywords: patch
messages: 284813
nosy: Yury.Selivanov, belopolsky, haypo, inada.naoki, ncoghlan
priority: normal
severity: normal
status: open
title: Adding bytes.frombuffer(byteslike) constructor
type: enhancement
versions: Python 3.7
Added file: http://bugs.python.org/file46173/bytes-frombuffer.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29178>
_______________________________________

From report at bugs.python.org  Fri Jan  6 06:39:26 2017
From: report at bugs.python.org (Evan)
Date: Fri, 06 Jan 2017 11:39:26 +0000
Subject: [issue29132] shlex.shlex with punctuation_chars and posix doesn't
 handle punctuation next to quotes
In-Reply-To: <1483361783.89.0.00336686739256.issue29132@psf.upfronthosting.co.za>
Message-ID: <1483702766.81.0.173420634256.issue29132@psf.upfronthosting.co.za>


Evan added the comment:

I've just submitted the form.

I'm attaching a second patch which also addresses another similar bug demonstrated in the test case. The fix is to check for the 'c' state before checking for quotes or escape characters.

----------
Added file: http://bugs.python.org/file46174/shlex-posix-quote2.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29132>
_______________________________________

From report at bugs.python.org  Fri Jan  6 07:05:18 2017
From: report at bugs.python.org (Xavier de Gaye)
Date: Fri, 06 Jan 2017 12:05:18 +0000
Subject: [issue29177] skip tests using socketserver.UnixStreamServer when
 bind() raises PermissionError
In-Reply-To: <1483701262.25.0.0295401910206.issue29177@psf.upfronthosting.co.za>
Message-ID: <1483704318.16.0.857791729842.issue29177@psf.upfronthosting.co.za>


Xavier de Gaye added the comment:

test_logging fails also with the following backtrace:

======================================================================
FAIL: test_output (test.test_logging.UnixSocketHandlerTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/test/test_logging.py", line 1527, in setUp
    SocketHandlerTest.setUp(self)
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/test/test_logging.py", line 1442, in setUp
    BaseTest.setUp(self)
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/test/test_logging.py", line 111, in setUp
    raise AssertionError('Unexpected handlers: %s' % hlist)
AssertionError: Unexpected handlers: [<StreamHandler (NOTSET)>]

This is because when a test fails or when it is skipped, the BaseTest.tearDown() method is not called and as a consequence the following test fails.

----------
nosy: +vinay.sajip

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29177>
_______________________________________

From report at bugs.python.org  Fri Jan  6 07:37:37 2017
From: report at bugs.python.org (Cornelius Diekmann)
Date: Fri, 06 Jan 2017 12:37:37 +0000
Subject: [issue29054] pty.py: pty.spawn hangs after client disconnect over nc
 (netcat)
In-Reply-To: <1482502637.76.0.304881442895.issue29054@psf.upfronthosting.co.za>
Message-ID: <1483706257.55.0.480372883993.issue29054@psf.upfronthosting.co.za>


Cornelius Diekmann added the comment:

[no status change, this issue currently does NOT need any attention]

To keep issues separate, I just wanted to document a comment about this issue mentioned in issue29070. It refers to the _copy loop.

        if STDIN_FILENO in rfds:
             data = stdin_read(STDIN_FILENO)
             if not data:
                 fds.remove(STDIN_FILENO)
+                # Proposal for future behavior change: Signal EOF to
+                # slave if STDIN of master is gone. Solves issue29054.
+                # os.write(master_fd, b'\x04')
             else:
                 _writen(master_fd, data)

> vadmium 2017/01/04 21:50:26
> I suggest leaving this for the other [issue29054, i.e. this] bug. Another option may be to send SIGHUP
> (though I am far from an expert on Unix terminals :).

http://bugs.python.org/review/29070/diff/19626/Lib/pty.py

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29054>
_______________________________________

From report at bugs.python.org  Fri Jan  6 07:38:36 2017
From: report at bugs.python.org (Segev Finer)
Date: Fri, 06 Jan 2017 12:38:36 +0000
Subject: [issue19764] subprocess: use PROC_THREAD_ATTRIBUTE_HANDLE_LIST with
 STARTUPINFOEX on Windows Vista
In-Reply-To: <1385370115.99.0.757096187315.issue19764@psf.upfronthosting.co.za>
Message-ID: <1483706316.5.0.553505188343.issue19764@psf.upfronthosting.co.za>


Segev Finer added the comment:

Though Python has taken measures to mark handles as non-inheritable there is still a possible race due to having to create inheritable handles while creating processes with stdio pipes (subprocess).

Attached is a Patch that implements subprocess.Popen(close_fds=True) with stdio handles on Windows using PROC_THREAD_ATTRIBUTE_HANDLE_LIST, which plugs that race completely.

I implemented this by adding the attribute STARTUPINFO._handleList, which when passed to _winapi.CreateProcess, will be passed to CreateProcess as a PROC_THREAD_ATTRIBUTE_HANDLE_LIST. subprocess.py can than use this attribute as needed with inherit_handles=True to only inherit the stdio handles.

The STARTUPINFO._handleList attribute can also be used to implement pass_fds later on. Though the exact behavior of how to convert a file descriptor list to a handle list might be a bit sensitive, so I left that out for now.

This patch obviously doesn't support Windows XP but Python 3 doesn't support XP anymore either.

----------
keywords: +patch
nosy: +Segev Finer
Added file: http://bugs.python.org/file46175/windows-subprocess-close-fds.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue19764>
_______________________________________

From report at bugs.python.org  Fri Jan  6 08:28:39 2017
From: report at bugs.python.org (Herman Schistad)
Date: Fri, 06 Jan 2017 13:28:39 +0000
Subject: [issue22231] httplib: unicode url will cause an ascii codec error
 when combined with a utf-8 string header
In-Reply-To: <1408502977.94.0.555848371144.issue22231@psf.upfronthosting.co.za>
Message-ID: <1483709319.53.0.0850201266256.issue22231@psf.upfronthosting.co.za>


Herman Schistad added the comment:

I can confirm that this patch solves the issues I've had where I can submit multipart forms provided I have a string URL, but not if it's unicode.

I'm using Python 2.7.12. Applying the patch fixes the issue.

Code which breaks, assuming the file contains binary data:


# -*- encoding: utf-8 -*-
import urllib3
pool_manager = urllib3.PoolManager(num_pools=2)
url = u'http://example.org/form' # removing the 'u' fixes it
content = open('/some/binary/file').read()
fields = [
    ('foo', 'something'),
    ('bar', ('/some/binary/file', content, 'application/octet-stream'))
]
pool_manager.request("POST", url, fields=fields, encode_multipart=True, headers={})

----------
nosy: +Herman Schistad

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue22231>
_______________________________________

From report at bugs.python.org  Fri Jan  6 08:30:44 2017
From: report at bugs.python.org (Chi Hsuan Yen)
Date: Fri, 06 Jan 2017 13:30:44 +0000
Subject: [issue29176] /tmp does not exist on Android and is used by
 curses.window.putwin()
In-Reply-To: <1483699620.77.0.827616599041.issue29176@psf.upfronthosting.co.za>
Message-ID: <1483709444.54.0.463675111533.issue29176@psf.upfronthosting.co.za>


Chi Hsuan Yen added the comment:

I guess replace mkstemp (C function) with tempfile.mkstemp (Python function) can solve the problem.

----------
nosy: +Chi Hsuan Yen

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29176>
_______________________________________

From report at bugs.python.org  Fri Jan  6 09:28:41 2017
From: report at bugs.python.org (Chun-Yu Tseng)
Date: Fri, 06 Jan 2017 14:28:41 +0000
Subject: [issue22135] allow to break into pdb with Ctrl-C for all the commands
 that resume execution
In-Reply-To: <1407185061.52.0.89159247831.issue22135@psf.upfronthosting.co.za>
Message-ID: <1483712921.45.0.665306292743.issue22135@psf.upfronthosting.co.za>


Chun-Yu Tseng added the comment:

Ping :)

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue22135>
_______________________________________

From report at bugs.python.org  Fri Jan  6 09:28:56 2017
From: report at bugs.python.org (Petr Viktorin)
Date: Fri, 06 Jan 2017 14:28:56 +0000
Subject: [issue29179] Py_UNUSED is not documented
Message-ID: <1483712936.43.0.521596791461.issue29179@psf.upfronthosting.co.za>


New submission from Petr Viktorin:

The Py_UNUSED macro, which was added in Python 3.4, is not documented.
Is this an omission, or is it undocumented on purpose?
I can prepare a patch if it's the former.


The macro was added in: http://bugs.python.org/issue19976
and referenced in: http://bugs.python.org/issue26179
Usage on Github: https://github.com/search?q=py_unused&type=Code

----------
assignee: docs at python
components: Documentation
messages: 284821
nosy: docs at python, encukou
priority: normal
severity: normal
status: open
title: Py_UNUSED is not documented
versions: Python 3.4, Python 3.5, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29179>
_______________________________________

From report at bugs.python.org  Fri Jan  6 09:31:26 2017
From: report at bugs.python.org (Christian Heimes)
Date: Fri, 06 Jan 2017 14:31:26 +0000
Subject: [issue27876] Add SSLContext.set_version_range(minver, maxver=None)
In-Reply-To: <1472293835.86.0.296775745483.issue27876@psf.upfronthosting.co.za>
Message-ID: <1483713086.24.0.783691076522.issue27876@psf.upfronthosting.co.za>


Christian Heimes added the comment:

PoC implementation:

from enum import Enum
import ssl

OP_NO_TLSv1_3 = getattr(ssl, 'OP_NO_TLSv1_3', 0)

OP_NO_FLAGS = [
    ssl.OP_NO_SSLv2,
    ssl.OP_NO_SSLv3,
    ssl.OP_NO_TLSv1,
    ssl.OP_NO_TLSv1_1,
    ssl.OP_NO_TLSv1_2,
    OP_NO_TLSv1_3
]

OP_NO_MASK = sum(OP_NO_FLAGS)


class TLSVersions(Enum):
    SSLv2 = 'SSL 2.0', 0x0200, 0
    SSLv3 = 'SSL 3.0', 0x0300, 1
    TLSv1 = 'TLS 1.0', 0x0301, 2
    TLSv1_1 = 'TLS 1.1', 0x0302, 3
    TLSv1_2 = 'TLS 1.2', 0x0303, 4

    if OP_NO_TLSv1_3:
        TLSv1_3 = 'TLS 1.3', 0x0304, 5
        MAX = TLSv1_3
    else:
        MAX = TLSv1_2

    MIN = TLSv1

    def __init__(self, prettyname, wireprotocol, offset):
        self.prettyname = prettyname
        self.wireprotocol = wireprotocol
        self.noflag = OP_NO_FLAGS[offset]
        self.minflag = sum(OP_NO_FLAGS[:offset])
        self.maxflag = sum(OP_NO_FLAGS[offset+1:])

    def __repr__(self):
        return ("<{0.__class__.__name__}.{0.name} "
                "({0.prettyname}, 0x{0.wireprotocol:x})>").format(self)

    __str__ = __repr__


class SSLContext(ssl.SSLContext):
    def set_version(self, minver=TLSVersions.MIN, maxver=TLSVersions.MAX):
        options = self.options & ~OP_NO_MASK
        self.options = options | minver.minflag | maxver.maxflag


if __name__ == '__main__':
    for name, member in TLSVersions.__members__.items():
        print(name, member)

    ctx = SSLContext(ssl.PROTOCOL_SSLv23)
    print(ctx.options)
    ctx.set_version(minver=TLSVersions.SSLv3, maxver=TLSVersions.TLSv1_1)
    print(ctx.options)

----------
versions: +Python 3.7 -Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue27876>
_______________________________________

From report at bugs.python.org  Fri Jan  6 09:59:50 2017
From: report at bugs.python.org (Michael Foord)
Date: Fri, 06 Jan 2017 14:59:50 +0000
Subject: [issue28961] unittest.mock._Call ignores `name` parameter
In-Reply-To: <1481636868.17.0.0411235027553.issue28961@psf.upfronthosting.co.za>
Message-ID: <1483714790.87.0.814786844273.issue28961@psf.upfronthosting.co.za>


Michael Foord added the comment:

Yep, LGTM as well. Nicely spotted!

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28961>
_______________________________________

From report at bugs.python.org  Fri Jan  6 10:14:52 2017
From: report at bugs.python.org (Xavier de Gaye)
Date: Fri, 06 Jan 2017 15:14:52 +0000
Subject: [issue29180] skip tests that raise PermissionError in test_os
 (non-root user on Android)
Message-ID: <1483715692.75.0.979129057464.issue29180@psf.upfronthosting.co.za>


New submission from Xavier de Gaye:

======================================================================                   [1633/2616]
ERROR: test_link (test.test_os.LinkTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/test/test_os.py", line 1703, in test_link
    self._test_link(self.file1, self.file2)
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/test/test_os.py", line 1698, in _test_link
    os.link(file1, file2)
PermissionError: [Errno 13] Permission denied: '@test_2295_tmp' -> '@test_2295_tmp2'

======================================================================
ERROR: test_link_bytes (test.test_os.LinkTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/test/test_os.py", line 1707, in test_link_bytes
    bytes(self.file2, sys.getfilesystemencoding()))
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/test/test_os.py", line 1698, in _test_link
    os.link(file1, file2)
PermissionError: [Errno 13] Permission denied: b'@test_2295_tmp' -> b'@test_2295_tmp2'

======================================================================
ERROR: test_unicode_name (test.test_os.LinkTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/test/test_os.py", line 1717, in test_unicode_name
    self._test_link(self.file1, self.file2)
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/test/test_os.py", line 1698, in _test_link
    os.link(file1, file2)
PermissionError: [Errno 13] Permission denied: '@test_2295_tmp?' -> '@test_2295_tmp?2'

======================================================================
ERROR: test_stty_match (test.test_os.TermsizeTests)
Check if stty returns the same results
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/test/test_os.py", line 2795, in test_stty_match
    size = subprocess.check_output(['stty', 'size']).decode().split()
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/subprocess.py", line 336, in check_output
    **kwargs).stdout
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/subprocess.py", line 403, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/subprocess.py", line 707, in __init__
    restore_signals, start_new_session)
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/subprocess.py", line 1323, in _execute_child
    raise child_exception_type(errno_num, err_msg)
PermissionError: [Errno 13] Permission denied

======================================================================
ERROR: test_attributes (test.test_os.TestScandir)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/test/test_os.py", line 3150, in test_attributes
    os.link(filename, os.path.join(self.path, "link_file.txt"))
PermissionError: [Errno 13] Permission denied: '/data/local/tmp/test_python_2295/@test_2295_tmp/file
.txt' -> '/data/local/tmp/test_python_2295/@test_2295_tmp/link_file.txt'

----------------------------------------------------------------------
Ran 238 tests in 4.675s

FAILED (errors=5, skipped=40)
test test_os failed

----------
assignee: xdegaye
components: Tests
messages: 284824
nosy: xdegaye
priority: normal
severity: normal
stage: needs patch
status: open
title: skip tests that raise PermissionError in test_os (non-root user on Android)
type: behavior
versions: Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29180>
_______________________________________

From report at bugs.python.org  Fri Jan  6 10:33:15 2017
From: report at bugs.python.org (R. David Murray)
Date: Fri, 06 Jan 2017 15:33:15 +0000
Subject: [issue29175] Tutorial links to file object methods are broken.
In-Reply-To: <1483696723.95.0.735667565924.issue29175@psf.upfronthosting.co.za>
Message-ID: <1483716795.41.0.96556643183.issue29175@psf.upfronthosting.co.za>


R. David Murray added the comment:

The tutorial links are definitely bugs, and the section may need some tweaking.  The 'file object' documentation is replaced by the io module, which completely defines the file object API.  There is a link from the 'file object' glossary entry to the IO module indicating this.

Only 3.6 and 3.7 docs get documentation updates at this point, so I've removed the others from versions (we use versions to say where things will get fixed).  (Well, 3.5 docs can still get updates at the moment, but that won't last long, final is soon.)

----------
nosy: +r.david.murray
title: Documentation for File objects missing -> Tutorial links to file object methods are broken.
versions:  -Python 3.3, Python 3.4, Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29175>
_______________________________________

From report at bugs.python.org  Fri Jan  6 10:42:02 2017
From: report at bugs.python.org (Xavier de Gaye)
Date: Fri, 06 Jan 2017 15:42:02 +0000
Subject: [issue29177] skip tests using socketserver.UnixStreamServer when
 bind() raises PermissionError
In-Reply-To: <1483701262.25.0.0295401910206.issue29177@psf.upfronthosting.co.za>
Message-ID: <1483717322.96.0.778615356336.issue29177@psf.upfronthosting.co.za>


Xavier de Gaye added the comment:

To reproduce the test_logging cleanup problem, insert skipTest() in setUp():

diff -r 4a97fa319bf7 Lib/test/test_logging.py
--- a/Lib/test/test_logging.py  Fri Jan 06 09:52:19 2017 +0100
+++ b/Lib/test/test_logging.py  Fri Jan 06 16:39:38 2017 +0100
@@ -1440,6 +1440,7 @@
         """Set up a TCP server to receive log messages, and a SocketHandler
         pointing to that server's address and port."""
         BaseTest.setUp(self)
+        self.skipTest('Skip test in setUp().')
         self.server = server = self.server_class(self.address,
                                                  self.handle_socket, 0.01)
         server.start()

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29177>
_______________________________________

From report at bugs.python.org  Fri Jan  6 10:48:52 2017
From: report at bugs.python.org (Xavier de Gaye)
Date: Fri, 06 Jan 2017 15:48:52 +0000
Subject: [issue29177] skip tests using socketserver.UnixStreamServer when
 bind() raises PermissionError
In-Reply-To: <1483701262.25.0.0295401910206.issue29177@psf.upfronthosting.co.za>
Message-ID: <1483717732.64.0.0633942643433.issue29177@psf.upfronthosting.co.za>


Xavier de Gaye added the comment:

The test_logging cleanup problem induces another problem. When test_lib2to3 is run after the failing test_logging then test_lib2to3 fails with:

======================================================================
FAIL: test_filename_changing_on_output_single_dir (lib2to3.tests.test_main.TestMain)
2to3 a single directory with a new output dir and suffix.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/lib2to3/tests/test_main.py", line 91, in test_file
name_changing_on_output_single_dir
    self.py3_dest_dir, self.py2_src_dir), stderr)
AssertionError: "Output in '/data/local/tmp/tmp4eyn96tg/python3_project' will mirror the input direc
tory '/data/local/tmp/tmp4eyn96tg/python2_project' layout" not found in 'WARNING: --write-unchanged-
files/-W implies -w.\n'

----------------------------------------------------------------------
Ran 616 tests in 140.986s

FAILED (failures=1, expected failures=1)
Warning -- logging._handlerList was modified by test_lib2to3
test test_lib2to3 failed


It seems that this is related to the test_logging failure because the expected string "Output in '/data/local/tmp/tmp4eyn96tg/python3_project' will mirror the input directory '/data/local/tmp/tmp4eyn96tg/python2_project' layout" is output by logger.info() in Lib/lib2to3/main.py:243.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29177>
_______________________________________

From report at bugs.python.org  Fri Jan  6 11:07:02 2017
From: report at bugs.python.org (Vinay Sajip)
Date: Fri, 06 Jan 2017 16:07:02 +0000
Subject: [issue29177] skip tests using socketserver.UnixStreamServer when
 bind() raises PermissionError
In-Reply-To: <1483701262.25.0.0295401910206.issue29177@psf.upfronthosting.co.za>
Message-ID: <1483718822.2.0.644139877356.issue29177@psf.upfronthosting.co.za>


Vinay Sajip added the comment:

> To reproduce the test_logging cleanup problem, insert skipTest() in setUp()

My understanding is that skipTest would normally be raised in the test method itself or as a decorator to it, and not in setUp() itself. (It wouldn't make sense to, as that would skip every test in the test case - not the obvious thing to do.) I can understand that failures that happen in setUp() may cause tearDown() not to be called.

I would guess that setUp() should recover from any problem and set a flag which is then used to skip in the other tests (which rely on that problem not being there).

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29177>
_______________________________________

From report at bugs.python.org  Fri Jan  6 11:08:29 2017
From: report at bugs.python.org (Xavier de Gaye)
Date: Fri, 06 Jan 2017 16:08:29 +0000
Subject: [issue22135] allow to break into pdb with Ctrl-C for all the commands
 that resume execution
In-Reply-To: <1407185061.52.0.89159247831.issue22135@psf.upfronthosting.co.za>
Message-ID: <1483718908.99.0.387017105504.issue22135@psf.upfronthosting.co.za>


Xavier de Gaye added the comment:

The code review of your first patch still applies to your last patch.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue22135>
_______________________________________

From report at bugs.python.org  Fri Jan  6 11:08:47 2017
From: report at bugs.python.org (Vinay Sajip)
Date: Fri, 06 Jan 2017 16:08:47 +0000
Subject: [issue29168] multiprocessing pickle error
In-Reply-To: <1483618368.97.0.107401413332.issue29168@psf.upfronthosting.co.za>
Message-ID: <1483718927.37.0.462181334262.issue29168@psf.upfronthosting.co.za>


Vinay Sajip added the comment:

I'm closing this on the basis that I don't believe there's a bug here, but you can of course re-open if you have evidence to the contrary.

----------
resolution:  -> not a bug
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29168>
_______________________________________

From report at bugs.python.org  Fri Jan  6 11:19:52 2017
From: report at bugs.python.org (Xavier de Gaye)
Date: Fri, 06 Jan 2017 16:19:52 +0000
Subject: [issue29177] skip tests using socketserver.UnixStreamServer when
 bind() raises PermissionError
In-Reply-To: <1483701262.25.0.0295401910206.issue29177@psf.upfronthosting.co.za>
Message-ID: <1483719592.93.0.80842815126.issue29177@psf.upfronthosting.co.za>


Xavier de Gaye added the comment:

> My understanding is that skipTest would normally be raised in the test method itself or as a decorator to it, and not in setUp() itself.

Agreed. Would that make sense to move the server.start() part out of setUp() and in its own method that would be called by each test (I am not very familiar with test_logging) ?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29177>
_______________________________________

From report at bugs.python.org  Fri Jan  6 11:31:35 2017
From: report at bugs.python.org (Xavier de Gaye)
Date: Fri, 06 Jan 2017 16:31:35 +0000
Subject: [issue29181] skip tests that raise PermissionError in test_tarfile
 (non-root user on Android)
Message-ID: <1483720295.96.0.810571843517.issue29181@psf.upfronthosting.co.za>


New submission from Xavier de Gaye:

======================================================================                    [339/2616]
ERROR: test_link_size (test.test_tarfile.Bz2WriteTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/test/test_tarfile.py", line 1102, in test_link_siz
e
    os.link(target, link)
PermissionError: [Errno 13] Permission denied: '/data/local/tmp/test_python_2295/@test_2295_tmp-tard
ir/link_target' -> '/data/local/tmp/test_python_2295/@test_2295_tmp-tardir/link'

======================================================================
ERROR: test_link_size (test.test_tarfile.GzipWriteTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/test/test_tarfile.py", line 1102, in test_link_siz
e
    os.link(target, link)
PermissionError: [Errno 13] Permission denied: '/data/local/tmp/test_python_2295/@test_2295_tmp-tard
ir/link_target' -> '/data/local/tmp/test_python_2295/@test_2295_tmp-tardir/link'

======================================================================
ERROR: test_add_hardlink (test.test_tarfile.HardlinkTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/test/test_tarfile.py", line 1563, in setUp
    os.link(self.foo, self.bar)
PermissionError: [Errno 13] Permission denied: '/data/local/tmp/test_python_2295/@test_2295_tmp-tard
ir/foo' -> '/data/local/tmp/test_python_2295/@test_2295_tmp-tardir/bar'

======================================================================
ERROR: test_add_twice (test.test_tarfile.HardlinkTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/test/test_tarfile.py", line 1563, in setUp
    os.link(self.foo, self.bar)
PermissionError: [Errno 13] Permission denied: '/data/local/tmp/test_python_2295/@test_2295_tmp-tard
ir/foo' -> '/data/local/tmp/test_python_2295/@test_2295_tmp-tardir/bar'

======================================================================
ERROR: test_dereference_hardlink (test.test_tarfile.HardlinkTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/test/test_tarfile.py", line 1563, in setUp
    os.link(self.foo, self.bar)
PermissionError: [Errno 13] Permission denied: '/data/local/tmp/test_python_2295/@test_2295_tmp-tard
ir/foo' -> '/data/local/tmp/test_python_2295/@test_2295_tmp-tardir/bar'

======================================================================
ERROR: test_link_size (test.test_tarfile.WriteTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/test/test_tarfile.py", line 1102, in test_link_siz
e
    os.link(target, link)
PermissionError: [Errno 13] Permission denied: '/data/local/tmp/test_python_2295/@test_2295_tmp-tard
ir/link_target' -> '/data/local/tmp/test_python_2295/@test_2295_tmp-tardir/link'

----------------------------------------------------------------------
Ran 426 tests in 17.296s

FAILED (errors=6, skipped=80)
test test_tarfile failed

----------
assignee: xdegaye
components: Tests
messages: 284832
nosy: xdegaye
priority: normal
severity: normal
stage: needs patch
status: open
title: skip tests that raise PermissionError in test_tarfile (non-root user on Android)
type: behavior
versions: Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29181>
_______________________________________

From report at bugs.python.org  Fri Jan  6 11:33:35 2017
From: report at bugs.python.org (Yury Selivanov)
Date: Fri, 06 Jan 2017 16:33:35 +0000
Subject: [issue29178] Adding bytes.frombuffer(byteslike) constructor
In-Reply-To: <1483701377.3.0.893514886442.issue29178@psf.upfronthosting.co.za>
Message-ID: <1483720415.98.0.616131028094.issue29178@psf.upfronthosting.co.za>


Yury Selivanov added the comment:

I've added a couple of review comments.  Also, it looks like you can count Antoine Pitrou as +1 too.

Two questions:

1. length or count?  Need to look through builtins/stdlib and see what is more common in CPython.

2. Maybe we should make length/count and offset keyword-only arguments?

----------
nosy: +yselivanov

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29178>
_______________________________________

From report at bugs.python.org  Fri Jan  6 11:45:17 2017
From: report at bugs.python.org (Eryk Sun)
Date: Fri, 06 Jan 2017 16:45:17 +0000
Subject: [issue19764] subprocess: use PROC_THREAD_ATTRIBUTE_HANDLE_LIST with
 STARTUPINFOEX on Windows Vista
In-Reply-To: <1385370115.99.0.757096187315.issue19764@psf.upfronthosting.co.za>
Message-ID: <1483721117.64.0.257724925649.issue19764@psf.upfronthosting.co.za>


Eryk Sun added the comment:

Implementing pass_fds on Windows is a problem if Popen has to implement the undocumented use of the STARTUPINFO cbReserved2 and lpReserved2 fields to inherit CRT file descriptors. I suppose we could implement this ourselves in _winapi since it's unlikely that the data format will ever change. Just copy what the CRT's accumulate_inheritable_handles() function does, but constrained by an array of file descriptors.

----------
components: +Library (Lib)
nosy: +eryksun
stage:  -> patch review
versions: +Python 3.7 -Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue19764>
_______________________________________

From report at bugs.python.org  Fri Jan  6 11:54:53 2017
From: report at bugs.python.org (luke_16)
Date: Fri, 06 Jan 2017 16:54:53 +0000
Subject: [issue29158] Possible glitch in the interaction of a thread and a
 multiprocessing manager
In-Reply-To: <1483557543.41.0.26156873729.issue29158@psf.upfronthosting.co.za>
Message-ID: <1483721693.84.0.470997305665.issue29158@psf.upfronthosting.co.za>


luke_16 added the comment:

Regarding Davin's last paragraph: "Without pulling apart your code...", I would like to point out that what I'm doing is what the Documentation instructs:

https://docs.python.org/2/library/multiprocessing.html#using-a-remote-manager

So, I want to access a process running in another machine, which is the server.py, from another machine running client.py. But if I want tho run both separately in the same machine, I should also be able to.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29158>
_______________________________________

From report at bugs.python.org  Fri Jan  6 12:01:57 2017
From: report at bugs.python.org (Senthil Kumaran)
Date: Fri, 06 Jan 2017 17:01:57 +0000
Subject: [issue29182] Remove the warning in urllib docs that it doesn't do
 certificate validate by default.
Message-ID: <1483722117.81.0.319313681571.issue29182@psf.upfronthosting.co.za>


New submission from Senthil Kumaran:

It started as a discussion in this issue: http://bugs.python.org/issue22417#msg284604

I think, that warning can be removed.  If no one has any objections, I will commit this attached patch.

----------
assignee: orsenthil
components: Documentation
files: remove_warning.patch
keywords: patch
messages: 284836
nosy: benjamin.peterson, christian.heimes, clopez, orsenthil
priority: normal
severity: normal
stage: patch review
status: open
title: Remove the warning in urllib docs that it doesn't do certificate validate by default.
type: behavior
versions: Python 2.7
Added file: http://bugs.python.org/file46176/remove_warning.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29182>
_______________________________________

From report at bugs.python.org  Fri Jan  6 12:08:06 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Fri, 06 Jan 2017 17:08:06 +0000
Subject: [issue29178] Adding bytes.frombuffer(byteslike) constructor
In-Reply-To: <1483701377.3.0.893514886442.issue29178@psf.upfronthosting.co.za>
Message-ID: <1483722486.96.0.947335023496.issue29178@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

Count me as -1 too.

This is just a two-liner:

    with memoryview(bytelike) as m:
        bs = bytes(m[start:end])

In most cases, when all content is used, the bytes constructor works fine.

    bs = bytes(bytelike)

This works not just with bytes, but with bytearray and most other bytes-like arrays. With frombuffer() you need to add a new class method to all these classes.

Adding new method to builtin type has high bar. I doubts that there are enough use cases in which bytes.frombuffer() has an advantage.

The signature of bytes.frombuffer() looks questionable. Why length and offset instead of start and stop indices as in slices? Why length is first and offset is last? This contradicts the interface of Python 2 buffer(), socket.sendfile(), os.sendfile(), etc.

There is also a problem with returned type for subclasses (this is always a problem for alternate constructors). Should B.frombuffer() where B is a bytes subclass return an instance of bytes or B? If it always returns a bytes object, we need to use a constructor for subclasses, if it returns an instance of a subclass, how can it be implemented efficiently?

----------
nosy: +serhiy.storchaka

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29178>
_______________________________________

From report at bugs.python.org  Fri Jan  6 12:13:33 2017
From: report at bugs.python.org (Yury Selivanov)
Date: Fri, 06 Jan 2017 17:13:33 +0000
Subject: [issue29178] Adding bytes.frombuffer(byteslike) constructor
In-Reply-To: <1483701377.3.0.893514886442.issue29178@psf.upfronthosting.co.za>
Message-ID: <1483722813.74.0.841750617729.issue29178@psf.upfronthosting.co.za>


Yury Selivanov added the comment:

> This is just a two-liner:
>
>    with memoryview(bytelike) as m:
>        bs = bytes(m[start:end])

Which virtually no one follows :(


> Adding new method to builtin type has high bar. I doubts that there are enough use cases in which bytes.frombuffer() has an advantage.

Any protocol parsing code has a lot of slicing.

> The signature of bytes.frombuffer() looks questionable. Why length and offset instead of start and stop indices as in slices? Why length is first and offset is last? This contradicts the interface of Python 2 buffer(), socket.sendfile(), os.sendfile(), etc.

I propose to make both arguments keyword-only.

> There is also a problem with returned type for subclasses (this is always a problem for alternate constructors).

Good point. How do we usually solve this in CPython?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29178>
_______________________________________

From report at bugs.python.org  Fri Jan  6 12:16:32 2017
From: report at bugs.python.org (Roundup Robot)
Date: Fri, 06 Jan 2017 17:16:32 +0000
Subject: [issue28961] unittest.mock._Call ignores `name` parameter
In-Reply-To: <1481636868.17.0.0411235027553.issue28961@psf.upfronthosting.co.za>
Message-ID: <20170106171629.55379.38131.3FECC769@psf.io>


Roundup Robot added the comment:

New changeset 50424a903593 by Victor Stinner in branch '3.6':
Fix unittest.mock._Call: don't ignore name
https://hg.python.org/cpython/rev/50424a903593

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28961>
_______________________________________

From report at bugs.python.org  Fri Jan  6 12:18:32 2017
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 06 Jan 2017 17:18:32 +0000
Subject: [issue28961] unittest.mock._Call ignores `name` parameter
In-Reply-To: <1481636868.17.0.0411235027553.issue28961@psf.upfronthosting.co.za>
Message-ID: <1483723112.93.0.470381253223.issue28961@psf.upfronthosting.co.za>


STINNER Victor added the comment:

I applied the latest mock.patch to Python 3.6 and default (future 3.7). I prefer to wait for the 3.5.3 release before backporting the fix to 3.5, the fix is minor, I don't want to annoy the release manager yet.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28961>
_______________________________________

From report at bugs.python.org  Fri Jan  6 12:19:15 2017
From: report at bugs.python.org (Ivan Levkivskyi)
Date: Fri, 06 Jan 2017 17:19:15 +0000
Subject: [issue29116] Make str and bytes error messages on concatenation
 conform with other sequences
In-Reply-To: <1483124211.75.0.715869443236.issue29116@psf.upfronthosting.co.za>
Message-ID: <1483723155.85.0.178653587808.issue29116@psf.upfronthosting.co.za>


Changes by Ivan Levkivskyi <levkivskyi at gmail.com>:


----------
nosy: +levkivskyi

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29116>
_______________________________________

From report at bugs.python.org  Fri Jan  6 12:54:55 2017
From: report at bugs.python.org (Vinay Sajip)
Date: Fri, 06 Jan 2017 17:54:55 +0000
Subject: [issue29177] skip tests using socketserver.UnixStreamServer when
 bind() raises PermissionError
In-Reply-To: <1483701262.25.0.0295401910206.issue29177@psf.upfronthosting.co.za>
Message-ID: <1483725295.37.0.517666264978.issue29177@psf.upfronthosting.co.za>


Vinay Sajip added the comment:

> Would that make sense to move the server.start() part out of setUp() and in its own method

My preference would be to just catch the error in SocketHandlerTest.setUp() and leave things in a tidy state (e.g. .server and .sock_hdlr set to None), make the tearDown() logic take that into account, and in each test just skip if .server is None.

Do you want to take this on, or do you want me to look at it?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29177>
_______________________________________

From report at bugs.python.org  Fri Jan  6 13:17:35 2017
From: report at bugs.python.org (Cornelius Diekmann)
Date: Fri, 06 Jan 2017 18:17:35 +0000
Subject: [issue29070] Integration tests for pty.spawn on Linux and all other
 platforms
In-Reply-To: <1482684225.17.0.467768789806.issue29070@psf.upfronthosting.co.za>
Message-ID: <1483726655.59.0.148206946373.issue29070@psf.upfronthosting.co.za>


Cornelius Diekmann added the comment:

Thank you Martin very much for this very helpful review. I updated and simplified the tests and implemented your suggestions.

There are three open issues left.

1)
> It looks like you depend on fixing Issue 26228, but the patch there will conflict with your changes. Maybe merge with the other patch, or propose an alternative fix.

I reverted my changes to pty.py, so the patches no longer conflict for this file. Currently, my test suite reveals the pty bug on FreeBSD but does not fix it. Since I'm not a FreeBSD expert and I don't want to steal Chris' patch [issue26228], I'd like to keep those issues separate. In other words, I'm proposing only a testsuite which introduces no behavioral changes and uncovers a bug.

This also means that this test suite is currently hanging on FreeBSD and OS X. The good news is, the patch in issue26228 fixes the problem.

> The documentation currently mentions the code is only tested on Linux, so it would be nice to update that.
Is it okay to keep this hanging around until the pty module is no longer broken on FreeBSD?

2)
> Why does the patch slow the tests down so much? Ideally, it is nice to keep the tests as fast as possible.

The test suite does some heavy integration testing. The test in the class PtyWhiteBoxIntegrationReadSlaveTest are the slow ones. The test suite reads a large amount of data from the spawned child process. The amount of data was chosen to be a few kB in size to make sure that we won't lose data of the child and that --in case of a bug in the code-- we don't accidentally succeed due to race conditions.

Is 1-2 seconds really too slow? We can reduce the amount of data, but increase the risk that a future code change introduces a race condition which is not covered by the test suite.


3)
About the system bell pretty printing test:
> This seems platform-dependent. Do you really need to test echoing of control
> characters? We only need to test Python, not the operating system.

Including these integration tests has advantages and disadvantages.

Advantages:
 * Integration tests which deliberately depend on the careful interplay of many modules make sure that the overall system is healthy.
 * Modules such as termios or terminal-specific parts of os seem not to be tested currently. This test suite depends on their correct functionality and is thus a high-level test for them. In addition, pty.spawn() is the perfect place to test them.
 * The tests test actual terminal features. Without, we could not uncover if e.g., a pty master/slave is replaced by a simple pipe.
 * Tests are set up by increasing order of complexity w.r.t. the underlying modules they depend on. If something breaks in the future, these tests are designed to ease debugging and directly point to the error.
 * They provide some documentation about the magic of terminals and examples.
 * Those tests are fast.

Disadvantages:
 * Control-Character pretty printing only works on Linux.
 * Relies on the (POSIX-compliant) internals of the operating system.

I wanted to include some tests which depend on some terminal-specific features to make sure that the complete setup works and is operational. For example, the test suite now documents the difference between ptys and pipes and would complain if any code change breaks terminal-specific features. I chose control character pretty printing in some tests because this is one of the simplest features of terminals to test, debug, and verify. In contrast, for example, it would be extremely fragile and probably complete overkill to test things such as terminal delays. 

My personal feeling says that the advantages outweigh the disadvantages. If you disagree, we can remove them :-)

----------
Added file: http://bugs.python.org/file46177/pty_tests.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29070>
_______________________________________

From report at bugs.python.org  Fri Jan  6 13:19:47 2017
From: report at bugs.python.org (Segev Finer)
Date: Fri, 06 Jan 2017 18:19:47 +0000
Subject: [issue19764] subprocess: use PROC_THREAD_ATTRIBUTE_HANDLE_LIST with
 STARTUPINFOEX on Windows Vista
In-Reply-To: <1385370115.99.0.757096187315.issue19764@psf.upfronthosting.co.za>
Message-ID: <1483726787.13.0.214730908571.issue19764@psf.upfronthosting.co.za>


Segev Finer added the comment:

Second version of the patch after review by eryksun.

Please pay attention to the hack in _execute_child due to having to 
temporarily override the handle_list if the user supplied
one.

As for pass_fds: as you noted, it has it's own share of complexities and issues and I think it's best to leave it to a separate patch/issue.

----------
Added file: http://bugs.python.org/file46178/windows-subprocess-close-fds-v2.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue19764>
_______________________________________

From report at bugs.python.org  Fri Jan  6 13:57:19 2017
From: report at bugs.python.org (Brett Cannon)
Date: Fri, 06 Jan 2017 18:57:19 +0000
Subject: [issue2771] Test issue
In-Reply-To: <1210005645.74.0.283923986194.issue2771@psf.upfronthosting.co.za>
Message-ID: <1483729039.86.0.130153351027.issue2771@psf.upfronthosting.co.za>


Changes by Brett Cannon <brett at python.org>:


----------
pull_requests: +9

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue2771>
_______________________________________

From report at bugs.python.org  Fri Jan  6 14:40:14 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Fri, 06 Jan 2017 19:40:14 +0000
Subject: [issue29116] Make str and bytes error messages on concatenation
 conform with other sequences
In-Reply-To: <1483124211.75.0.715869443236.issue29116@psf.upfronthosting.co.za>
Message-ID: <1483731614.31.0.263601588632.issue29116@psf.upfronthosting.co.za>


Changes by Raymond Hettinger <raymond.hettinger at gmail.com>:


----------
nosy: +rhettinger

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29116>
_______________________________________

From report at bugs.python.org  Fri Jan  6 14:41:41 2017
From: report at bugs.python.org (Jerome Leclanche)
Date: Fri, 06 Jan 2017 19:41:41 +0000
Subject: [issue29183] Unintuitive error handling in wsgiref when a crash
 happens in write() or close()
Message-ID: <1483731701.21.0.630358100063.issue29183@psf.upfronthosting.co.za>


New submission from Jerome Leclanche:

TLDR: When an error happens in the wsgiref's write() or close(), stack traces get inundated with irrelevant yet legitimate errors which make it hard to track down the real issue.

Couple of examples of this happening in practice:
https://stackoverflow.com/questions/28124461/how-do-i-solve-nonetype-object-is-not-callable-with-beautifulsoup-and-bottle

https://stackoverflow.com/questions/27518844/error-when-serving-html-with-wsgi

----

How to reproduce: The file I've attached reproduces the error on python 3.4, 3.5 and 3.6. The handler returns a string instead of bytes, which fails an early assert in handlers.py: write(data).

BaseHandler.run() triggers, gets as far as finish_response(), which triggers the above AssertionError. It falls into the except: block which attempts to handle_error(). But before doing that, it triggers self.close(), which sets result/headers/status/environ to None, bytes_sent to 0 and headers_sent to False.

Now when handle_error() triggers, `self.headers_sent` is False because of that, which attempts to trigger finish_response() *again*. This triggers a write() which attempts sending the headers, which checks client_is_modern(), subscripting `self.environ` which at that point has already been set to None. New error, which is caught in run()'s except block and re-raised after closing the connection (again).

I probably skipped some steps because the traceback is truly a mess. I think this could be improved, if only so that it doesn't get so confusing anymore.

----------
files: test_server.py
messages: 284844
nosy: jleclanche
priority: normal
severity: normal
status: open
title: Unintuitive error handling in wsgiref when a crash happens in write() or close()
versions: Python 3.4, Python 3.5, Python 3.6
Added file: http://bugs.python.org/file46179/test_server.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29183>
_______________________________________

From report at bugs.python.org  Fri Jan  6 14:48:57 2017
From: report at bugs.python.org (Xavier de Gaye)
Date: Fri, 06 Jan 2017 19:48:57 +0000
Subject: [issue29184] skip tests of test_socketserver when bind() raises
 PermissionError (non-root user on Android)
Message-ID: <1483732137.12.0.493215743488.issue29184@psf.upfronthosting.co.za>


New submission from Xavier de Gaye:

This happens on Android for a non-root user.  Multiple tests fail in test_socketserver with identical backtraces, only the first one is listed here.

======================================================================                    [905/2616]
ERROR: test_ForkingUnixDatagramServer (test.test_socketserver.SocketServerTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/test/test_socketserver.py", line 243, in test_Fork
ingUnixDatagramServer
    self.dgram_examine)
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/test/support/__init__.py", line 2040, in decorator
    return func(*args)
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/test/test_socketserver.py", line 121, in run_serve
r
    svrcls, hdlrbase)
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/test/test_socketserver.py", line 114, in make_serv
er
    server = MyServer(addr, MyHandler)
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/socketserver.py", line 452, in __init__
    self.server_bind()
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/socketserver.py", line 466, in server_bind
    self.socket.bind(self.server_address)
PermissionError: [Errno 13] Permission denied

----------
assignee: xdegaye
components: Tests
messages: 284845
nosy: xdegaye
priority: normal
severity: normal
stage: needs patch
status: open
title: skip tests of test_socketserver when bind() raises PermissionError (non-root user on Android)
type: behavior
versions: Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29184>
_______________________________________

From report at bugs.python.org  Fri Jan  6 14:53:57 2017
From: report at bugs.python.org (Xavier de Gaye)
Date: Fri, 06 Jan 2017 19:53:57 +0000
Subject: [issue29177] skip tests of test_logging when bind() raises
 PermissionError (non-root user on Android)
In-Reply-To: <1483701262.25.0.0295401910206.issue29177@psf.upfronthosting.co.za>
Message-ID: <1483732437.68.0.038667359829.issue29177@psf.upfronthosting.co.za>


Xavier de Gaye added the comment:

I have split this issue, issue 29184 is for fixing the tests on test_socketserver and this issue is for fixing the tests on test_logging.

----------
assignee: xdegaye -> 
title: skip tests using socketserver.UnixStreamServer when bind() raises PermissionError -> skip tests of test_logging when bind() raises PermissionError (non-root user on Android)

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29177>
_______________________________________

From report at bugs.python.org  Fri Jan  6 15:03:39 2017
From: report at bugs.python.org (Xavier de Gaye)
Date: Fri, 06 Jan 2017 20:03:39 +0000
Subject: [issue29177] skip tests of test_logging when bind() raises
 PermissionError (non-root user on Android)
In-Reply-To: <1483701262.25.0.0295401910206.issue29177@psf.upfronthosting.co.za>
Message-ID: <1483733019.59.0.536178827208.issue29177@psf.upfronthosting.co.za>


Xavier de Gaye added the comment:

> Do you want to take this on, or do you want me to look at it?

I would be very grateful if you would handle that :)
But if you cannot spare the time, I can give it a try.
One point I forgot to mention is that DatagramHandlerTest and SysLogHandlerTest also fail with PermissionError when run as their subclass UnixDatagramHandlerTest (resp. UnixSysLogHandlerTest).

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29177>
_______________________________________

From report at bugs.python.org  Fri Jan  6 15:47:31 2017
From: report at bugs.python.org (Jonathan Roach)
Date: Fri, 06 Jan 2017 20:47:31 +0000
Subject: [issue29175] Tutorial links to file object methods are broken.
In-Reply-To: <1483696723.95.0.735667565924.issue29175@psf.upfronthosting.co.za>
Message-ID: <1483735651.49.0.411905360251.issue29175@psf.upfronthosting.co.za>


Jonathan Roach added the comment:

OK, I understand that the older versions aren't going to be revised - that makes sense.

I think part of the reason I submitted this is the reader's path from the open() documentation to the most relevant part of the io documentation (the interface) is long and narrow: open()->file object(in the Glossary)->io(see note)..and then scroll several pages down.
Note: the link to io from the 'file object' glossary entry is visually tiny, and in last paragraph - effectively making it almost invisible. This also seems to be the only place in the chain of links from open() which gets you to the io documentation.
When trying to find the interface documentation I tried the visually bigger, earlier links and ended up like I was being given the run-around through the various glossary entries.
As a documentation user I'd ideally like a direct link (which is a bit visually bigger than 'io') from open() to the relevant, interface section of the io documentation. However, upstanding you have a better feel for the tone of the documentation, I can see you might take a different route.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29175>
_______________________________________

From report at bugs.python.org  Fri Jan  6 15:55:11 2017
From: report at bugs.python.org (Sworddragon)
Date: Fri, 06 Jan 2017 20:55:11 +0000
Subject: [issue19977] Use "surrogateescape" error handler for sys.stdin and
 sys.stdout on UNIX for the C locale
In-Reply-To: <1386952820.2.0.77364136667.issue19977@psf.upfronthosting.co.za>
Message-ID: <1483736111.14.0.512469519934.issue19977@psf.upfronthosting.co.za>


Sworddragon added the comment:

Bug #28180 has caused me to make a look at the "encoding" issue this and the tickets before have tried to solve more or less. Being a bit unsure what the root cause and intention for all this was I'm now at a point to actually check this ticket. Here is an example code (executed with Python 3.5.3 RC1 by having LANG set to C):

import sys
sys.stdout.write('?')


I thought with the surrogateescape error handler now being used for sys.stdout this would not throw an exception but I'm getting this:

UnicodeEncodeError: 'ascii' codec can't encode character '\xe4' in position 0: ordinal not in range(128)

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue19977>
_______________________________________

From report at bugs.python.org  Fri Jan  6 16:18:27 2017
From: report at bugs.python.org (Xavier de Gaye)
Date: Fri, 06 Jan 2017 21:18:27 +0000
Subject: [issue29185] test_distutils fails on Android API level 24
Message-ID: <1483737507.74.0.0285105710282.issue29185@psf.upfronthosting.co.za>


New submission from Xavier de Gaye:

======================================================================
ERROR: test_tarfile_vs_tar (distutils.tests.test_archive_util.ArchiveUtilTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/distutils/tests/test_archive_util.py", line 170, i
n test_tarfile_vs_tar
    spawn(gzip_cmd)
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/distutils/spawn.py", line 36, in spawn
    _spawn_posix(cmd, search_path, dry_run=dry_run)
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/distutils/spawn.py", line 159, in _spawn_posix
    % (cmd, exit_status))
distutils.errors.DistutilsExecError: command 'gzip' failed with exit status 1

======================================================================
FAIL: test_copy_file_hard_link (distutils.tests.test_file_util.FileUtilTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/distutils/tests/test_file_util.py", line 88, in te
st_copy_file_hard_link
    self.assertTrue(os.path.samestat(st2, st3), (st2, st3))
AssertionError: False is not true : (os.stat_result(st_mode=33206, st_ino=15948, st_dev=64800, st_nl
ink=1, st_uid=2000, st_gid=2000, st_size=12, st_atime=1483691935, st_mtime=1483691935, st_ctime=1483
691935), os.stat_result(st_mode=33206, st_ino=15949, st_dev=64800, st_nlink=1, st_uid=2000, st_gid=2
000, st_size=12, st_atime=1483691935, st_mtime=1483691935, st_ctime=1483691935))

----------------------------------------------------------------------
Ran 236 tests in 1.885s

FAILED (failures=1, errors=1, skipped=38)
test test_distutils failed

----------
assignee: xdegaye
components: Tests
messages: 284850
nosy: dstufft, eric.araujo, xdegaye
priority: normal
severity: normal
stage: needs patch
status: open
title: test_distutils fails on Android API level 24
type: behavior
versions: Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29185>
_______________________________________

From report at bugs.python.org  Fri Jan  6 16:20:04 2017
From: report at bugs.python.org (Larry Hastings)
Date: Fri, 06 Jan 2017 21:20:04 +0000
Subject: [issue29125] Shell injection via TIX_LIBRARY when using tkinter.tix
In-Reply-To: <1483210830.52.0.920210756798.issue29125@psf.upfronthosting.co.za>
Message-ID: <1483737604.17.0.425157335304.issue29125@psf.upfronthosting.co.za>


Larry Hastings added the comment:

I don't understand the fix.  Does this really prevent the injection?

I would fix it this way:

    if tixlib is not None and os.path.exists(tixlib):

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29125>
_______________________________________

From report at bugs.python.org  Fri Jan  6 16:22:11 2017
From: report at bugs.python.org (Larry Hastings)
Date: Fri, 06 Jan 2017 21:22:11 +0000
Subject: [issue29006] 2.7.13 _sqlite more prone to "database table is locked"
In-Reply-To: <1482095939.25.0.891516594682.issue29006@psf.upfronthosting.co.za>
Message-ID: <1483737731.83.0.368525928954.issue29006@psf.upfronthosting.co.za>


Larry Hastings added the comment:

FYI I'm keeping an eye on this for possible cherry-picking into 3.5.3 final, depending on the resolution.  Reverting 030e100f048a work for me, assuming that's a reasonable solution.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29006>
_______________________________________

From report at bugs.python.org  Fri Jan  6 16:22:55 2017
From: report at bugs.python.org (Xavier de Gaye)
Date: Fri, 06 Jan 2017 21:22:55 +0000
Subject: [issue29181] skip tests that raise PermissionError in test_tarfile
 (non-root user on Android)
In-Reply-To: <1483720295.96.0.810571843517.issue29181@psf.upfronthosting.co.za>
Message-ID: <1483737775.43.0.616345805179.issue29181@psf.upfronthosting.co.za>


Changes by Xavier de Gaye <xdegaye at gmail.com>:


----------
nosy: +lars.gustaebel

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29181>
_______________________________________

From report at bugs.python.org  Fri Jan  6 16:27:54 2017
From: report at bugs.python.org (Dmitry Shachnev)
Date: Fri, 06 Jan 2017 21:27:54 +0000
Subject: [issue28401] Don't support the PEP384 stable ABI in pydebug builds
In-Reply-To: <1476034292.17.0.316209799179.issue28401@psf.upfronthosting.co.za>
Message-ID: <1483738074.33.0.161831301622.issue28401@psf.upfronthosting.co.za>


Dmitry Shachnev added the comment:

[Matthias Klose (doko) 2016-10-27 15:45]
> I'm not sure that you really want this, because it would make it impossible to build an extension for the stable ABI for a debug build.

It looks like that is already impossible:

/usr/include/python3.5dm/object.h:65:2: error: #error Py_LIMITED_API is incompatible with Py_DEBUG, Py_TRACE_REFS, and Py_REF_DEBUG
 #error Py_LIMITED_API is incompatible with Py_DEBUG, Py_TRACE_REFS, and Py_REF_DEBUG
  ^~~~~

So in my opinion Stefano's patch makes sense.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28401>
_______________________________________

From report at bugs.python.org  Fri Jan  6 16:30:19 2017
From: report at bugs.python.org (Xavier de Gaye)
Date: Fri, 06 Jan 2017 21:30:19 +0000
Subject: [issue26865] Meta-issue: support of the android platform
In-Reply-To: <1461685011.99.0.617135447086.issue26865@psf.upfronthosting.co.za>
Message-ID: <1483738219.06.0.319889054387.issue26865@psf.upfronthosting.co.za>


Xavier de Gaye added the comment:

issue #29176: /tmp does not exist on Android and is used by curses.window.putwin()
issue #29177: skip tests of test_logging when bind() raises PermissionError (non-root user on Android)
issue #29180: skip tests that raise PermissionError in test_os (non-root user on Android)
issue #29181: skip tests that raise PermissionError in test_tarfile (non-root user on Android)
issue #29184: skip tests of test_socketserver when bind() raises PermissionError (non-root user on Android)
issue #29185: test_distutils fails on Android API level 24

----------
dependencies: +/tmp does not exist on Android and is used by curses.window.putwin(), skip tests of test_logging when bind() raises PermissionError (non-root user on Android), skip tests of test_socketserver when bind() raises PermissionError (non-root user on Android), skip tests that raise PermissionError in test_os (non-root user on Android), skip tests that raise PermissionError in test_tarfile (non-root user on Android), test_distutils fails on Android API level 24

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue26865>
_______________________________________

From report at bugs.python.org  Fri Jan  6 16:32:18 2017
From: report at bugs.python.org (Terry J. Reedy)
Date: Fri, 06 Jan 2017 21:32:18 +0000
Subject: [issue29116] Make str and bytes error messages on concatenation
 conform with other sequences
In-Reply-To: <1483124211.75.0.715869443236.issue29116@psf.upfronthosting.co.za>
Message-ID: <1483738338.61.0.334044025484.issue29116@psf.upfronthosting.co.za>


Terry J. Reedy added the comment:

By default, error message wording changes are 'enhancements' that wait for the next x.y.0 release unless the current wording is positively wrong'.  This is different from doc changes because there are tests depending on error messages. 'Inconsistent' or 'awkward' is not the same as 'wrong'.

I do agree that the clipped "must be str, not bytes' is awkward.  What (which) is it that must be str?  A fleshed out and positive "can only concatenate str (not bytes) to str." would be clearer and educate users better.

As Serhiy said, the exact equivalent cannot be said for bytes.

>>> b'a' + bytearray(b'b')
b'ab'
>>> b'a' + memoryview(b'b')
b'ab'

However "Can only concatenate bytes, bytearray, or memoryview (not x) to bytes." would be good if this is in fact complete. I need to be educated on this ;-)  I was not sure about memoryview until I tried it.

----------
nosy: +terry.reedy
type: behavior -> enhancement
versions:  -Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29116>
_______________________________________

From report at bugs.python.org  Fri Jan  6 16:37:02 2017
From: report at bugs.python.org (Ezio Melotti)
Date: Fri, 06 Jan 2017 21:37:02 +0000
Subject: [issue2771] Test issue
In-Reply-To: <1210005645.74.0.283923986194.issue2771@psf.upfronthosting.co.za>
Message-ID: <1483738622.68.0.647046447198.issue2771@psf.upfronthosting.co.za>


Ezio Melotti added the comment:

test

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue2771>
_______________________________________

From report at bugs.python.org  Fri Jan  6 16:41:12 2017
From: report at bugs.python.org (Vinay Sajip)
Date: Fri, 06 Jan 2017 21:41:12 +0000
Subject: [issue29177] skip tests of test_logging when bind() raises
 PermissionError (non-root user on Android)
In-Reply-To: <1483701262.25.0.0295401910206.issue29177@psf.upfronthosting.co.za>
Message-ID: <1483738872.4.0.832367293379.issue29177@psf.upfronthosting.co.za>


Vinay Sajip added the comment:

I've added a patch that should handle these errors (including the SysLogHandlerTest, which wasn't reported, but I think the same logic applies).

To simulate failure during setup, uncomment one or more of the lines which says

    # raise ValueError('dummy error raised')

----------
assignee:  -> vinay.sajip
keywords: +patch
stage: needs patch -> patch review
Added file: http://bugs.python.org/file46180/issue-29177-01.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29177>
_______________________________________

From report at bugs.python.org  Fri Jan  6 16:44:59 2017
From: report at bugs.python.org (paul j3)
Date: Fri, 06 Jan 2017 21:44:59 +0000
Subject: [issue29030] argparse: choices override metavar
In-Reply-To: <1482281546.0.0.613026945443.issue29030@psf.upfronthosting.co.za>
Message-ID: <1483739099.37.0.839355045314.issue29030@psf.upfronthosting.co.za>


paul j3 added the comment:

Here's the method in HelpFormatter that creates the metavar:

    def _metavar_formatter(self, action, default_metavar):
        if action.metavar is not None:
            result = action.metavar
        elif action.choices is not None:
            choice_strs = [str(choice) for choice in action.choices]
            result = '{%s}' % ','.join(choice_strs)
        else:
            result = default_metavar

        def format(tuple_size):
            if isinstance(result, tuple):
                return result
            else:
                return (result, ) * tuple_size
        return format

So in order of priority it uses: 

    the explicit metavar parameter
    formatted choices
    a default metavar (normally derived from the dest)

The MetavarTypeHelpFormatter subclass changes how that default is derived.  In the same spirit, you could write your own Formatter subclass that changes the above method, and its priority order.

In your example, the use of choices in the 'usage' like looks good to me.  I agree that its use in the 'help' line does not look so good.  But don't forget that for your end user, the positional 'dest' (or name) has no value.  Only you as the programmer sees and uses it.

This is one of many implementation details that could be included in the argparse docs.  But for some users those docs are already too complex.  The existing docs are not a formal module reference.

----------
nosy: +paul.j3

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29030>
_______________________________________

From report at bugs.python.org  Fri Jan  6 16:51:15 2017
From: report at bugs.python.org (Terry J. Reedy)
Date: Fri, 06 Jan 2017 21:51:15 +0000
Subject: [issue29117] dir(<bound method>) should include dunder attributes of
 the unbound method
In-Reply-To: <1483134670.11.0.964598948516.issue29117@psf.upfronthosting.co.za>
Message-ID: <1483739475.35.0.280446852998.issue29117@psf.upfronthosting.co.za>


Terry J. Reedy added the comment:

I tested your last claim and it is true as far as I went.

>>> C.f.__annotations__
{'a': <class 'int'>}
>>> C().f.__annotations__
{'a': <class 'int'>}
>>> C.f.__code__
<code object f at 0x000002610BDB5150, file "<pyshell#7>", line 2>
>>> C().f.__code__
<code object f at 0x000002610BDB5150, file "<pyshell#7>", line 2>

----------
nosy: +terry.reedy
stage:  -> test needed
type:  -> behavior
versions: +Python 3.6, Python 3.7 -Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29117>
_______________________________________

From report at bugs.python.org  Fri Jan  6 16:54:52 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Fri, 06 Jan 2017 21:54:52 +0000
Subject: [issue29125] Shell injection via TIX_LIBRARY when using tkinter.tix
In-Reply-To: <1483210830.52.0.920210756798.issue29125@psf.upfronthosting.co.za>
Message-ID: <1483739692.56.0.432312637536.issue29125@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

Yes this prevents the injection.

The injection is possible because the patch is substituted in the string without any escaping. Your fix is not enough. The real path to a Tix installation can contain special characters: '\', '{' or '}'.

My patch first sets a path to a Tcl variable (there is no an injection, because special API is used instead of evaluating a generated script), and then use this variable in the script (unlike to Unix shell Tcl doesn't reparse the command after substituting variables).

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29125>
_______________________________________

From report at bugs.python.org  Fri Jan  6 17:08:35 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Fri, 06 Jan 2017 22:08:35 +0000
Subject: [issue29116] Make str and bytes error messages on concatenation
 conform with other sequences
In-Reply-To: <1483124211.75.0.715869443236.issue29116@psf.upfronthosting.co.za>
Message-ID: <1483740515.9.0.216182149864.issue29116@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

You can concatenate any object supporting the buffer protocol to bytes and bytearray.

Current error message for bytes looks awkward too, because it says "bytes to othertype" instead of "othertype to bytes".

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29116>
_______________________________________

From report at bugs.python.org  Fri Jan  6 17:13:45 2017
From: report at bugs.python.org (Vinay Sajip)
Date: Fri, 06 Jan 2017 22:13:45 +0000
Subject: [issue29177] skip tests of test_logging when bind() raises
 PermissionError (non-root user on Android)
In-Reply-To: <1483701262.25.0.0295401910206.issue29177@psf.upfronthosting.co.za>
Message-ID: <1483740825.04.0.941694332825.issue29177@psf.upfronthosting.co.za>


Vinay Sajip added the comment:

> including the SysLogHandlerTest, which wasn't reported

Sorry, I appear to have lost the ability to read :-(

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29177>
_______________________________________

From report at bugs.python.org  Fri Jan  6 17:23:46 2017
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 06 Jan 2017 22:23:46 +0000
Subject: [issue19977] Use "surrogateescape" error handler for sys.stdin and
 sys.stdout on UNIX for the C locale
In-Reply-To: <1386952820.2.0.77364136667.issue19977@psf.upfronthosting.co.za>
Message-ID: <1483741425.99.0.0990175724991.issue19977@psf.upfronthosting.co.za>


STINNER Victor added the comment:

"I thought with the surrogateescape error handler now being used for sys.stdout this would not throw an exception but I'm getting this: (...)"

Please see the two recently proposed PEP: Nick's PEP 538 and my PEP 540, both propose (two different) solutions to your issue, especially for the POSIX locale (aka "C" locale).

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue19977>
_______________________________________

From report at bugs.python.org  Fri Jan  6 17:29:08 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Fri, 06 Jan 2017 22:29:08 +0000
Subject: [issue29178] Adding bytes.frombuffer(byteslike) constructor
In-Reply-To: <1483722813.74.0.841750617729.issue29178@psf.upfronthosting.co.za>
Message-ID: <2919773.es5mQSBHGX@raxxla>


Serhiy Storchaka added the comment:

> Which virtually no one follows :(

Sad. But adding bytes.frombuffer() wouldn't make it magically used. If you are 
aware of the problem, you can use the above two-liner as well as 
bytes.frombuffer(). You even can use it in the current code with older Python 
releases, unlike to bytes.frombuffer() which would need 3.7.

> Any protocol parsing code has a lot of slicing.

How much code you expect to update with bytes.frombuffer()? And why not use the 
above two-liner instead?

> > There is also a problem with returned type for subclasses (this is always
> > a problem for alternate constructors).
> Good point. How do we usually solve this in CPython?

It is deemed that returning an instance of a subclass is more preferable. 
Otherwise you could use separate function rather of a class method. But it is 
not easy. You need either pass a memoryview or bytes instance to class 
constructor, or (only for mutable arrays) create an empty instance and 
concatenate a buffer to it.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29178>
_______________________________________

From report at bugs.python.org  Fri Jan  6 17:42:51 2017
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 06 Jan 2017 22:42:51 +0000
Subject: [issue19764] subprocess: use PROC_THREAD_ATTRIBUTE_HANDLE_LIST with
 STARTUPINFOEX on Windows Vista
In-Reply-To: <1385370115.99.0.757096187315.issue19764@psf.upfronthosting.co.za>
Message-ID: <1483742571.33.0.186013652789.issue19764@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Python already has a multiprocessing module which is able to pass handles (maybe also FD? I don't know) to child processes on Windows. I found some code in Lib/multiprocessing/reduction.py:
- duplicate()
- steal_handle()
- send_handle()

But the design doesn't really fit the subprocess module, since this design requires that the child process communicates with the parent process. On UNIX, fork()+exec() is used, so we can execute a few instructions after fork, which allows to pass an exception from the child to the parent. On Windows, CreateProcess() is used which doesn't allow directly to execute code before running the final child process.

The PEP 446 describes a solution using a wrapper process, so parent+wrapper+child, 3 processes. IMHO the best design for subprocess is really PROC_THREAD_ATTRIBUTE_HANDLE_LIST.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue19764>
_______________________________________

From report at bugs.python.org  Fri Jan  6 17:51:51 2017
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 06 Jan 2017 22:51:51 +0000
Subject: [issue19764] subprocess: use PROC_THREAD_ATTRIBUTE_HANDLE_LIST with
 STARTUPINFOEX on Windows Vista
In-Reply-To: <1385370115.99.0.757096187315.issue19764@psf.upfronthosting.co.za>
Message-ID: <1483743111.59.0.00714880377339.issue19764@psf.upfronthosting.co.za>


STINNER Victor added the comment:

I dislike adding a lpAttributeList attribute: it's too close to the exact implementation of Windows may change in the future. I would prefer a more high level API.

Since the only known use case today is to pass handles, I propose to focus on this use case: add a new pass_handles parameter to Popen, similar to pass_fds.

I see that your patch is able to set close_fds to True on Windows: great job! It would be a great achievement to finally fix this last known race condition of subprocess on Windows!

So thank you for working on this!


> As for pass_fds: as you noted, it has it's own share of complexities and issues and I think it's best to leave it to a separate patch/issue.

pass_fds would be "nice to have", but I prefer to stick first to the native and well supported handles on Windows. For me, using file descriptors on Windows is more a "hack" to be able to write code working on Windows and UNIX, but since it's not natively supported on Windows, it comes with own set of issues.

IMHO it's better supported to work on handles.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue19764>
_______________________________________

From report at bugs.python.org  Fri Jan  6 17:54:44 2017
From: report at bugs.python.org (Brett Cannon)
Date: Fri, 06 Jan 2017 22:54:44 +0000
Subject: [issue2771] Test issue
In-Reply-To: <1210005645.74.0.283923986194.issue2771@psf.upfronthosting.co.za>
Message-ID: <1483743284.83.0.167737857602.issue2771@psf.upfronthosting.co.za>


Changes by Brett Cannon <brett at python.org>:


----------
pull_requests: +10

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue2771>
_______________________________________

From report at bugs.python.org  Fri Jan  6 17:54:45 2017
From: report at bugs.python.org (ita1024)
Date: Fri, 06 Jan 2017 22:54:45 +0000
Subject: [issue29174] 'NoneType' object is not callable in subprocess.py
In-Reply-To: <1483688938.47.0.507597836624.issue29174@psf.upfronthosting.co.za>
Message-ID: <1483743285.2.0.241375657716.issue29174@psf.upfronthosting.co.za>


ita1024 added the comment:

The point #3 was referring to the new requirement for an atexit handler in order to not only kill the processes but to also wait for them at interpreter shutdown. The sub-processes (and associated resources) in the example are definitely freed as the parent process is terminating.

The recommended handler is not even always desirable (spawning daemon processes, key agents), it increases the code verbosity, impacts performance, and can even cause problems as child processes cannot always be waited on reliably (python 2 but also child -D state and platform-specific restrictions).

I suggest to disable such warnings during interpreter shutdown.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29174>
_______________________________________

From report at bugs.python.org  Fri Jan  6 17:55:58 2017
From: report at bugs.python.org (YoSTEALTH)
Date: Fri, 06 Jan 2017 22:55:58 +0000
Subject: [issue29186] TimeoutError isn't being raised?
Message-ID: <1483743358.76.0.0752156268987.issue29186@psf.upfronthosting.co.za>


New submission from YoSTEALTH:

TimeoutError isn't being raised?

My Python Version: 3.5.1 (64bit, linux)

# Document:
https://docs.python.org/3/library/exceptions.html#TimeoutError
""" exception TimeoutError
    Raised when a system function timed out at the system level. Corresponds to errno ETIMEDOUT.
New in version 3.3: All the above OSError subclasses were added.
See also PEP 3151 - Reworking the OS and IO exception hierarchy """

# PEP: According to pep-3151
link: https://www.python.org/dev/peps/pep-3151/
""" TimeoutError : connection timed out (ETIMEDOUT); this can be re-cast as a generic timeout exception, replacing socket.timeout and also useful for other types of timeout (for example in Lock.acquire()) """


# This Does NOT Work:
def Send(conn, data):
    # Set Timeout.
    conn.settimeout(3.0)
    try:
        while data:
            sent = conn.send(data)
            data = data[sent:]
    except TimeoutError as e:
        print("TimeoutError:", e)  #
        close_connection()
    else:
        pass  # Do Stuff...


# This Works
def Send(conn, data):
    # Set Timeout.
    conn.settimeout(3.0)
    try:
        while data:
            sent = conn.send(data)
            data = data[sent:]
    except socket.timeout as e:
        print("socket.timeout:", e)  # socket.timeout: timed out
        close_connection()
    else:
        pass  # Do Stuff...


# This Works
def Send(conn, data):
    # Set Timeout.
    conn.settimeout(3.0)
    try:
        while data:
            sent = conn.send(data)
            data = data[sent:]
    except OSError as e:
        print('ERROR Send:', e)  # ERROR Send: timed out
        close_connection()
    else:
        pass  # Do Stuff...


According to PEP "TimeoutError" is suppose to replace "socket.timeout" but it doesn't seem to work! Any ideas why?

----------
messages: 284868
nosy: YoSTEALTH
priority: normal
severity: normal
status: open
title: TimeoutError isn't being raised?
type: behavior
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29186>
_______________________________________

From report at bugs.python.org  Fri Jan  6 18:06:54 2017
From: report at bugs.python.org (Matt Dodge)
Date: Fri, 06 Jan 2017 23:06:54 +0000
Subject: [issue29187] Pickle failure is raising AttributeError and not
 PicklingError
Message-ID: <1483744014.75.0.396569327718.issue29187@psf.upfronthosting.co.za>


New submission from Matt Dodge:

When failing to pickle something (like a locally scoped class) the documentation indicates that a PicklingError should be raised.
Doc links:
 - https://docs.python.org/3/library/pickle.html?highlight=pickle#pickle-picklable
 - https://docs.python.org/3.5/library/pickle.html#what-can-be-pickled-and-unpickled

However, instead I'm seeing AttributeError get raised instead, starting in Python 3.5. In Python 3.4 PicklingErrror was raised, as expected.

To reproduce, use the following file <pickletest.py>
    def func():
        class C: pass
        return C
    import pickle
    pickle.dumps(func()())

In Python 3.4 you see:
Traceback (most recent call last):
  File "pickletest.py", line 5, in <module>
    pickle.dumps(func()())
_pickle.PicklingError: Can't pickle <class '__main__.func.<locals>.C'>: attribute lookup C on __main__ failed

But in 3.5/3.6 you see:
Traceback (most recent call last):
  File "pickletest.py", line 5, in <module>
    pickle.dumps(func()())
AttributeError: Can't pickle local object 'func.<locals>.C'

I don't necessarily mind that a different exception is being raised, but how should we be handling exceptions while pickling? Catch all exceptions? That doesn't feel right to me, but if we're trying to pickle data out of our control I'm not sure what else to do.

FYI, the UnpicklingError documentation (https://docs.python.org/3/library/pickle.html?highlight=pickle#pickle.UnpicklingError) indicates that other exceptions can possibly be raised during unpickling. I assume that is more related to the fact that the pickled data may not fit into the current class/module structure though, so I think it's unrelated.

----------
components: Library (Lib)
messages: 284869
nosy: Matt.Dodge
priority: normal
severity: normal
status: open
title: Pickle failure is raising AttributeError and not PicklingError
versions: Python 3.4, Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29187>
_______________________________________

From report at bugs.python.org  Fri Jan  6 18:08:50 2017
From: report at bugs.python.org (Roundup Robot)
Date: Fri, 06 Jan 2017 23:08:50 +0000
Subject: [issue29157] random.c: Prefer getrandom() over getentropy(),
 handle ENOSYS in py_getentropy()
In-Reply-To: <1483551181.22.0.980001779001.issue29157@psf.upfronthosting.co.za>
Message-ID: <20170106230838.20290.43566.3A7C968A@psf.io>


Roundup Robot added the comment:

New changeset f8e24a0a1124 by Victor Stinner in branch '3.6':
Issue #29157: Prefer getrandom() over getentropy()
https://hg.python.org/cpython/rev/f8e24a0a1124

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29157>
_______________________________________

From report at bugs.python.org  Fri Jan  6 18:14:32 2017
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 06 Jan 2017 23:14:32 +0000
Subject: [issue29157] random.c: Prefer getrandom() over getentropy(),
 handle ENOSYS in py_getentropy()
In-Reply-To: <1483551181.22.0.980001779001.issue29157@psf.upfronthosting.co.za>
Message-ID: <1483744472.5.0.164407791697.issue29157@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Christian Heimes: "I'm doing a review now."

Follow-up on #python-dev (IRC):

<Crys> haypo: yes, I looked at the patch and did not see any obvious problem with it. Didn't I tell you?
<Crys> haypo: maybe I forgot :)

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29157>
_______________________________________

From report at bugs.python.org  Fri Jan  6 18:39:02 2017
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 06 Jan 2017 23:39:02 +0000
Subject: [issue29157] random.c: Prefer getrandom() over getentropy(),
 handle ENOSYS in py_getentropy()
In-Reply-To: <1483551181.22.0.980001779001.issue29157@psf.upfronthosting.co.za>
Message-ID: <1483745942.92.0.397447331564.issue29157@psf.upfronthosting.co.za>


STINNER Victor added the comment:

random-py35.patch: Patch for the 3.5 branch. My prepared commit message:
---
Issue #29157: Prefer getrandom() over getentropy()

Copy and then adapt Python/random.c from default branch. Difference between 3.5
and default branches:

* Python 3.5 only uses getrandom() in non-blocking mode: flags=GRND_NONBLOCK
* If getrandom() fails with EAGAIN: py_getrandom() immediately fails and
  remembers that getrandom() doesn't work.
* Python 3.5 has no _PyOS_URandomNonblock() function: _PyOS_URandom()
  works in non-blocking mode on Python 3.5
---

It seems like Python 3.5 is close to a release, I prefer to wait after the release to fix this issue. I don't think that many Linux distributions are affected, since the issue only occurs with glibc 2.24 which is very recent.

@Larry: Do you want this change in Python 3.5.3? The change is quite large.

----------
nosy: +larry
Added file: http://bugs.python.org/file46181/random-py35.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29157>
_______________________________________

From report at bugs.python.org  Fri Jan  6 18:41:30 2017
From: report at bugs.python.org (Sworddragon)
Date: Fri, 06 Jan 2017 23:41:30 +0000
Subject: [issue19977] Use "surrogateescape" error handler for sys.stdin and
 sys.stdout on UNIX for the C locale
In-Reply-To: <1386952820.2.0.77364136667.issue19977@psf.upfronthosting.co.za>
Message-ID: <1483746090.0.0.223760788579.issue19977@psf.upfronthosting.co.za>


Sworddragon added the comment:

The point is this ticket claims to be using the surrogateescape error handler for sys.stdout and sys.stdin for the C locale. I have never used surrogateescape explicitly before and thus have no experience for it and consulting the documentation mentions throwing an exception only for the strict error handler. I don't see anything that would make me think that surrogateescape would throw here an exception too. But maybe I'm just missing something.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue19977>
_______________________________________

From report at bugs.python.org  Fri Jan  6 18:49:16 2017
From: report at bugs.python.org (paul j3)
Date: Fri, 06 Jan 2017 23:49:16 +0000
Subject: [issue29030] argparse: choices override metavar
In-Reply-To: <1482281546.0.0.613026945443.issue29030@psf.upfronthosting.co.za>
Message-ID: <1483746556.23.0.725870129211.issue29030@psf.upfronthosting.co.za>


paul j3 added the comment:

subparsers is an example of choices displaying as the metavar.  From the documentation example:

usage: PROG [-h] [--foo] {a,b} ...

positional arguments:
  {a,b}   sub-command help

-----------------

To the main parser, the 'subparser' is a positional argument, with an optional 'dest' parameter (default is SUPPRESS), and {a,b} are the choices derived from the define subparser names.

The 'title' and 'description' parameters are used to create an Argument_group.

So any attempt to change how the metavar is created from choices has the potential of changing the subparser display.

That does suggest another formatting option - put your positional argument (with choices) in a custom Argument_Group.

In [658]: p=argparse.ArgumentParser()
In [659]: g=p.add_argument_group(title='lengths')
In [660]: g.add_argument('length',choices=[1,2,3]);
In [661]: p.print_help()
usage: ipython3 [-h] {1,2,3}

optional arguments:
  -h, --help  show this help message and exit

lengths:
  {1,2,3}

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29030>
_______________________________________

From report at bugs.python.org  Fri Jan  6 18:53:10 2017
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 06 Jan 2017 23:53:10 +0000
Subject: [issue19977] Use "surrogateescape" error handler for sys.stdin and
 sys.stdout on UNIX for the C locale
In-Reply-To: <1386952820.2.0.77364136667.issue19977@psf.upfronthosting.co.za>
Message-ID: <1483746790.6.0.097260579878.issue19977@psf.upfronthosting.co.za>


STINNER Victor added the comment:

> But maybe I'm just missing something.

This issue fixed exactly one use case: "List a directory into stdout" (similar to the UNIX "ls" or Windows "dir" commands):
https://www.python.org/dev/peps/pep-0540/#list-a-directory-into-stdout

Your use case is more "Display Unicode characters into stdout":
https://www.python.org/dev/peps/pep-0540/#display-unicode-characters-into-stdout

This use case is not supported by the issue. It should be fixed by PEP 538 or PEP 540.

Please join the happy discussion on the python-ideas mailing list to discuss how to "force UTF-8": this issue is closed, you shouldn't add new comments (other people will not see your comments).

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue19977>
_______________________________________

From report at bugs.python.org  Fri Jan  6 18:53:54 2017
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 06 Jan 2017 23:53:54 +0000
Subject: [issue29188] Backport random.c from Python 3.5 to Python 2.7
Message-ID: <1483746834.46.0.642378238515.issue29188@psf.upfronthosting.co.za>


New submission from STINNER Victor:

Python 3.6 uses the new getrandom() function/syscall on Linux and Solaris to get random bytes with no file descriptor: it prevents EMFILE and ENFILE errors or surprises when opening a first file (and looking at its file descriptor).

I propose to copy and adapt Python/random.c from Python 3.5 when Python 3.5 will be updated for the issue #29157. Python 2.7 requires extra changes:

* configure.ac: need to check linux/random.h in AC_CHECK_HEADERS()
* random.c: need to keep vms_urandom() function (Python 2.7 still supports VMS!)
* Python 2.7 doesn't implement the PEP 475 (EINTR) and so don't have functions like _Py_read() which handles EINTR for us.

See also the issue #29157 for the latest change in random.c: prefer getrandom() over getentropy() to support the glibc 2.24.

----------
messages: 284876
nosy: haypo
priority: normal
severity: normal
status: open
title: Backport random.c from Python 3.5 to Python 2.7
type: security
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29188>
_______________________________________

From report at bugs.python.org  Fri Jan  6 18:54:18 2017
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 06 Jan 2017 23:54:18 +0000
Subject: [issue29157] random.c: Prefer getrandom() over getentropy() to
 support glibc 2.24 on Linux
In-Reply-To: <1483551181.22.0.980001779001.issue29157@psf.upfronthosting.co.za>
Message-ID: <1483746858.05.0.0323557214115.issue29157@psf.upfronthosting.co.za>


Changes by STINNER Victor <victor.stinner at gmail.com>:


----------
title: random.c: Prefer getrandom() over getentropy(), handle ENOSYS in py_getentropy() -> random.c: Prefer getrandom() over getentropy() to support glibc 2.24 on Linux

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29157>
_______________________________________

From report at bugs.python.org  Fri Jan  6 19:06:57 2017
From: report at bugs.python.org (YoSTEALTH)
Date: Sat, 07 Jan 2017 00:06:57 +0000
Subject: [issue29186] TimeoutError isn't being raised?
In-Reply-To: <1483743358.76.0.0752156268987.issue29186@psf.upfronthosting.co.za>
Message-ID: <1483747617.51.0.819668756191.issue29186@psf.upfronthosting.co.za>


Changes by YoSTEALTH <ritesh at stealthcentral.com>:


----------
resolution:  -> not a bug
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29186>
_______________________________________

From report at bugs.python.org  Fri Jan  6 19:13:53 2017
From: report at bugs.python.org (Roundup Robot)
Date: Sat, 07 Jan 2017 00:13:53 +0000
Subject: [issue29023] Results of random.seed() call with integer argument
 should be claimed deterministic.
In-Reply-To: <1482237974.75.0.612413216492.issue29023@psf.upfronthosting.co.za>
Message-ID: <20170107001341.27780.99181.CAD1F858@psf.io>


Roundup Robot added the comment:

New changeset 7d6ebd206cd6 by Raymond Hettinger in branch '2.7':
Issue #29023:  Clarify that ints and longs are always deterministic seeds for random.
https://hg.python.org/cpython/rev/7d6ebd206cd6

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29023>
_______________________________________

From report at bugs.python.org  Fri Jan  6 19:14:13 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Sat, 07 Jan 2017 00:14:13 +0000
Subject: [issue29023] Results of random.seed() call with integer argument
 should be claimed deterministic.
In-Reply-To: <1482237974.75.0.612413216492.issue29023@psf.upfronthosting.co.za>
Message-ID: <1483748053.13.0.615598319336.issue29023@psf.upfronthosting.co.za>


Changes by Raymond Hettinger <raymond.hettinger at gmail.com>:


----------
resolution:  -> fixed
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29023>
_______________________________________

From report at bugs.python.org  Fri Jan  6 19:15:11 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Sat, 07 Jan 2017 00:15:11 +0000
Subject: [issue29188] Backport random.c from Python 3.5 to Python 2.7
In-Reply-To: <1483746834.46.0.642378238515.issue29188@psf.upfronthosting.co.za>
Message-ID: <1483748111.31.0.345629986046.issue29188@psf.upfronthosting.co.za>


Raymond Hettinger added the comment:

I think it is far too late to be making these kind of changes to 2.7.

----------
nosy: +rhettinger

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29188>
_______________________________________

From report at bugs.python.org  Fri Jan  6 19:34:46 2017
From: report at bugs.python.org (Martin Panter)
Date: Sat, 07 Jan 2017 00:34:46 +0000
Subject: [issue27632] build on AIX fails when builddir != srcdir,
 more than bad path to ld_so_aix
In-Reply-To: <1469608112.86.0.803958819971.issue27632@psf.upfronthosting.co.za>
Message-ID: <1483749286.81.0.488531048128.issue27632@psf.upfronthosting.co.za>


Martin Panter added the comment:

Regarding reopening Issue 10656, whatever you think is more appropriate. You just have to judge whether it is the same use case, the same code affected, etc.

Issue 16189 and Issue 25825 were about updating to match recent changes to directory names, and I thought we decided the changes were not applicable to 2.7.

Regarding LDSHARED vs BLDSHARED, isn?t this the same as Issue 28311, which lead to Issue 18235? If you can try the patch I mentioned at <https://bugs.python.org/issue18235#msg277745>, I suspect it may help. Let me know if you need help adapting the patch for 2.7.

----------
nosy: +martin.panter

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue27632>
_______________________________________

From report at bugs.python.org  Fri Jan  6 19:35:17 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Sat, 07 Jan 2017 00:35:17 +0000
Subject: [issue29179] Py_UNUSED is not documented
In-Reply-To: <1483712936.43.0.521596791461.issue29179@psf.upfronthosting.co.za>
Message-ID: <1483749317.89.0.683895353334.issue29179@psf.upfronthosting.co.za>


Changes by Raymond Hettinger <raymond.hettinger at gmail.com>:


----------
assignee: docs at python -> larry
nosy: +larry

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29179>
_______________________________________

From report at bugs.python.org  Fri Jan  6 20:45:21 2017
From: report at bugs.python.org (STINNER Victor)
Date: Sat, 07 Jan 2017 01:45:21 +0000
Subject: [issue29188] Backport random.c from Python 3.5 to Python 2.7
In-Reply-To: <1483746834.46.0.642378238515.issue29188@psf.upfronthosting.co.za>
Message-ID: <1483753521.01.0.872320724397.issue29188@psf.upfronthosting.co.za>


STINNER Victor added the comment:

> I think it is far too late to be making these kind of changes to 2.7.

I would prefer to use the "same code" (or almost) on all maintained versions of Python: 2.7, 3.5, 3.6 and 3.7. It should ease the maintenance for bugfixes and enhancements.

It seems like we want to backport security enhancements from Python 3 to Python 2.7: see the PEP 466. Copying random.c from Python 3 would add support for getrandom() which is nice to have since it avoids a private file descriptor (which causes many issues, even if the most important issues are already worked around in Python 2.7 using fstat()).

The minimum required change on Python 2.7 is to not use getentropy() on Linux to support the glibc 2.24: see attached getentropy_linux.patch if you don't want the backport.

----------
keywords: +patch
Added file: http://bugs.python.org/file46182/getentropy_linux.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29188>
_______________________________________

From report at bugs.python.org  Fri Jan  6 21:30:15 2017
From: report at bugs.python.org (Martin Panter)
Date: Sat, 07 Jan 2017 02:30:15 +0000
Subject: [issue29174] 'NoneType' object is not callable in subprocess.py
In-Reply-To: <1483688938.47.0.507597836624.issue29174@psf.upfronthosting.co.za>
Message-ID: <1483756215.02.0.7886864591.issue29174@psf.upfronthosting.co.za>


Martin Panter added the comment:

The ResourceWarning was added by Issue 26741.

I agree that there are legitimate reasons why pre-3.6 code may avoid calling Popen.wait() and equivalent. Victor opened Issue 27068 about adding a Popen.detach() method, which such code could use to opt out of the warning.

I don?t think there should be a special exemption for the warning at shutdown time. I think we should either:

1. Accept that you should never destroy a 3.6 Popen object without first ?waiting? on its child (or zombie), or:

2. Revert the warning, and in a future release (e.g. 3.7), add it back along with a way to opt out of the warning.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29174>
_______________________________________

From report at bugs.python.org  Fri Jan  6 21:33:14 2017
From: report at bugs.python.org (Sworddragon)
Date: Sat, 07 Jan 2017 02:33:14 +0000
Subject: [issue28180] sys.getfilesystemencoding() should default to utf-8
In-Reply-To: <1474024622.65.0.838685461995.issue28180@psf.upfronthosting.co.za>
Message-ID: <1483756394.75.0.895559599676.issue28180@psf.upfronthosting.co.za>


Sworddragon added the comment:

On looking into PEP 538 and PEP 540 I think PEP 540 is the way to go. It provides an option for a stronger encapsulation for the de-/encoding logic between the interpreter and the developer. Instead of caring about error handling the developer has now to care about mojibake handling (for me and maybe others that is explicitly preferred but maybe this depends on each individual). If I'm not wrong PEP 538 improves this for the output too but input handling will still suffer from the overall issue while PEP 540 does also solve this case. Also PEP 540 would not make the C locale and thus eventually some systems potentially unsupported (but it might be an acceptable trade-off if we should really go PEP 538).


Specific for PEP 540:

> The POSIX locale enables the UTF-8 mode

Non-strict I assume?


> UTF-8 /backslashreplace

Was/is the reason to use backslashreplace for sys.stderr to guarantee that the developer/user sees the error messages? Might it make sense to also use surrogateescape instead of backslashescape for sys.stderr in UTF-8 non-strict mode to be consistent here?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28180>
_______________________________________

From report at bugs.python.org  Fri Jan  6 21:42:41 2017
From: report at bugs.python.org (STINNER Victor)
Date: Sat, 07 Jan 2017 02:42:41 +0000
Subject: [issue29174] 'NoneType' object is not callable in subprocess.py
In-Reply-To: <1483756215.02.0.7886864591.issue29174@psf.upfronthosting.co.za>
Message-ID: <CAMpsgwaUMyPb4x6o3RJA6WJLzekLP2Q8_7t0WeoEz11uiAgdSQ@mail.gmail.com>


STINNER Victor added the comment:

Martin Panter:
> Victor opened Issue 27068 about adding a Popen.detach() method, which such code could use to opt out of the warning.

I opened the issue because you asked me to open it, but I'm not
convinced yet that the design would work. I don't understand yet who
is responsible of the pipes for example, especially pipes opened by
the Popen object itself (ex: stdout=PIPE), not passed to Popen
constructor. It's not as simple as getting a file descriptor as
file.detach() or socket.detach(), a Popen object is made of multiple
resources (pid and pipes at least).

> 2. Revert the warning, and in a future release (e.g. 3.7), add it back along with a way to opt out of the warning.

For this specific issue, the ResourceWarning is correct. I don't
understand the use case of explicitly turning this warning off on this
specific example?

If your output is flooded by ResourceWarning warnings, it's easy to
configure Python to ignore them. Example, simplest option: python3
-Wignore script.py. But you are only going to hide a real issue in
your code. ResourceWarning exists to help you to reduce your resource
consumption.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29174>
_______________________________________

From report at bugs.python.org  Fri Jan  6 21:53:55 2017
From: report at bugs.python.org (STINNER Victor)
Date: Sat, 07 Jan 2017 02:53:55 +0000
Subject: [issue28180] sys.getfilesystemencoding() should default to utf-8
In-Reply-To: <1483756394.75.0.895559599676.issue28180@psf.upfronthosting.co.za>
Message-ID: <CAMpsgwb6=MiYQ3a_BjUkG_eSZOqaUcKDcWqnTPqmwaqw4pxAxg@mail.gmail.com>


STINNER Victor added the comment:

Sworddragon added the comment:
> (for me and maybe others that is explicitly preferred but maybe this depends on each individual)

That's why the PEP 540 has options to enable to disable its UTF-8 mode(s).

> If I'm not wrong PEP 538 improves this for the output too but input handling will still suffer from the overall issue while PEP 540 does also solve this case.

The PEP 538 works fine if all inputs and outputs are encoded to UTF-8.
I understand that it's a deliberate choice to fail on
decoding/encoding error (to not use surrogateescape), but I can be
wrong.

> Also PEP 540 would not make the C locale and thus eventually some systems potentially unsupported (but it might be an acceptable trade-off if we should really go PEP 538).

What do you mean by "make the C locale"?

> Specific for PEP 540:
>
>> The POSIX locale enables the UTF-8 mode
>
> Non-strict I assume?

Yes, non strict.

I'm not sure of the name of each mode yet.

After having written the "Use Cases" section and especially the
Mojibake column of results, I consider the option of renaming the
"UTF-8 mode" to "YOLO mode".

>> UTF-8 /backslashreplace
>
> Was/is the reason to use backslashreplace for sys.stderr to guarantee that the developer/user sees the error messages?

Yes.

> Might it make sense to also use surrogateescape instead of backslashescape for sys.stderr in UTF-8 non-strict mode to be consistent here?

Using surrogateescape means that you pass through undecodable bytes
from inputs to stderr which can cause various kinds of bad surprises.

stderr is used to log errors. Getting a new error when trying to log
an error is kind of annoying.

Victor

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28180>
_______________________________________

From report at bugs.python.org  Sat Jan  7 00:04:12 2017
From: report at bugs.python.org (Berker Peksag)
Date: Sat, 07 Jan 2017 05:04:12 +0000
Subject: [issue28961] unittest.mock._Call ignores `name` parameter
In-Reply-To: <1481636868.17.0.0411235027553.issue28961@psf.upfronthosting.co.za>
Message-ID: <1483765452.82.0.0907002148509.issue28961@psf.upfronthosting.co.za>


Berker Peksag added the comment:

IIRC 3.5.3rc1 is already tagged so the 3.5 branch is open for 3.5.4. Anything that needs to be in 3.5.3 should be cherry-picked by the RM at this point if I'm not mistaken (and note that I don't have time check, but 3.5.3 may be the last bugfix release of 3.5 series so you may just skip 3.5 :))

Like I already said in msg284589, the test code doesn't follow the current style in Lib/unittest/test/testmock/testhelpers.py and it can be simplified like the following patch:

     def test_call_with_name(self):
-        self.assertEqual(
-            'foo',
-            _Call((), 'foo')[0],
-        )
-        self.assertEqual(
-            '',
-            _Call((('bar', 'barz'), ), )[0]
-        )
-        self.assertEqual(
-            '',
-            _Call((('bar', 'barz'), {'hello': 'world'}), )[0]
-        )
+        self.assertEqual(_Call((), 'foo')[0], 'foo')
+        self.assertEqual(_Call((('bar', 'barz'),),)[0], '')
+        self.assertEqual(_Call((('bar', 'barz'), {'hello': 'world'}),)[0], '')

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28961>
_______________________________________

From report at bugs.python.org  Sat Jan  7 00:14:19 2017
From: report at bugs.python.org (Sworddragon)
Date: Sat, 07 Jan 2017 05:14:19 +0000
Subject: [issue28180] sys.getfilesystemencoding() should default to utf-8
In-Reply-To: <1474024622.65.0.838685461995.issue28180@psf.upfronthosting.co.za>
Message-ID: <1483766059.89.0.867084207587.issue28180@psf.upfronthosting.co.za>


Sworddragon added the comment:

> What do you mean by "make the C locale"?

I was pointing to the Platform Support Changes of PEP 538.


> I'm not sure of the name of each mode yet.
>
> After having written the "Use Cases" section and especially the
> Mojibake column of results, I consider the option of renaming the
> "UTF-8 mode" to "YOLO mode".

Assumingly YOLO is meant to be negative: Things are whirling in my mind. Eventually you want to save your joker :>


> Using surrogateescape means that you pass through undecodable bytes
> from inputs to stderr which can cause various kinds of bad surprises.
>
> stderr is used to log errors. Getting a new error when trying to log
> an error is kind of annoying.

Hm, what bad surprise/error could appear that would not appear with backslashescape?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28180>
_______________________________________

From report at bugs.python.org  Sat Jan  7 00:24:57 2017
From: report at bugs.python.org (INADA Naoki)
Date: Sat, 07 Jan 2017 05:24:57 +0000
Subject: [issue28180] sys.getfilesystemencoding() should default to utf-8
In-Reply-To: <1474024622.65.0.838685461995.issue28180@psf.upfronthosting.co.za>
Message-ID: <1483766697.5.0.413483067372.issue28180@psf.upfronthosting.co.za>


INADA Naoki added the comment:

>> stderr is used to log errors. Getting a new error when trying to log
>> an error is kind of annoying.
>
> Hm, what bad surprise/error could appear that would not appear with backslashescape?

$ cat badfilename.py 
badfn = "?????".encode('euc-jp').decode('utf-8', 'surrogateescape')
print("bad filename:", badfn)

$ PYTHONIOENCODING=utf-8:backslashreplace python3 badfilename.py 
bad filename: \udca4\udcb3\udca4\udcf3\udca4?\udcc1\udca4\udccf

$ PYTHONIOENCODING=utf-8:surrogateescape python3 badfilename.py 
bad filename: ?????????

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28180>
_______________________________________

From report at bugs.python.org  Sat Jan  7 00:39:59 2017
From: report at bugs.python.org (Berker Peksag)
Date: Sat, 07 Jan 2017 05:39:59 +0000
Subject: [issue29182] Remove the warning in urllib docs that it doesn't do
 certificate validate by default.
In-Reply-To: <1483722117.81.0.319313681571.issue29182@psf.upfronthosting.co.za>
Message-ID: <1483767599.79.0.866501363274.issue29182@psf.upfronthosting.co.za>


Changes by Berker Peksag <berker.peksag at gmail.com>:


----------
stage: patch review -> commit review

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29182>
_______________________________________

From report at bugs.python.org  Sat Jan  7 00:49:55 2017
From: report at bugs.python.org (Berker Peksag)
Date: Sat, 07 Jan 2017 05:49:55 +0000
Subject: [issue29189] Broken indentation in FancyURLopener documentation
Message-ID: <1483768195.47.0.481112354487.issue29189@psf.upfronthosting.co.za>


New submission from Berker Peksag:

I noticed this while taking a look at urllib docs for issue 29182. Indentation of prompt_user_passwd() is broken and it looks like it's part of the note directive: https://docs.python.org/2/library/urllib.html#urllib.FancyURLopener.prompt_user_passwd

Python 3 version renders fine: https://docs.python.org/3/library/urllib.request.html#urllib.request.FancyURLopener.prompt_user_passwd

Here is a patch.

----------
assignee: docs at python
components: Documentation
files: docfix.diff
keywords: patch
messages: 284888
nosy: berker.peksag, docs at python, orsenthil
priority: normal
severity: normal
stage: patch review
status: open
title: Broken indentation in FancyURLopener documentation
type: behavior
versions: Python 2.7
Added file: http://bugs.python.org/file46183/docfix.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29189>
_______________________________________

From report at bugs.python.org  Sat Jan  7 00:53:46 2017
From: report at bugs.python.org (Senthil Kumaran)
Date: Sat, 07 Jan 2017 05:53:46 +0000
Subject: [issue29189] Broken indentation in FancyURLopener documentation
In-Reply-To: <1483768195.47.0.481112354487.issue29189@psf.upfronthosting.co.za>
Message-ID: <1483768426.87.0.446168583265.issue29189@psf.upfronthosting.co.za>


Senthil Kumaran added the comment:

The patch looks good to me, and can committed. Thanks!

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29189>
_______________________________________

From report at bugs.python.org  Sat Jan  7 01:14:31 2017
From: report at bugs.python.org (Roundup Robot)
Date: Sat, 07 Jan 2017 06:14:31 +0000
Subject: [issue29189] Broken indentation in FancyURLopener documentation
In-Reply-To: <1483768195.47.0.481112354487.issue29189@psf.upfronthosting.co.za>
Message-ID: <20170107061408.27780.64086.ACDB4519@psf.io>


Roundup Robot added the comment:

New changeset 31172ecb9e40 by Berker Peksag in branch '2.7':
Issue #29189: Fix broken indentation in FancyURLopener documentation
https://hg.python.org/cpython/rev/31172ecb9e40

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29189>
_______________________________________

From report at bugs.python.org  Sat Jan  7 01:15:01 2017
From: report at bugs.python.org (Berker Peksag)
Date: Sat, 07 Jan 2017 06:15:01 +0000
Subject: [issue29189] Broken indentation in FancyURLopener documentation
In-Reply-To: <1483768195.47.0.481112354487.issue29189@psf.upfronthosting.co.za>
Message-ID: <1483769701.69.0.570265498553.issue29189@psf.upfronthosting.co.za>


Berker Peksag added the comment:

Thanks for the review, Senthil.

----------
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29189>
_______________________________________

From report at bugs.python.org  Sat Jan  7 01:31:33 2017
From: report at bugs.python.org (Berker Peksag)
Date: Sat, 07 Jan 2017 06:31:33 +0000
Subject: [issue16026] csv.DictReader argument names documented incorrectly
In-Reply-To: <1348505583.77.0.632308357055.issue16026@psf.upfronthosting.co.za>
Message-ID: <1483770693.28.0.869797838811.issue16026@psf.upfronthosting.co.za>


Berker Peksag added the comment:

Thanks for the patches James and Greg!

----------
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed
versions:  -Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue16026>
_______________________________________

From report at bugs.python.org  Sat Jan  7 01:32:23 2017
From: report at bugs.python.org (Roundup Robot)
Date: Sat, 07 Jan 2017 06:32:23 +0000
Subject: [issue16026] csv.DictReader argument names documented incorrectly
In-Reply-To: <1348505583.77.0.632308357055.issue16026@psf.upfronthosting.co.za>
Message-ID: <20170107063035.26952.99555.6F543B6B@psf.io>


Roundup Robot added the comment:

New changeset 198edd926751 by Berker Peksag in branch '3.6':
Issue #16026: Fix parameter names of DictReader and DictWriter
https://hg.python.org/cpython/rev/198edd926751

New changeset 63c5531cfdf7 by Berker Peksag in branch 'default':
Issue #16026: Merge from 3.6
https://hg.python.org/cpython/rev/63c5531cfdf7

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue16026>
_______________________________________

From report at bugs.python.org  Sat Jan  7 01:49:43 2017
From: report at bugs.python.org (Berker Peksag)
Date: Sat, 07 Jan 2017 06:49:43 +0000
Subject: [issue29133] Minor inaccuracy in shlex.shlex punctuation_chars example
In-Reply-To: <1483362012.78.0.704413629859.issue29133@psf.upfronthosting.co.za>
Message-ID: <1483771783.19.0.436198408517.issue29133@psf.upfronthosting.co.za>


Berker Peksag added the comment:

I'd probably write it without the for loop:

    text = "a && b; c && d || e; f >'abc'; (def \"ghi\")"

    result = shlex.shlex(text)
    print(f"Old behavior: {list(result)}")

    result = shlex.shlex(text, punctuation_chars=True)
    print(f"New behavior: {list(result)}")

Or just:

    >>> import shlex
    >>> text = "a && b; c && d || e; f >'abc'; (def \"ghi\")"
    >>> list(shlex.shlex(text))
    ['a', '&', '&', 'b', ';', 'c', '&', '&', 'd', '|', '|', 'e', ';', 'f', '>', "'abc'", ';', '(', 'def', '"ghi"', ')']
    >>> list(shlex.shlex(text, punctuation_chars=True))
    ['a', '&&', 'b', ';', 'c', '&&', 'd', '||', 'e', ';', 'f', '>', "'abc'", ';', '(', 'def', '"ghi"', ')']

(Adding Vinay to nosy list to get his feedback since he wrote the original example.)

----------
nosy: +berker.peksag, vinay.sajip
stage:  -> patch review
type:  -> behavior

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29133>
_______________________________________

From report at bugs.python.org  Sat Jan  7 02:10:06 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sat, 07 Jan 2017 07:10:06 +0000
Subject: [issue29190] Avoid possible errors in comparing strings
Message-ID: <1483773006.32.0.276673146687.issue29190@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka:

PyUnicode_Compare() and PyUnicode_RichCompare() can raise an exception if one of arguments is not ready unicode object. The result is not always checked for error. Proposed patch gets rid of possible bugs. PyUnicode_Compare() and PyUnicode_RichCompare() in Modules/_pickle.c are replaced with _PyUnicode_EqualToASCIIString() and _PyUnicode_EqualToASCIIId() which never fail. Additional check is added in Modules/_decimal/_decimal.c to ensure that the string which is came from a user code is ready.

All other occurrences of PyUnicode_Compare() seems are called only with ready unicode objects.

----------
components: Extension Modules
files: unicode_compare.patch
keywords: patch
messages: 284895
nosy: serhiy.storchaka
priority: normal
severity: normal
stage: patch review
status: open
title: Avoid possible errors in comparing strings
type: behavior
versions: Python 3.5, Python 3.6, Python 3.7
Added file: http://bugs.python.org/file46184/unicode_compare.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29190>
_______________________________________

From report at bugs.python.org  Sat Jan  7 02:15:10 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sat, 07 Jan 2017 07:15:10 +0000
Subject: [issue29187] Pickle failure is raising AttributeError and not
 PicklingError
In-Reply-To: <1483744014.75.0.396569327718.issue29187@psf.upfronthosting.co.za>
Message-ID: <1483773310.6.0.36742235054.issue29187@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

Python implementation of pickle still raises PicklingError. Seems this was not intentional change.

>>> pickle._dumps(func()())
Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/pickle.py", line 918, in save_global
    obj2, parent = _getattribute(module, name)
  File "/home/serhiy/py/cpython/Lib/pickle.py", line 266, in _getattribute
    .format(name, obj))
AttributeError: Can't get local attribute 'func.<locals>.C' on <function func at 0xb7118d1c>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/serhiy/py/cpython/Lib/pickle.py", line 1544, in _dumps
    _Pickler(f, protocol, fix_imports=fix_imports).dump(obj)
  File "/home/serhiy/py/cpython/Lib/pickle.py", line 409, in dump
    self.save(obj)
  File "/home/serhiy/py/cpython/Lib/pickle.py", line 521, in save
    self.save_reduce(obj=obj, *rv)
  File "/home/serhiy/py/cpython/Lib/pickle.py", line 605, in save_reduce
    save(cls)
  File "/home/serhiy/py/cpython/Lib/pickle.py", line 476, in save
    f(self, obj) # Call unbound method with explicit self
  File "/home/serhiy/py/cpython/Lib/pickle.py", line 978, in save_type
    return self.save_global(obj)
  File "/home/serhiy/py/cpython/Lib/pickle.py", line 922, in save_global
    (obj, module_name, name))
_pickle.PicklingError: Can't pickle <class '__main__.func.<locals>.C'>: it's not found as __main__.func.<locals>.C

----------
components: +Extension Modules -Library (Lib)
nosy: +alexandre.vassalotti, serhiy.storchaka
stage:  -> patch review
type:  -> behavior
versions: +Python 3.7 -Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29187>
_______________________________________

From report at bugs.python.org  Sat Jan  7 02:25:53 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Sat, 07 Jan 2017 07:25:53 +0000
Subject: [issue29186] TimeoutError isn't being raised?
In-Reply-To: <1483743358.76.0.0752156268987.issue29186@psf.upfronthosting.co.za>
Message-ID: <1483773953.9.0.927358612082.issue29186@psf.upfronthosting.co.za>


Changes by Xiang Zhang <angwerzx at 126.com>:


----------
stage:  -> resolved

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29186>
_______________________________________

From report at bugs.python.org  Sat Jan  7 02:28:29 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sat, 07 Jan 2017 07:28:29 +0000
Subject: [issue29145] failing overflow checks in replace_*
In-Reply-To: <1483468602.77.0.231507148809.issue29145@psf.upfronthosting.co.za>
Message-ID: <1483774109.55.0.665668268431.issue29145@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

The code in PyUnicode_Join() checks on overflow after making an addition.

        sz += PyUnicode_GET_LENGTH(item);
        item_maxchar = PyUnicode_MAX_CHAR_VALUE(item);
        maxchar = Py_MAX(maxchar, item_maxchar);
        if (i != 0)
            sz += seplen;
        if (sz < old_sz || sz > PY_SSIZE_T_MAX) {
            PyErr_SetString(PyExc_OverflowError,
                            "join() result is too long for a Python string");
            goto onError;
        }

Maybe there are other cases.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29145>
_______________________________________

From report at bugs.python.org  Sat Jan  7 02:55:23 2017
From: report at bugs.python.org (Berker Peksag)
Date: Sat, 07 Jan 2017 07:55:23 +0000
Subject: [issue28647] python --help: -u is misdocumented as binary mode
In-Reply-To: <1478687165.61.0.754522524524.issue28647@psf.upfronthosting.co.za>
Message-ID: <1483775723.47.0.472108357171.issue28647@psf.upfronthosting.co.za>


Changes by Berker Peksag <berker.peksag at gmail.com>:


----------
nosy: +berker.peksag
stage:  -> patch review
type:  -> behavior
versions: +Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28647>
_______________________________________

From report at bugs.python.org  Sat Jan  7 03:16:35 2017
From: report at bugs.python.org (Segev Finer)
Date: Sat, 07 Jan 2017 08:16:35 +0000
Subject: [issue19764] subprocess: use PROC_THREAD_ATTRIBUTE_HANDLE_LIST with
 STARTUPINFOEX on Windows Vista
In-Reply-To: <1385370115.99.0.757096187315.issue19764@psf.upfronthosting.co.za>
Message-ID: <1483776995.1.0.375339851045.issue19764@psf.upfronthosting.co.za>


Segev Finer added the comment:

I removed previous_handle_list in _execute_child since I noticed subprocess already clobbers the other attributes in startupinfo anyhow.

I figured there will be some discussion about how to pass the handle list, so here's my two cents:

* subprocess already exposes a bit of Windows specific flags like creationflags and STARTUPINFO.

* Windows doesn't really break it's API in backwards incompatible ways often (Heck it barely breaks it ever, which is why we have so many Ex functions and reserved parameters :P).

* The _winapi module tries to expose WinAPI functions as is. So I implemented this as an internal attribute on STARTUPINFO, in the first version, since I wasn't sure we want this exposed to users, but I still wanted to try and mimic the original WinAPI functions internally. The lpAttributeList is a change requested by eryksun that brings it even closer to WinAPI and exposes it for further extension with additional attributes.

----------
Added file: http://bugs.python.org/file46185/windows-subprocess-close-fds-v3.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue19764>
_______________________________________

From report at bugs.python.org  Sat Jan  7 03:27:49 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sat, 07 Jan 2017 08:27:49 +0000
Subject: [issue29190] Avoid possible errors in comparing strings
In-Reply-To: <1483773006.32.0.276673146687.issue29190@psf.upfronthosting.co.za>
Message-ID: <1483777669.64.0.659151351008.issue29190@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

An alternative patch checks the result of PyUnicode_Compare() (as Xiang suggested) instead of checking the value before calling PyUnicode_Compare(). Stephan, what way do you prefer?

----------
nosy: +facundobatista, mark.dickinson, rhettinger, skrah
Added file: http://bugs.python.org/file46186/unicode_compare_2.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29190>
_______________________________________

From report at bugs.python.org  Sat Jan  7 03:41:17 2017
From: report at bugs.python.org (Nick Coghlan)
Date: Sat, 07 Jan 2017 08:41:17 +0000
Subject: [issue28180] sys.getfilesystemencoding() should default to utf-8
In-Reply-To: <1474024622.65.0.838685461995.issue28180@psf.upfronthosting.co.za>
Message-ID: <1483778477.18.0.402571304479.issue28180@psf.upfronthosting.co.za>


Nick Coghlan added the comment:

I just pushed an update to PEP 538 based on PEP 540 and the feedback in the linux-sig discussion: https://github.com/python/peps/commit/221099d8765125bbd798e869846b005bcca84b47

I'll be starting a thread for that on python-ideas shortly, but in the context of the discussion here:

* There are good reasons to go back to strict error handling by default on the standard streams when we're using UTF-8 as the default encoding rather than ASCII: https://www.python.org/dev/peps/pep-0538/#using-strict-error-handling-by-default
* The right overall answer might actually be to create a hybrid merger of the two PEPs, rather than seeing them as strictly competitors: https://www.python.org/dev/peps/pep-0538/#relationship-with-other-peps

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28180>
_______________________________________

From report at bugs.python.org  Sat Jan  7 03:44:03 2017
From: report at bugs.python.org (Spacemage 33)
Date: Sat, 07 Jan 2017 08:44:03 +0000
Subject: [issue29118] Randit
In-Reply-To: <1483159359.92.0.751907144442.issue29118@psf.upfronthosting.co.za>
Message-ID: <CA+oPQhkvngvH-2cYoBnHy0dvqHWapeRpJAd7=5s9W_zQWNzYZw@mail.gmail.com>


Spacemage 33 added the comment:

Thank you very much.

Now it works fine!

2016-12-31 5:42 GMT+01:00 Terry J. Reedy <report at bugs.python.org>:

>
> Terry J. Reedy added the comment:
>
> This issue has nothing to do with IDLE.  There is also no bug here.
> Python's random module has a randint function that works fine.  Either you
> or the book have misspelled 'randint' as 'randit' or the book is using its
> own random module and not the stdlib version.  Please post on python-list
> if you think you have found a bug.
>
> ----------
> resolution:  -> not a bug
> stage:  -> resolved
> status: open -> closed
>
> _______________________________________
> Python tracker <report at bugs.python.org>
> <http://bugs.python.org/issue29118>
> _______________________________________
>

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29118>
_______________________________________

From report at bugs.python.org  Sat Jan  7 03:48:55 2017
From: report at bugs.python.org (Segev Finer)
Date: Sat, 07 Jan 2017 08:48:55 +0000
Subject: [issue29191] liblzma is missing from pcbuild.sln
Message-ID: <1483778935.32.0.9795840646.issue29191@psf.upfronthosting.co.za>


New submission from Segev Finer:

liblzma is missing from pcbuild.sln. This causes the build of _lzma to fail when building the solution and not using build.bat.

Attached is a patch that fixes this for the default branch. Note that it also includes some missing project configurations for the _asyncio module which Visual Studio will add every single time you open the solution.

A similar patch is simple enough to create for older versions/branches. Just Add->Existing Project liblzma.vcxproj and make sure to revert any lines in the sln which Visual Studio added that you don't want to commit. (Like VisualStudioVersion = ...)

----------
components: Windows
files: add-liblzma-to-pcbuild-sln.patch
keywords: patch
messages: 284902
nosy: Segev Finer, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: liblzma is missing from pcbuild.sln
type: compile error
versions: Python 3.5, Python 3.6, Python 3.7
Added file: http://bugs.python.org/file46187/add-liblzma-to-pcbuild-sln.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29191>
_______________________________________

From report at bugs.python.org  Sat Jan  7 03:52:39 2017
From: report at bugs.python.org (Eryk Sun)
Date: Sat, 07 Jan 2017 08:52:39 +0000
Subject: [issue19764] subprocess: use PROC_THREAD_ATTRIBUTE_HANDLE_LIST with
 STARTUPINFOEX on Windows Vista
In-Reply-To: <1385370115.99.0.757096187315.issue19764@psf.upfronthosting.co.za>
Message-ID: <1483779159.93.0.545451929015.issue19764@psf.upfronthosting.co.za>


Eryk Sun added the comment:

> Python already has a multiprocessing module which is able to pass 
> handles (maybe also FD? I don't know) to child processes on 
> Windows.

Popen doesn't implement the undocumented CRT protocol that's used to smuggle the file-descriptor mapping in the STARTUPINFO cbReserved2 and lpReserved2 fields. This is a feature of the CRT's spawn and exec functions. For example:

    fdr, fdw = os.pipe()
    os.set_inheritable(fdw, True)
    os.spawnl(os.P_WAIT, os.environ['ComSpec'], 'cmd /c "echo spam >&%d"' % fdw)

    >>> os.read(fdr, 10)
    b'spam \r\n'

We don't have to worry about implementing fd inheritance so long as os.spawn* uses the CRT. Someone that needs this functionality can simply be instructed to use os.spawn.

> I dislike adding a lpAttributeList attribute: it's too close to 
> the exact implementation of Windows may change in the future.

If you're going to worry about lpAttributeList, why stop there? 
Aren't dwFlags, wShowWindow, hStdInput, hStdOutput, and hStdError also too close to the exact implementation? My thoughts when suggesting this were actually to make this as close to the underlying API as possible, and extensible to support other attributes if there's a demand for it. 

Passing a list of handles is atypical usage, and since Python and subprocess use file descriptors instead of Windows handles, I prefer isolating this in a Windows structure such as STARTUPINFO, rather than adding even more confusion to Popen's constructor.

> Since the only known use case today is to pass handles

In the review of the first patch, I listed 3 additional attributes that might be useful to add in 3.7: IDEAL_PROCESSOR, GROUP_AFFINITY, and PREFERRED_NODE (simplified by the fact that 3.7 no longer supports Vista). Currently the way to set the latter two is to use the built-in `start` command of the cmd shell.

> I propose to focus on this use case: add a new pass_handles parameter
> to Popen, similar to pass_fds.

This is a messy situation. Python 3's file I/O is built on the CRT's POSIX layer. If it had been implemented directly on the Windows API using handles, then pass_fds would obviously use handles. That's the current situation with socket module because Winsock makes no attempt to hide AFD handles behind POSIX file descriptors. 

Popen's constructor accepts file descriptors -- not Windows handles -- for its stdin, stdout, and stderr arguments, and the parameter to control inheritance is named "close_fds". It seems out of place to add a "pass_handles" parameter.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue19764>
_______________________________________

From report at bugs.python.org  Sat Jan  7 04:21:58 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Sat, 07 Jan 2017 09:21:58 +0000
Subject: [issue29190] Avoid possible errors in comparing strings
In-Reply-To: <1483773006.32.0.276673146687.issue29190@psf.upfronthosting.co.za>
Message-ID: <1483780918.78.0.480397454552.issue29190@psf.upfronthosting.co.za>


Xiang Zhang added the comment:

Paste my point here:

I prefer checking the result PyUnicode_Compare to see it's a success or failure.
getrandom doesn't use any lower level unicode operations so it doesn't get a
responsibility to ready the unicode.

----------
nosy: +xiang.zhang

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29190>
_______________________________________

From report at bugs.python.org  Sat Jan  7 06:12:07 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sat, 07 Jan 2017 11:12:07 +0000
Subject: [issue29192] Remove deprecated features from http.cookies
Message-ID: <1483787527.13.0.320164851178.issue29192@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka:

Some behavior of http.cookies was deprecated in 3.5. Following patch finishes the deprecation period and removes the code for handling deprecated features. Former warnings become errors.

----------
components: Library (Lib)
files: cookies_finish_deprecation.patch
keywords: patch
messages: 284905
nosy: demian.brecht, serhiy.storchaka
priority: normal
severity: normal
stage: patch review
status: open
title: Remove deprecated features from http.cookies
type: enhancement
versions: Python 3.7
Added file: http://bugs.python.org/file46188/cookies_finish_deprecation.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29192>
_______________________________________

From report at bugs.python.org  Sat Jan  7 06:31:27 2017
From: report at bugs.python.org (Vinay Sajip)
Date: Sat, 07 Jan 2017 11:31:27 +0000
Subject: [issue29133] Minor inaccuracy in shlex.shlex punctuation_chars example
In-Reply-To: <1483362012.78.0.704413629859.issue29133@psf.upfronthosting.co.za>
Message-ID: <1483788687.11.0.948047863374.issue29133@psf.upfronthosting.co.za>


Vinay Sajip added the comment:

I'm fine with removing the loop, as there are only ever going to be the two cases - so Berker's suggestion would seem to cover both doctest and unittest cases.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29133>
_______________________________________

From report at bugs.python.org  Sat Jan  7 06:35:11 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sat, 07 Jan 2017 11:35:11 +0000
Subject: [issue29193] Remove support of format_string as keyword argument in
 string.Formatter().format()
Message-ID: <1483788911.55.0.222681922661.issue29193@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka:

Passing a format string as a keyword argument to string.Formatter.format() was deprecated in 3.5 (issue23671). Proposed patch finishes the deprecation period and converts a warning to an error.

Python 3.5-3.6:

>>> string.Formatter().format(format_string='foo: {foo}', foo=123)
__main__:1: DeprecationWarning: Passing 'format_string' as keyword argument is deprecated
'foo: 123'

Python 3.7:

>>> string.Formatter().format(format_string='foo: {foo}', foo=123)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/serhiy/py/cpython/Lib/string.py", line 179, in format
    "argument: 'format_string'") from None
TypeError: format() missing 1 required positional argument: 'format_string'

----------
components: Library (Lib)
files: string_formatter_positional_only.patch
keywords: patch
messages: 284907
nosy: serhiy.storchaka
priority: normal
severity: normal
stage: patch review
status: open
title: Remove support of format_string as keyword argument in string.Formatter().format()
type: enhancement
versions: Python 3.7
Added file: http://bugs.python.org/file46189/string_formatter_positional_only.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29193>
_______________________________________

From report at bugs.python.org  Sat Jan  7 06:43:47 2017
From: report at bugs.python.org (Nick Coghlan)
Date: Sat, 07 Jan 2017 11:43:47 +0000
Subject: [issue28180] sys.getfilesystemencoding() should default to utf-8
In-Reply-To: <1474024622.65.0.838685461995.issue28180@psf.upfronthosting.co.za>
Message-ID: <1483789427.67.0.981394666863.issue28180@psf.upfronthosting.co.za>


Nick Coghlan added the comment:

While the attached PEP 538 patches include their own tests, the uploaded pep538-check-click.sh script is the one I've been using to check that the changes have the desired effect of letting click "just work", even when the nominal locale is cleared, explicitly set to C, or explicitly set to POSIX.

----------
Added file: http://bugs.python.org/file46190/pep538-check-click.sh

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28180>
_______________________________________

From report at bugs.python.org  Sat Jan  7 06:58:58 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Sat, 07 Jan 2017 11:58:58 +0000
Subject: [issue29178] Adding bytes.frombuffer(byteslike) constructor
In-Reply-To: <1483701377.3.0.893514886442.issue29178@psf.upfronthosting.co.za>
Message-ID: <1483790338.04.0.541796902294.issue29178@psf.upfronthosting.co.za>


Xiang Zhang added the comment:

I'm -1 if the intention is about easiness and efficiency. 

I think a new API is usually added due to functional defect not performance defect. We get a way here though the performance seems not ideal, according to INADA's mail. I think we should first check if memoryview gets an optimization chance to fit more in such a case. Creating a memoryview is not cheap enough in such a case.

About easiness to use, when a user considering such low level details, it's reasonable to know memoryview and it needs to be released.

But if this API is added to simplify bytes(), I think it makes sense but it's not only about adding a frombuffer().

----------
nosy: +xiang.zhang

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29178>
_______________________________________

From report at bugs.python.org  Sat Jan  7 07:02:13 2017
From: report at bugs.python.org (John Lehmann)
Date: Sat, 07 Jan 2017 12:02:13 +0000
Subject: [issue29194] importlib reload fails for module supplied as argument
 to command line
Message-ID: <1483790533.68.0.771948929262.issue29194@psf.upfronthosting.co.za>


New submission from John Lehmann:

Modules that have been loaded as an argument to the command line cannot be reloaded using importlib.reload.

For example with the attached file:

  $ python reloader.py
  Traceback (most recent call last):
    File "reloader.py", line 31, in <module>
      reload_module("__main__")
    File "reloader.py", line 28, in reload_module
      importlib.reload(module)
    File "/usr/local/var/pyenv/versions/3.5.2/lib/python3.5/importlib/__init__.py", line 166, in reload
      _bootstrap._exec(spec, module)
    File "<frozen importlib._bootstrap>", line 607, in _exec
  AttributeError: 'NoneType' object has no attribute 'name'

----------
components: Library (Lib)
files: reloader.py
messages: 284910
nosy: j1o1h1n
priority: normal
severity: normal
status: open
title: importlib reload fails for module supplied as argument to command line
versions: Python 3.5
Added file: http://bugs.python.org/file46191/reloader.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29194>
_______________________________________

From report at bugs.python.org  Sat Jan  7 07:04:54 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sat, 07 Jan 2017 12:04:54 +0000
Subject: [issue29195] Get rid of supporting outdated wrong keyword arguments
 in re methods
Message-ID: <1483790693.93.0.397432841389.issue29195@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka:

Many re methods accepted the string parameter as a keyword argument by wrong name. This was fixed in issue20283. Wrong keyword names were still accepted, but a deprecation warning was emitted if use them. Proposed patch finishes the deprecation period (started since 3.4) and removes the support of wrong names.

Python 2.7.7, 3.3.6, 3.4-3.6:

>>> import re
>>> re.compile('.').match(pattern='a')
__main__:1: DeprecationWarning: The 'pattern' keyword parameter name is deprecated.  Use 'string' instead.
<_sre.SRE_Match object; span=(0, 1), match='a'>
>>> re.compile('.').match(string='a')
<_sre.SRE_Match object; span=(0, 1), match='a'>

Python 3.7:

>>> re.compile('.').match(pattern='a')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: Required argument 'string' (pos 1) not found

----------
components: Extension Modules
files: re_wrong_string_argument_name.patch
keywords: patch
messages: 284911
nosy: serhiy.storchaka
priority: normal
severity: normal
stage: patch review
status: open
title: Get rid of supporting outdated wrong keyword arguments in re methods
type: enhancement
versions: Python 3.7
Added file: http://bugs.python.org/file46192/re_wrong_string_argument_name.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29195>
_______________________________________

From report at bugs.python.org  Sat Jan  7 07:11:12 2017
From: report at bugs.python.org (ita1024)
Date: Sat, 07 Jan 2017 12:11:12 +0000
Subject: [issue29174] 'NoneType' object is not callable in subprocess.py
In-Reply-To: <1483688938.47.0.507597836624.issue29174@psf.upfronthosting.co.za>
Message-ID: <1483791072.99.0.331924458052.issue29174@psf.upfronthosting.co.za>


ita1024 added the comment:

> For this specific issue, the ResourceWarning is correct. I don't
understand the use case of explicitly turning this warning off on this
specific example?

No, on this specific example the child processes do terminate properly as the parent process terminate. There is no extra resource consumption so the warning is incorrect and annoying.

> If your output is flooded by ResourceWarning warnings, it's easy to
configure Python to ignore them. Example, simplest option: python3
-Wignore script.py. But you are only going to hide a real issue in
your code. ResourceWarning exists to help you to reduce your resource
consumption.

At the moment the outputs are flooded with "exception ignored" messages that show how well this was thought out. Asking users to add -Wignore options for perfectly legitimate applications also shows how much consideration is being given to user experience, but we should not be surprised because that is how Python3 is being developed?

Please remove this mess until a proper design is made and documented (3.7).

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29174>
_______________________________________

From report at bugs.python.org  Sat Jan  7 07:19:35 2017
From: report at bugs.python.org (R. David Murray)
Date: Sat, 07 Jan 2017 12:19:35 +0000
Subject: [issue29194] importlib reload fails for module supplied as argument
 to command line
In-Reply-To: <1483790533.68.0.771948929262.issue29194@psf.upfronthosting.co.za>
Message-ID: <1483791575.64.0.516434387262.issue29194@psf.upfronthosting.co.za>


R. David Murray added the comment:

I don't think reloading __main__ is something we have any desire to support, but there may be something in this error sequence worth thinking about anyway...the error message isn't very clear as to what went wrong, even with the traceback.

What is your use case for reloading __main__?

----------
nosy: +r.david.murray

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29194>
_______________________________________

From report at bugs.python.org  Sat Jan  7 07:47:08 2017
From: report at bugs.python.org (INADA Naoki)
Date: Sat, 07 Jan 2017 12:47:08 +0000
Subject: [issue29178] Adding bytes.frombuffer(byteslike) constructor
In-Reply-To: <1483701377.3.0.893514886442.issue29178@psf.upfronthosting.co.za>
Message-ID: <1483793228.76.0.423850300358.issue29178@psf.upfronthosting.co.za>


INADA Naoki added the comment:

> I'm -1 if the intention is about easiness and efficiency. 

Do you +1 when adding it to stdlib (say "bufferlib")?

> Creating a memoryview is not cheap enough in such a case.

Actually speaking, it's 5 calls + 2 temporary memoriview.

1. creating memoryview.
2. call __enter__.
3. slicing memoryview
4. create bytes from it
5. call __exit__

It can be bottleneck very easily, when writing protocol parser like HTTP parser.

> About easiness to use, when a user considering such low level details, it's reasonable to know memoryview and it needs to be released.

closing memoryview is not strict requirements in some cases.
When memoryview is created from immutable, relying to GC is safe.
That's why closing memoryview is easily forgotten.

> But if this API is added to simplify bytes(), I think it makes sense but it's not only about adding a frombuffer().

Yes. See msg284813.
There are PEP 467 already to make bytes() simple.

But if PEP 467 is accepted, bytes() constructor is simple enough.
Adding `offset` and `length` keyword-only argument to bytes() make sense.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29178>
_______________________________________

From report at bugs.python.org  Sat Jan  7 09:09:48 2017
From: report at bugs.python.org (Stefan Krah)
Date: Sat, 07 Jan 2017 14:09:48 +0000
Subject: [issue29190] Avoid possible errors in comparing strings
In-Reply-To: <1483773006.32.0.276673146687.issue29190@psf.upfronthosting.co.za>
Message-ID: <1483798188.87.0.0545019895651.issue29190@psf.upfronthosting.co.za>


Stefan Krah added the comment:

Quite honestly I prefer to do nothing. What is the worst that can happen?
A SystemError?

Not-ready unicode strings are an application bug.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29190>
_______________________________________

From report at bugs.python.org  Sat Jan  7 09:14:20 2017
From: report at bugs.python.org (Stefan Krah)
Date: Sat, 07 Jan 2017 14:14:20 +0000
Subject: [issue29190] Avoid possible errors in comparing strings
In-Reply-To: <1483773006.32.0.276673146687.issue29190@psf.upfronthosting.co.za>
Message-ID: <1483798460.35.0.245092925377.issue29190@psf.upfronthosting.co.za>


Stefan Krah added the comment:

Also, if anyone changes the rounding-mode constants it is really their problem.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29190>
_______________________________________

From report at bugs.python.org  Sat Jan  7 09:28:27 2017
From: report at bugs.python.org (Matt)
Date: Sat, 07 Jan 2017 14:28:27 +0000
Subject: [issue14102] argparse: add ability to create a man page
In-Reply-To: <1330031760.56.0.8938390885.issue14102@psf.upfronthosting.co.za>
Message-ID: <1483799307.83.0.301331123286.issue14102@psf.upfronthosting.co.za>


Changes by Matt <matthew at dockstudios.co.uk>:


----------
nosy: +matthewjohn

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue14102>
_______________________________________

From report at bugs.python.org  Sat Jan  7 09:38:30 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sat, 07 Jan 2017 14:38:30 +0000
Subject: [issue29190] Avoid possible errors in comparing strings
In-Reply-To: <1483773006.32.0.276673146687.issue29190@psf.upfronthosting.co.za>
Message-ID: <1483799910.62.0.935039574759.issue29190@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

In worst case ignoring an error can cause a mystical error later during executing an unrelated code or a crash in debug build. In both cases it is hard to find the location of the bug.

----------
nosy: +haypo

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29190>
_______________________________________

From report at bugs.python.org  Sat Jan  7 09:42:29 2017
From: report at bugs.python.org (Stefan Krah)
Date: Sat, 07 Jan 2017 14:42:29 +0000
Subject: [issue29190] Avoid possible errors in comparing strings
In-Reply-To: <1483773006.32.0.276673146687.issue29190@psf.upfronthosting.co.za>
Message-ID: <1483800149.94.0.144320051145.issue29190@psf.upfronthosting.co.za>


Stefan Krah added the comment:

To expand a little, you use the terminology "possible bugs". All I can
see is a double exception if PyUnicode_Compare() returns -1.

I think I did it on purpose because this function is speed sensitive and
no user will change a valid rounding mode *constant* to a string that
is not ready.

So getting a double exception after deliberately sabotaging the module
seems pretty benign to me. :)


Consider the decimal part rejected.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29190>
_______________________________________

From report at bugs.python.org  Sat Jan  7 10:07:16 2017
From: report at bugs.python.org (Xavier de Gaye)
Date: Sat, 07 Jan 2017 15:07:16 +0000
Subject: [issue29177] skip tests of test_logging when bind() raises
 PermissionError (non-root user on Android)
In-Reply-To: <1483701262.25.0.0295401910206.issue29177@psf.upfronthosting.co.za>
Message-ID: <1483801636.25.0.0304876383012.issue29177@psf.upfronthosting.co.za>


Xavier de Gaye added the comment:

issue-29177-02.diff is a modified version of the previous patch that uses support.unlink() instead of os.remove() to handle the failure occuring for a non-existent file
when the test is skipped because of Permission denied. With this patch test_logging is ok, and test_lib2to3 runs fine after test_logging.

issue-29177-03.patch adds the following modifications to the previous patches:
  * Catch only OSError: it is better not to catch programming errors as they may go unnoticed when the test is skipped.
  * Report the OSError in the skip message.
  * Reduce the scope of the try clause to just the code that uses system calls.
  * The cleanup done in setUp() by the previous patch in the except clause does not seem to be needed as the tearDown() method is invoked when the test is skipped.

----------
Added file: http://bugs.python.org/file46193/issue-29177-02.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29177>
_______________________________________

From report at bugs.python.org  Sat Jan  7 10:07:32 2017
From: report at bugs.python.org (Xavier de Gaye)
Date: Sat, 07 Jan 2017 15:07:32 +0000
Subject: [issue29177] skip tests of test_logging when bind() raises
 PermissionError (non-root user on Android)
In-Reply-To: <1483701262.25.0.0295401910206.issue29177@psf.upfronthosting.co.za>
Message-ID: <1483801652.49.0.0950156291281.issue29177@psf.upfronthosting.co.za>


Changes by Xavier de Gaye <xdegaye at gmail.com>:


Added file: http://bugs.python.org/file46194/issue-29177-03.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29177>
_______________________________________

From report at bugs.python.org  Sat Jan  7 10:10:31 2017
From: report at bugs.python.org (Steve Dower)
Date: Sat, 07 Jan 2017 15:10:31 +0000
Subject: [issue29191] liblzma is missing from pcbuild.sln
In-Reply-To: <1483778935.32.0.9795840646.issue29191@psf.upfronthosting.co.za>
Message-ID: <1483801831.13.0.616590688201.issue29191@psf.upfronthosting.co.za>


Steve Dower added the comment:

LGTM. Though as you say, it's probably easiest for one of us to go through each branch and use VS to get it right. I think both of those projects were added from other operating systems, so updating the sln wouldn't be so easy.

I'll get to it at some point unless someone else does. The merging necessary is going to make using patch files difficult.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29191>
_______________________________________

From report at bugs.python.org  Sat Jan  7 10:37:24 2017
From: report at bugs.python.org (Xavier de Gaye)
Date: Sat, 07 Jan 2017 15:37:24 +0000
Subject: [issue29185] test_distutils fails on Android API level 24
In-Reply-To: <1483737507.74.0.0285105710282.issue29185@psf.upfronthosting.co.za>
Message-ID: <1483803444.93.0.933191789152.issue29185@psf.upfronthosting.co.za>


Xavier de Gaye added the comment:

Patch attached.

----------
keywords: +patch
stage: needs patch -> patch review
Added file: http://bugs.python.org/file46195/test_distutils.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29185>
_______________________________________

From report at bugs.python.org  Sat Jan  7 10:49:30 2017
From: report at bugs.python.org (Shane Smith)
Date: Sat, 07 Jan 2017 15:49:30 +0000
Subject: [issue27603] Migrate IDLE context menu items to shell extension
In-Reply-To: <1469332799.74.0.70339682021.issue27603@psf.upfronthosting.co.za>
Message-ID: <1483804170.64.0.643038994226.issue27603@psf.upfronthosting.co.za>


Shane Smith added the comment:

Would it be possible to allow the user to select whether they'd prefer a nested or flat context menu at install?  I believe it went to nested as a result of issue23546.

Unless there are a large number of installations (arbitrarily... perhaps 4 or more), the nested menu is a pain for a user like me who wants to open .py files in IDLE more than any other action.  (If I may quote the fifth line of The Zen of Python... "Flat is better than nested.")  With a little research, it was a fairly easy registry hack to de-nest the context menu.

Still, a lot of people don't like to mess around in the registry, and user choice is usually a good thing.  Thoughts?

----------
nosy: +Shane Smith

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue27603>
_______________________________________

From report at bugs.python.org  Sat Jan  7 11:04:40 2017
From: report at bugs.python.org (Stefan Krah)
Date: Sat, 07 Jan 2017 16:04:40 +0000
Subject: [issue29190] Avoid possible errors in comparing strings
In-Reply-To: <1483773006.32.0.276673146687.issue29190@psf.upfronthosting.co.za>
Message-ID: <1483805080.41.0.446602980606.issue29190@psf.upfronthosting.co.za>


Stefan Krah added the comment:

I'm generally a little concerned about the way "bugs" are presented here recently:

In #28701 you write:

'Correctness. Since no caller checks the error of PyUnicode_CompareWithASCIIString(), it is incorrectly interpreted as "less then".'

This is just not true. When testing for equality "-1" is "not equal"
and an exception would follow anyway.

I'm also not happy with broad coccinelle patches that I see months later.

I'm not sure if you realize that other people's reputation is at
stake here. You label something as a bug (good for you), everyone
who reads your reports and commits think that a bug has been found,
which is not the case.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29190>
_______________________________________

From report at bugs.python.org  Sat Jan  7 11:06:18 2017
From: report at bugs.python.org (Aviv Palivoda)
Date: Sat, 07 Jan 2017 16:06:18 +0000
Subject: [issue29006] 2.7.13 _sqlite more prone to "database table is locked"
In-Reply-To: <1482095939.25.0.891516594682.issue29006@psf.upfronthosting.co.za>
Message-ID: <1483805178.42.0.854286772022.issue29006@psf.upfronthosting.co.za>


Changes by Aviv Palivoda <palaviv at gmail.com>:


----------
nosy: +palaviv

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29006>
_______________________________________

From report at bugs.python.org  Sat Jan  7 11:12:13 2017
From: report at bugs.python.org (Aviv Palivoda)
Date: Sat, 07 Jan 2017 16:12:13 +0000
Subject: [issue29021] Custom functions in sqlite receive None on invalid UTF-8
In-Reply-To: <1482225891.27.0.11020777474.issue29021@psf.upfronthosting.co.za>
Message-ID: <1483805533.46.0.428835235048.issue29021@psf.upfronthosting.co.za>


Aviv Palivoda added the comment:

After looking more into the _pysqlite_set_result function fail in Ingo example I think this is the expected behavior. The PyUnicode_AsUTF8 fails but this is expected as the value is an invalid UTF-8.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29021>
_______________________________________

From report at bugs.python.org  Sat Jan  7 11:52:20 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sat, 07 Jan 2017 16:52:20 +0000
Subject: [issue29196] Remove old-deprecated plistlib features
Message-ID: <1483807940.54.0.471253591392.issue29196@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka:

Seems many features of plistlib was deprecated from the start. The plistlib module was added in 2.6 and its code already contained deprecated features:

* The _InternalDict class is a dict subclass with implemented __getattr__/__setattr__/__delattr__ methods, but all these methods have been deprecated. Since deprecated methods shouldn't be used and they are not implemented in builting dict, _InternalDict can be replaced by builting dict.

* The Dict class has been deprecated. It doesn't used in the module. A user should use builting dict instead.

* The Plist class has been deprecated. It doesn't used in the module. A user should use module level functions instead of Plist methods.

It seems to me that all these classes can be removed. This doesn't break compatibility, since they were deprecated in 2.6. They are even not documented in 2.7.

----------
components: Library (Lib)
files: plistlib_remove_deprecated_classes.patch
keywords: patch
messages: 284925
nosy: ronaldoussoren, serhiy.storchaka
priority: normal
severity: normal
stage: patch review
status: open
title: Remove old-deprecated plistlib features
type: enhancement
versions: Python 3.7
Added file: http://bugs.python.org/file46196/plistlib_remove_deprecated_classes.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29196>
_______________________________________

From report at bugs.python.org  Sat Jan  7 12:17:38 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sat, 07 Jan 2017 17:17:38 +0000
Subject: [issue29197] Remove os.path.splitunc()
Message-ID: <1483809458.59.0.701425626614.issue29197@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka:

Proposed patch removes os.path.splitunc(). It is Windows specific and has been deprecated in 3.1. It is superseded by splitdrive() in 3.x and in 2.7 since 2.7.8 (see issue21672).

----------
components: Library (Lib)
files: remove_os_path_splitunc.patch
keywords: patch
messages: 284926
nosy: serhiy.storchaka
priority: normal
severity: normal
stage: patch review
status: open
title: Remove os.path.splitunc()
type: enhancement
versions: Python 3.7
Added file: http://bugs.python.org/file46197/remove_os_path_splitunc.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29197>
_______________________________________

From report at bugs.python.org  Sat Jan  7 13:22:09 2017
From: report at bugs.python.org (Brett Cannon)
Date: Sat, 07 Jan 2017 18:22:09 +0000
Subject: [issue29194] importlib reload fails for module supplied as argument
 to command line
In-Reply-To: <1483790533.68.0.771948929262.issue29194@psf.upfronthosting.co.za>
Message-ID: <1483813329.64.0.0822628518305.issue29194@psf.upfronthosting.co.za>


Brett Cannon added the comment:

The error actually makes sense when you think about the fact that the module is being passed in by filename. That means Python has no clue what values to put in for the spec because Python literally exec()'s the file contents and stuffs it into a new module name __main__. There's no fancy attempt to construct a proper module from scratch like if you had used -m which does fill in the spec. And since importlib.reload() relies entirely on the module's spec to handle reloads it simply can't handle the `__spec__ == None` case that this is triggering.

IOW use `python3 -m reloader` and it will probably work. Closing this as "not a bug".

----------
nosy: +brett.cannon
resolution:  -> not a bug
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29194>
_______________________________________

From report at bugs.python.org  Sat Jan  7 13:33:47 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sat, 07 Jan 2017 18:33:47 +0000
Subject: [issue29050] xml.etree.ElementTree in Python 3.6 is incompatible with
 defusedxml
In-Reply-To: <1482428926.43.0.656400422228.issue29050@psf.upfronthosting.co.za>
Message-ID: <1483814027.13.0.575452001871.issue29050@psf.upfronthosting.co.za>


Changes by Serhiy Storchaka <storchaka+cpython at gmail.com>:


----------
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29050>
_______________________________________

From report at bugs.python.org  Sat Jan  7 13:39:54 2017
From: report at bugs.python.org (Simon Percivall)
Date: Sat, 07 Jan 2017 18:39:54 +0000
Subject: [issue29167] Race condition in enum.py:_decompose()
In-Reply-To: <1483613666.47.0.301459916652.issue29167@psf.upfronthosting.co.za>
Message-ID: <1483814394.92.0.80531432637.issue29167@psf.upfronthosting.co.za>


Simon Percivall added the comment:

Run this a couple of times (it fails for me the first time, but it's a race, so YMMV):

```
import enum
from concurrent.futures import ThreadPoolExecutor


class MyEnum(enum.IntFlag):
    one = 1


with ThreadPoolExecutor() as executor:
    print(list(executor.map(MyEnum.one.__or__, range(1000))))
```

An easy fix would be:

```
diff --git a/Lib/enum.py b/Lib/enum.py
index e79b0382eb..eca56ec3a7 100644
--- a/Lib/enum.py
+++ b/Lib/enum.py
@@ -846,7 +846,7 @@ def _decompose(flag, value):
         # check for named flags and powers-of-two flags
         flags_to_check = [
                 (m, v)
-                for v, m in flag._value2member_map_.items()
+                for v, m in list(flag._value2member_map_.items())
                 if m.name is not None or _power_of_two(v)
                 ]
     members = []
```

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29167>
_______________________________________

From report at bugs.python.org  Sat Jan  7 14:43:09 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Sat, 07 Jan 2017 19:43:09 +0000
Subject: [issue29188] Backport random.c from Python 3.5 to Python 2.7
In-Reply-To: <1483746834.46.0.642378238515.issue29188@psf.upfronthosting.co.za>
Message-ID: <1483818189.55.0.465235431837.issue29188@psf.upfronthosting.co.za>


Raymond Hettinger added the comment:

> I would prefer to use the "same code" (or almost) on all maintained
> versions of Python: 2.7, 3.5, 3.6 and 3.7. It should ease the
> maintenance for bugfixes and enhancements.

This is VERY far from our historical policy for backports.  Python 2.7 is supposed to be getting more stable over time (that is one of its chief virtues).  We don't want to risk the kind of mini-catastrophe that got published in 3.6 (issue29085).  

If you want to push for this, there needs to be a thorough discussion on python-dev (there are tons of possible backports that could be made if the rationale was "I would prefer to use the same code on all maintained versions").

----------
nosy: +benjamin.peterson

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29188>
_______________________________________

From report at bugs.python.org  Sat Jan  7 14:54:02 2017
From: report at bugs.python.org (=?utf-8?q?Alex_Gr=C3=B6nholm?=)
Date: Sat, 07 Jan 2017 19:54:02 +0000
Subject: [issue23242] asyncio: Process must close the transport when the
 process exit
In-Reply-To: <1421277078.82.0.431621848437.issue23242@psf.upfronthosting.co.za>
Message-ID: <1483818842.72.0.612010097325.issue23242@psf.upfronthosting.co.za>


Changes by Alex Gr?nholm <alex.gronholm+python at nextday.fi>:


----------
nosy: +alex.gronholm
versions: +Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23242>
_______________________________________

From report at bugs.python.org  Sat Jan  7 14:58:07 2017
From: report at bugs.python.org (=?utf-8?q?Alex_Gr=C3=B6nholm?=)
Date: Sat, 07 Jan 2017 19:58:07 +0000
Subject: [issue23242] asyncio: Process must close the transport when the
 process exit
In-Reply-To: <1421277078.82.0.431621848437.issue23242@psf.upfronthosting.co.za>
Message-ID: <1483819087.7.0.339645172526.issue23242@psf.upfronthosting.co.za>


Alex Gr?nholm added the comment:

Are you sure this has been fixed? The attached script reproduces the bug 100% reliably on my laptop.

----------
Added file: http://bugs.python.org/file46198/read_subprocess.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23242>
_______________________________________

From report at bugs.python.org  Sat Jan  7 15:03:24 2017
From: report at bugs.python.org (Christian Heimes)
Date: Sat, 07 Jan 2017 20:03:24 +0000
Subject: [issue29188] Backport random.c from Python 3.5 to Python 2.7
In-Reply-To: <1483746834.46.0.642378238515.issue29188@psf.upfronthosting.co.za>
Message-ID: <1483819404.46.0.966038900461.issue29188@psf.upfronthosting.co.za>


Changes by Christian Heimes <lists at cheimes.de>:


----------
nosy: +christian.heimes

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29188>
_______________________________________

From report at bugs.python.org  Sat Jan  7 15:41:08 2017
From: report at bugs.python.org (Xavier de Gaye)
Date: Sat, 07 Jan 2017 20:41:08 +0000
Subject: [issue29181] skip tests that raise PermissionError in test_tarfile
 (non-root user on Android)
In-Reply-To: <1483720295.96.0.810571843517.issue29181@psf.upfronthosting.co.za>
Message-ID: <1483821668.85.0.509901127002.issue29181@psf.upfronthosting.co.za>


Xavier de Gaye added the comment:

Patch attached.

----------
keywords: +patch
stage: needs patch -> patch review
Added file: http://bugs.python.org/file46199/test_tarfile.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29181>
_______________________________________

From report at bugs.python.org  Sat Jan  7 15:53:11 2017
From: report at bugs.python.org (Xavier de Gaye)
Date: Sat, 07 Jan 2017 20:53:11 +0000
Subject: [issue29180] skip tests that raise PermissionError in test_os
 (non-root user on Android)
In-Reply-To: <1483715692.75.0.979129057464.issue29180@psf.upfronthosting.co.za>
Message-ID: <1483822391.91.0.387144349956.issue29180@psf.upfronthosting.co.za>


Xavier de Gaye added the comment:

Patch attached.

----------
keywords: +patch
stage: needs patch -> patch review
Added file: http://bugs.python.org/file46200/test_os.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29180>
_______________________________________

From report at bugs.python.org  Sat Jan  7 15:57:52 2017
From: report at bugs.python.org (Martin Panter)
Date: Sat, 07 Jan 2017 20:57:52 +0000
Subject: [issue27068] Add a detach() method to subprocess.Popen
In-Reply-To: <1463739584.18.0.0161123331788.issue27068@psf.upfronthosting.co.za>
Message-ID: <1483822672.21.0.341785676986.issue27068@psf.upfronthosting.co.za>


Martin Panter added the comment:

The user can access pipes and close them directly, or keep their own reference. I don?t think detach() should touch pipes, and __exit__() should probably continue to close them. Maybe call the method detach_pid() if that makes it clearer that pipes are unaffected.

For the Windows process handle, I suggest detach() should close it. I believe this would make Windows work like Unix when you set SIGCHLD to automatically reap children.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue27068>
_______________________________________

From report at bugs.python.org  Sat Jan  7 15:58:58 2017
From: report at bugs.python.org (Martin Panter)
Date: Sat, 07 Jan 2017 20:58:58 +0000
Subject: [issue29174] 'NoneType' object is not callable in subprocess.py
In-Reply-To: <1483688938.47.0.507597836624.issue29174@psf.upfronthosting.co.za>
Message-ID: <1483822738.15.0.655156045001.issue29174@psf.upfronthosting.co.za>


Martin Panter added the comment:

The code in test.py is not realistic. It spawns children only to terminate them straight away, and you could easily reap each child after calling terminate(). You might have more influence with a realistic use case.

Victor has committed a fix for the ?exception ignored? problem, so assuming it works, in the next release of Python it should be gone. You should still see a ResourceWarning if warnings are enabled, but I don?t think -Wignore would be necessary to suppress them; such warnings are disabled by default.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29174>
_______________________________________

From report at bugs.python.org  Sat Jan  7 16:04:25 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sat, 07 Jan 2017 21:04:25 +0000
Subject: [issue29181] skip tests that raise PermissionError in test_tarfile
 (non-root user on Android)
In-Reply-To: <1483720295.96.0.810571843517.issue29181@psf.upfronthosting.co.za>
Message-ID: <1483823065.32.0.698946212139.issue29181@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

Why os.link() is failed? If hard links are not supported on Android, shouldn't os.link be not implemented? tarfile try to make a copy of the referenced file instead of a link if a link can't be created.

----------
nosy: +serhiy.storchaka

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29181>
_______________________________________

From report at bugs.python.org  Sat Jan  7 16:19:58 2017
From: report at bugs.python.org (Xavier de Gaye)
Date: Sat, 07 Jan 2017 21:19:58 +0000
Subject: [issue29184] skip tests of test_socketserver when bind() raises
 PermissionError (non-root user on Android)
In-Reply-To: <1483732137.12.0.493215743488.issue29184@psf.upfronthosting.co.za>
Message-ID: <1483823998.47.0.248676952541.issue29184@psf.upfronthosting.co.za>


Xavier de Gaye added the comment:

Patch attached.

----------
keywords: +patch
stage: needs patch -> patch review
Added file: http://bugs.python.org/file46201/test_socketserver.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29184>
_______________________________________

From report at bugs.python.org  Sat Jan  7 16:37:41 2017
From: report at bugs.python.org (Jelle Zijlstra)
Date: Sat, 07 Jan 2017 21:37:41 +0000
Subject: [issue29198] AsyncGenerator is missing from typing
Message-ID: <1483825061.03.0.644790904886.issue29198@psf.upfronthosting.co.za>


New submission from Jelle Zijlstra:

PEP 525 async generators weren't added to typing.py, probably by oversight.

I sent pull requests to typing and typeshed on GitHub to add an AsyncGenerator class and stub:
- https://github.com/python/typing/pull/346
- https://github.com/python/typeshed/pull/815

However, typing docs are not in GitHub, so I'm attaching a doc patch here.

----------
components: Library (Lib)
files: asyncgenerator.patch
keywords: patch
messages: 284937
nosy: Jelle Zijlstra, gvanrossum, yselivanov
priority: normal
severity: normal
status: open
title: AsyncGenerator is missing from typing
versions: Python 3.6, Python 3.7
Added file: http://bugs.python.org/file46202/asyncgenerator.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29198>
_______________________________________

From report at bugs.python.org  Sat Jan  7 16:41:22 2017
From: report at bugs.python.org (Xavier de Gaye)
Date: Sat, 07 Jan 2017 21:41:22 +0000
Subject: [issue29181] skip tests that raise PermissionError in test_tarfile
 (non-root user on Android)
In-Reply-To: <1483720295.96.0.810571843517.issue29181@psf.upfronthosting.co.za>
Message-ID: <1483825282.92.0.0513800761995.issue29181@psf.upfronthosting.co.za>


Xavier de Gaye added the comment:

> Why os.link() is failed? If hard links are not supported on Android, shouldn't os.link be not implemented?

Android has a restrictive security model based on SELinux [1].

With the Android adb shell on the emulator at API level 24:

$ >foo
$ ln foo bar
ln: cannot create hard link from 'foo' to 'bar': Permission denied
$ su
# ln foo bar
# ls -li foo bar
15688 -rw-rw-rw- 2 shell shell 0 2017-01-07 22:29 bar
15688 -rw-rw-rw- 2 shell shell 0 2017-01-07 22:29 foo

Here is the Android commit message that does not grant hard link capabilities by default:
https://android.googlesource.com/platform/external/sepolicy/+/85ce2c7

[1] https://source.android.com/security/selinux/

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29181>
_______________________________________

From report at bugs.python.org  Sat Jan  7 16:45:23 2017
From: report at bugs.python.org (Ivan Levkivskyi)
Date: Sat, 07 Jan 2017 21:45:23 +0000
Subject: [issue29198] AsyncGenerator is missing from typing
In-Reply-To: <1483825061.03.0.644790904886.issue29198@psf.upfronthosting.co.za>
Message-ID: <1483825523.9.0.665079271209.issue29198@psf.upfronthosting.co.za>


Changes by Ivan Levkivskyi <levkivskyi at gmail.com>:


----------
nosy: +levkivskyi

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29198>
_______________________________________

From report at bugs.python.org  Sat Jan  7 16:56:28 2017
From: report at bugs.python.org (Ethan Furman)
Date: Sat, 07 Jan 2017 21:56:28 +0000
Subject: [issue29167] Race condition in enum.py:_decompose()
In-Reply-To: <1483613666.47.0.301459916652.issue29167@psf.upfronthosting.co.za>
Message-ID: <1483826188.81.0.00246923768632.issue29167@psf.upfronthosting.co.za>


Ethan Furman added the comment:

Thanks.  I'll go through and audit all my dictionary iterations.

----------
nosy: +barry, eli.bendersky

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29167>
_______________________________________

From report at bugs.python.org  Sat Jan  7 17:00:24 2017
From: report at bugs.python.org (Guido van Rossum)
Date: Sat, 07 Jan 2017 22:00:24 +0000
Subject: [issue29002] typing.AnyStr doc is unclear about python2 unicode
 support
In-Reply-To: <1482018824.86.0.562219638154.issue29002@psf.upfronthosting.co.za>
Message-ID: <1483826424.12.0.441569386406.issue29002@psf.upfronthosting.co.za>


Guido van Rossum added the comment:

I agree that it would just be confusing if the Python 3 docs were to explain Python 2 specific things.  Maybe this should be explained in the mypy docs (or maybe it's really a mypy bug).  Can you open an issue there?

----------
nosy: +gvanrossum
resolution:  -> wont fix
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29002>
_______________________________________

From report at bugs.python.org  Sat Jan  7 17:00:41 2017
From: report at bugs.python.org (Vinay Sajip)
Date: Sat, 07 Jan 2017 22:00:41 +0000
Subject: [issue29177] skip tests of test_logging when bind() raises
 PermissionError (non-root user on Android)
In-Reply-To: <1483701262.25.0.0295401910206.issue29177@psf.upfronthosting.co.za>
Message-ID: <1483826441.32.0.105273482029.issue29177@psf.upfronthosting.co.za>


Vinay Sajip added the comment:

Thanks for the improvements to my patch. The 03-version looks good to me.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29177>
_______________________________________

From report at bugs.python.org  Sat Jan  7 17:17:02 2017
From: report at bugs.python.org (Martin Panter)
Date: Sat, 07 Jan 2017 22:17:02 +0000
Subject: [issue29070] Integration tests for pty.spawn on Linux and all other
 platforms
In-Reply-To: <1482684225.17.0.467768789806.issue29070@psf.upfronthosting.co.za>
Message-ID: <1483827422.69.0.853457996013.issue29070@psf.upfronthosting.co.za>


Martin Panter added the comment:

I would prefer to commit Chris?s fix for BSDs (Issue 26228) with a regression test. I can explain in the commit message who contributed to which part, or do two separate commits if you prefer. But the point is to add the test with the fix. I?m not going to commit the test on its own, because we know it will fail.

Even in Lib/test/test_pty.py, it looks like the two patches will collide. I was hoping you could combine the patches, or supply a set of patches that are tested and compatible. I don?t have OS X or BSD so I have to rely on you and the buildbots to confirm that a patch doesn?t break any tests.

I haven?t looked too closely regarding the slow tests, but another option is use the ?cpu? resource to mark the test as being slow.

I realized that PtyWhiteBoxIntegrationTermiosTest is a reasonable test for the termios module, so it is beneficial. Though it may have been simpler to write it using pty.openpty(), avoiding an extra thread that just copies data between other threads.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29070>
_______________________________________

From report at bugs.python.org  Sat Jan  7 17:20:07 2017
From: report at bugs.python.org (Sworddragon)
Date: Sat, 07 Jan 2017 22:20:07 +0000
Subject: [issue28180] sys.getfilesystemencoding() should default to utf-8
In-Reply-To: <1474024622.65.0.838685461995.issue28180@psf.upfronthosting.co.za>
Message-ID: <1483827607.11.0.00354428831813.issue28180@psf.upfronthosting.co.za>


Sworddragon added the comment:

> $ cat badfilename.py 
> badfn = "?????".encode('euc-jp').decode('utf-8', 'surrogateescape')
> print("bad filename:", badfn)
>
> $ PYTHONIOENCODING=utf-8:backslashreplace python3 badfilename.py 
> bad filename: \udca4\udcb3\udca4\udcf3\udca4?\udcc1\udca4\udccf
>
> $ PYTHONIOENCODING=utf-8:surrogateescape python3 badfilename.py 
> bad filename: ?????????

The first example is still readable (but effectively for an user not so much) while the second example appears to be not readable anymore at all. But the second example is actually technically still readable and there is no data loss, isn't it? As in this case it would probably not speak against surrogateescape for sys.stderr in UTF-8 non-strict mode. Otherwise backslashescape might be indeed the better choice.


I have thought about this a bit more and in case we go PEP 538 with keeping strict errors more or less the old way there might be another solution that could improve the overall issue: print() could get an option to allow changing the error handler on demand (with 'strict' still being the default).

Most things that I do output with print() are deterministic or optional and not important application data. Being able to print this information without caring for de-/encoding errors would mitigate this issue. In case application data is being printed where data loss is not desired exceptions can still be thrown.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28180>
_______________________________________

From report at bugs.python.org  Sat Jan  7 18:30:54 2017
From: report at bugs.python.org (Stefan Krah)
Date: Sat, 07 Jan 2017 23:30:54 +0000
Subject: [issue28701] Replace PyUnicode_CompareWithASCIIString with
 _PyUnicode_EqualToASCIIString
In-Reply-To: <1479239405.55.0.31899621759.issue28701@psf.upfronthosting.co.za>
Message-ID: <1483831854.71.0.636285305327.issue28701@psf.upfronthosting.co.za>


Stefan Krah added the comment:

For the record: This is all that happened in decimal if a) you
pass a legacy string and b) force _PyUnicode_Ready() to throw
a MemoryError:

>>> from decimal import *
>>> import _testcapi
>>> context = Context()
>>> traps = _testcapi.unicode_legacy_string('traps')
>>> getattr(context, traps)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError



Both a) and b) are not trivial to accomplish at all and the result
is completely benign.

----------
nosy: +skrah

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28701>
_______________________________________

From report at bugs.python.org  Sat Jan  7 19:04:20 2017
From: report at bugs.python.org (Martin Panter)
Date: Sun, 08 Jan 2017 00:04:20 +0000
Subject: [issue21242] Generalize configure check for working Python executable
In-Reply-To: <1397584205.03.0.366670026915.issue21242@psf.upfronthosting.co.za>
Message-ID: <1483833860.97.0.615417966793.issue21242@psf.upfronthosting.co.za>


Martin Panter added the comment:

It is still not clear what change you were proposing. Perhaps factor out the common code for ADSLGEN and OPCODEHGEN? If so, that has been done as part of Issue 26662 in 3.5+.

----------
nosy: +martin.panter
superseder:  -> configure/Makefile doesn't check if "python" command works, needed to build Objects/typeslots.inc

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue21242>
_______________________________________

From report at bugs.python.org  Sat Jan  7 19:42:01 2017
From: report at bugs.python.org (Berker Peksag)
Date: Sun, 08 Jan 2017 00:42:01 +0000
Subject: [issue29180] skip tests that raise PermissionError in test_os
 (non-root user on Android)
In-Reply-To: <1483715692.75.0.979129057464.issue29180@psf.upfronthosting.co.za>
Message-ID: <1483836121.98.0.613025334438.issue29180@psf.upfronthosting.co.za>


Berker Peksag added the comment:

Are you planning to use test.support.os_link() to fix other tests in near future? If test_os is the only user of the helper, I'd say let's keep it in Lib/test/test_os.py for now. test.support is already big enough and it would be better not to put every snippet in it :)

If there are other tests that can benefit from using it, the patch looks good to me.

----------
nosy: +berker.peksag

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29180>
_______________________________________

From report at bugs.python.org  Sat Jan  7 20:14:56 2017
From: report at bugs.python.org (ppperry)
Date: Sun, 08 Jan 2017 01:14:56 +0000
Subject: [issue29199] test_regrtest fails if PCBuild directory doesn't exist
Message-ID: <1483838095.77.0.195289202994.issue29199@psf.upfronthosting.co.za>


New submission from ppperry:

======================================================================
ERROR: test_pcbuild_rt (test.test_regrtest.ProgramsTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Program Files (x86)\Python36-32\lib\test\test_regrtest.py", line 564, in test_pcbuild_rt
    self.run_batch(script, *rt_args, *self.regrtest_args, *self.tests)
  File "C:\Program Files (x86)\Python36-32\lib\test\test_regrtest.py", line 539, in run_batch
    proc = self.run_command(args)
  File "C:\Program Files (x86)\Python36-32\lib\test\test_regrtest.py", line 436, in run_command
    **kw)
  File "C:\Program Files (x86)\Python36-32\lib\subprocess.py", line 403, in run
    with Popen(*popenargs, **kwargs) as process:
  File "C:\Program Files (x86)\Python36-32\lib\subprocess.py", line 707, in __init__
    restore_signals, start_new_session)
  File "C:\Program Files (x86)\Python36-32\lib\subprocess.py", line 990, in _execute_child
    startupinfo)
FileNotFoundError: [WinError 2] The system cannot find the file specified

----------
messages: 284947
nosy: ppperry
priority: normal
severity: normal
status: open
title: test_regrtest fails if PCBuild directory doesn't exist
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29199>
_______________________________________

From report at bugs.python.org  Sat Jan  7 20:15:11 2017
From: report at bugs.python.org (ppperry)
Date: Sun, 08 Jan 2017 01:15:11 +0000
Subject: [issue29199] test_regrtest fails if PCBuild directory doesn't exist
In-Reply-To: <1483838095.77.0.195289202994.issue29199@psf.upfronthosting.co.za>
Message-ID: <1483838111.49.0.206796774977.issue29199@psf.upfronthosting.co.za>


Changes by ppperry <mapreader at olum.org>:


----------
components: +Tests

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29199>
_______________________________________

From report at bugs.python.org  Sat Jan  7 20:34:39 2017
From: report at bugs.python.org (Jelle Zijlstra)
Date: Sun, 08 Jan 2017 01:34:39 +0000
Subject: [issue29198] AsyncGenerator is missing from typing
In-Reply-To: <1483825061.03.0.644790904886.issue29198@psf.upfronthosting.co.za>
Message-ID: <1483839279.61.0.846714930021.issue29198@psf.upfronthosting.co.za>


Changes by Jelle Zijlstra <jelle.zijlstra at gmail.com>:


Added file: http://bugs.python.org/file46203/asyncgenerator2.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29198>
_______________________________________

From report at bugs.python.org  Sat Jan  7 20:40:22 2017
From: report at bugs.python.org (Martin Panter)
Date: Sun, 08 Jan 2017 01:40:22 +0000
Subject: [issue29082] In 2.7.13,
 _ctypes.LoadLibrary no longer accepts Unicode objects
In-Reply-To: <1482833341.48.0.70408057792.issue29082@psf.upfronthosting.co.za>
Message-ID: <1483839622.72.0.149604202569.issue29082@psf.upfronthosting.co.za>


Martin Panter added the comment:

Other tests are skipped if libc_name is None, so your assertion is inconsistent.

FTR there are reports open about problems with bootstrap files like asdl_c.py, e.g. Issue 28143 proposing to port that file to Python 3, and Issue 23404 about the future of ?make touch?.

----------
stage: needs patch -> patch review

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29082>
_______________________________________

From report at bugs.python.org  Sat Jan  7 20:44:53 2017
From: report at bugs.python.org (ppperry)
Date: Sun, 08 Jan 2017 01:44:53 +0000
Subject: [issue29199] test_regrtest fails if PCBuild directory doesn't exist
In-Reply-To: <1483838095.77.0.195289202994.issue29199@psf.upfronthosting.co.za>
Message-ID: <1483839893.76.0.087308259622.issue29199@psf.upfronthosting.co.za>


Changes by ppperry <mapreader at olum.org>:


----------
type:  -> behavior

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29199>
_______________________________________

From report at bugs.python.org  Sat Jan  7 21:02:12 2017
From: report at bugs.python.org (Jiajun Huang)
Date: Sun, 08 Jan 2017 02:02:12 +0000
Subject: [issue29200] is it a bug in `functools._HashedSeq`
Message-ID: <1483840932.91.0.717045884042.issue29200@psf.upfronthosting.co.za>


New submission from Jiajun Huang:

the class definition:

class _HashedSeq(list):
    """ This class guarantees that hash() will be called no more than once
        per element.  This is important because the lru_cache() will hash
        the key multiple times on a cache miss.

    """

    __slots__ = 'hashvalue'

    def __init__(self, tup, hash=hash):
        self[:] = tup
        self.hashvalue = hash(tup)

    def __hash__(self):
        return self.hashvalue

and I've test for it:

In [1]: from functools import _HashedSeq

In [2]: from unittest.mock import Mock

In [3]: test_tup = 1, 2, 3, "hello", "world"

In [4]: hash_func = Mock()

In [5]: _HashedSeq(test_tup, hash=hash_func)
Out[5]: [1, 2, 3, 'hello', 'world']

In [6]: _HashedSeq(test_tup, hash=hash_func)
Out[6]: [1, 2, 3, 'hello', 'world']

In [7]: _HashedSeq(test_tup, hash=hash_func)
Out[7]: [1, 2, 3, 'hello', 'world']

In [8]: hash_func.call_count
Out[8]: 3

the hash function had been called 3 times rather than 1.

----------
components: Library (Lib)
messages: 284949
nosy: Jiajun Huang
priority: normal
severity: normal
status: open
title: is it a bug in `functools._HashedSeq`
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29200>
_______________________________________

From report at bugs.python.org  Sat Jan  7 21:05:24 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Sun, 08 Jan 2017 02:05:24 +0000
Subject: [issue29200] is it a bug in `functools._HashedSeq`
In-Reply-To: <1483840932.91.0.717045884042.issue29200@psf.upfronthosting.co.za>
Message-ID: <1483841124.22.0.77653063519.issue29200@psf.upfronthosting.co.za>


Changes by Raymond Hettinger <raymond.hettinger at gmail.com>:


----------
assignee:  -> rhettinger
nosy: +rhettinger

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29200>
_______________________________________

From report at bugs.python.org  Sat Jan  7 21:07:23 2017
From: report at bugs.python.org (INADA Naoki)
Date: Sun, 08 Jan 2017 02:07:23 +0000
Subject: [issue15216] Support setting the encoding on a text stream after
 creation
In-Reply-To: <1340880558.16.0.0136840668667.issue15216@psf.upfronthosting.co.za>
Message-ID: <1483841243.07.0.0601646224526.issue15216@psf.upfronthosting.co.za>


INADA Naoki added the comment:

Updated the patch for default branch.

----------
versions: +Python 3.7 -Python 3.6
Added file: http://bugs.python.org/file46204/set_encoding-7.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue15216>
_______________________________________

From report at bugs.python.org  Sat Jan  7 21:16:07 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Sun, 08 Jan 2017 02:16:07 +0000
Subject: [issue29200] is it a bug in `functools._HashedSeq`
In-Reply-To: <1483840932.91.0.717045884042.issue29200@psf.upfronthosting.co.za>
Message-ID: <1483841767.42.0.101211404539.issue29200@psf.upfronthosting.co.za>


Raymond Hettinger added the comment:

Sorry, but I think you've missed the point of _HashedSeq.  The hash() is called no more than once per instance, not once per instance creation.

    >>> from functools import _HashedSeq
    >>> from unittest.mock import Mock
    >>> test_tup = 1, 2, 3, "hello", "world"
    >>> hash_func = Mock(return_value=999)
    >>> hs = _HashedSeq(test_tup, hash=hash_func)
    >>> hash(hs)
    999
    >>> hash(hs)
    999
    >>> hash(hs)
    999
    >>> hash_func.call_count
    1

----------
resolution:  -> not a bug
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29200>
_______________________________________

From report at bugs.python.org  Sat Jan  7 21:22:40 2017
From: report at bugs.python.org (Nick Coghlan)
Date: Sun, 08 Jan 2017 02:22:40 +0000
Subject: [issue28180] sys.getfilesystemencoding() should default to utf-8
In-Reply-To: <1474024622.65.0.838685461995.issue28180@psf.upfronthosting.co.za>
Message-ID: <1483842160.91.0.371058285475.issue28180@psf.upfronthosting.co.za>


Nick Coghlan added the comment:

Uploaded one last version of the patch implementing the previous PEP 538 design. This refactors the test cases so they systematically cover 4 cases that we expect to be reported as "the C locale":

- LC_ALL, LC_CTYPE, and LANG all empty
- one of them set to "C", others empty
- one of them set to "POSIX", others empty
- one of them set to an unknown locale, others empty

The next version of the patch will update it to match the latest draft of the PEP (PYTHONCOERCECLOCALE, different message wording, etc)

----------
Added file: http://bugs.python.org/file46205/pep538_coerce_legacy_c_locale_v3.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28180>
_______________________________________

From report at bugs.python.org  Sat Jan  7 21:35:10 2017
From: report at bugs.python.org (Roundup Robot)
Date: Sun, 08 Jan 2017 02:35:10 +0000
Subject: [issue28815] test_socket fails if /proc/modules is existent but not
 readable
In-Reply-To: <1480263378.99.0.648069549511.issue28815@psf.upfronthosting.co.za>
Message-ID: <20170108023506.53365.70290.DA1C904A@psf.io>


Roundup Robot added the comment:

New changeset 5a417e160b34 by Martin Panter in branch '3.5':
Issue #28815: Use new exception subclasses
https://hg.python.org/cpython/rev/5a417e160b34

New changeset 401e70317976 by Martin Panter in branch '3.6':
Issue #28815: Merge test tweak from 3.5
https://hg.python.org/cpython/rev/401e70317976

New changeset 82fb37281954 by Martin Panter in branch 'default':
Issue #28815: Merge test tweak from 3.6
https://hg.python.org/cpython/rev/82fb37281954

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28815>
_______________________________________

From report at bugs.python.org  Sat Jan  7 22:22:43 2017
From: report at bugs.python.org (Eryk Sun)
Date: Sun, 08 Jan 2017 03:22:43 +0000
Subject: [issue29178] Adding bytes.frombuffer(byteslike) constructor
In-Reply-To: <1483701377.3.0.893514886442.issue29178@psf.upfronthosting.co.za>
Message-ID: <1483845763.9.0.721185667787.issue29178@psf.upfronthosting.co.za>


Eryk Sun added the comment:

Isn't the proposed workaround also relying on CPython reference counting to immediately deallocate the sliced view? It fails if I keep a reference to the sliced view:

    byteslike = bytearray(b'abc')

    with memoryview(byteslike) as m1:
        m2 = m1[1:]
        bs = bytes(m2)
     
    >>> byteslike += b'd'
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    BufferError: Existing exports of data: object cannot be re-sized

It seems to me that it should be written as follows:

    with memoryview(byteslike) as m1:
        with m1[1:] as m2:
            bs = bytes(m2)

    >>> byteslike += b'd'
    >>> byteslike
    bytearray(b'abcd')

The memoryview constructor could take start, stop, and step keyword-only arguments to avoid having to immediately slice a new view.

----------
nosy: +eryksun

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29178>
_______________________________________

From report at bugs.python.org  Sat Jan  7 22:23:28 2017
From: report at bugs.python.org (Nick Coghlan)
Date: Sun, 08 Jan 2017 03:23:28 +0000
Subject: [issue15216] Support setting the encoding on a text stream after
 creation
In-Reply-To: <1340880558.16.0.0136840668667.issue15216@psf.upfronthosting.co.za>
Message-ID: <1483845808.33.0.1652145511.issue15216@psf.upfronthosting.co.za>


Nick Coghlan added the comment:

Reviewing Inada-san's latest version of the patch, we seem to be in a somewhat hybrid state where:

1. The restriction to only being used with seekable() streams if there is currently unread data in the read buffer is in place

2. We don't actually call seek() anywhere to set the stream back to the beginning of the file. Instead, we try to shuffle data out of the old decoder and into the new one.

I'm starting to wonder if the best option here might be to attempt to make the API work for arbitrary codecs and non-seekable streams, and then simply accept that it may take a few maintenance releases before that's actually true. If we decide to go down that path, then I'd suggest the follow stress test:

- make a longish test string out of repeated copies of "??????"
- pick a few pairs of multibyte non-universal/universal encodings for use with surrogateescape and strict as their respective error handlers (e.g. ascii/utf8, ascii/utf16le, ascii/utf32, ascii/shift_jis, ascii/iso2022_jp, ascii/gb18030, gbk/gb18030)
- for each pair, make the test data by encoding from str to bytes with the relevant universal encoding
- switch the encoding multiple times on the same stream at different points

Optionally:

- extract "codecs._switch_decoder" and "codecs._switch_encoder" helper functions to make this all a bit easier to test and debug (with a Python version in the codecs module and the C version accessible via the _codecs modules)

That way, confidence in the reliability of the feature (including across Python implementations) can be based on the strength of the test cases covering it.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue15216>
_______________________________________

From report at bugs.python.org  Sat Jan  7 22:41:20 2017
From: report at bugs.python.org (INADA Naoki)
Date: Sun, 08 Jan 2017 03:41:20 +0000
Subject: [issue29178] Adding bytes.frombuffer(byteslike) constructor
In-Reply-To: <1483701377.3.0.893514886442.issue29178@psf.upfronthosting.co.za>
Message-ID: <1483846880.57.0.364032608363.issue29178@psf.upfronthosting.co.za>


INADA Naoki added the comment:

You're right!  How difficult working with memoryview!

> The memoryview constructor could take start, stop, and step keyword-only arguments to avoid having to immediately slice a new view.

Maybe, memoryview.to_bytes() is better place to add such options.

memoryview(x) can accept multi dimensional arrays, and itemsize can be >=1.
It's too complex when working with byte sequence.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29178>
_______________________________________

From report at bugs.python.org  Sat Jan  7 23:24:20 2017
From: report at bugs.python.org (Edward Laier)
Date: Sun, 08 Jan 2017 04:24:20 +0000
Subject: [issue29201] Python 3.6 not working within VS 2015
Message-ID: <1483849460.53.0.951526575508.issue29201@psf.upfronthosting.co.za>


New submission from Edward Laier:

I installed Python 3.6 on my Microsoft server 2016 , running VS 2015 Community. I tried the path to Python Environment, then auto-detect, VS crashes and then when it restarts it will have the "+ Custom" grayed out. Where I then have to remove the key to ungray it as mentioned here  [link](https://stackoverflow.com/questions/40430831/vs-2015-python-environments-greyed-out) 
Trying to put in all the variables for the environment doesn't work as 3.6 isn't even an option in the version drop-box for VS. The only option I see for now until this is fixed is removing 3.6 and installing 3.5

----------
components: Windows
messages: 284957
nosy: Edward Laier, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Python 3.6 not working within VS 2015
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29201>
_______________________________________

From report at bugs.python.org  Sat Jan  7 23:35:21 2017
From: report at bugs.python.org (Nick Coghlan)
Date: Sun, 08 Jan 2017 04:35:21 +0000
Subject: [issue29178] Adding bytes.frombuffer(byteslike) constructor
In-Reply-To: <1483701377.3.0.893514886442.issue29178@psf.upfronthosting.co.za>
Message-ID: <1483850121.73.0.247432416825.issue29178@psf.upfronthosting.co.za>


Nick Coghlan added the comment:

The complexity you're hitting here is the main reason I'm a fan of creating a dedicated library for dealing with these problems, rather than trying to handle them directly on the builtins.

Given a bufferlib module, for example, you could have a higher level API like:

    def snapshot(source, *, result_type=bytes):
        ...

That handled any source object that supported the buffer protocol, and any target type that accepted a memoryview instance as input to the constructor.

    # Default bytes snapshot
    data = snapshot(original)

    # Mutable snapshot without copying and promptly releasing the view
    data = snapshot(original, result_type=bytearray)

The start/stop/step or length+offset question could be handled at that level by allowing both, but also offering lower level APIs with less argument processing overhead:

    def snapshot_slice(source, start, stop, step=1, *, result_type=bytes):
        ...

    def snapshot_at(source, *, offset=0, count=None, result_type=bytes):
        ...

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29178>
_______________________________________

From report at bugs.python.org  Sat Jan  7 23:45:00 2017
From: report at bugs.python.org (Roundup Robot)
Date: Sun, 08 Jan 2017 04:45:00 +0000
Subject: [issue29200] is it a bug in `functools._HashedSeq`
In-Reply-To: <1483840932.91.0.717045884042.issue29200@psf.upfronthosting.co.za>
Message-ID: <20170108044457.92958.62168.8CBBEF74@psf.io>


Roundup Robot added the comment:

New changeset 52d3cbcf546f by Raymond Hettinger in branch 'default':
Issue #29200: Add test for lru cache only calling __hash__ once
https://hg.python.org/cpython/rev/52d3cbcf546f

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29200>
_______________________________________

From report at bugs.python.org  Sat Jan  7 23:45:32 2017
From: report at bugs.python.org (Berker Peksag)
Date: Sun, 08 Jan 2017 04:45:32 +0000
Subject: [issue9338] argparse optionals with nargs='?',
 '*' or '+' can't be followed by positionals
In-Reply-To: <1279881989.28.0.725806934086.issue9338@psf.upfronthosting.co.za>
Message-ID: <1483850732.44.0.897831086885.issue9338@psf.upfronthosting.co.za>


Changes by Berker Peksag <berker.peksag at gmail.com>:


----------
nosy: +berker.peksag

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue9338>
_______________________________________

From report at bugs.python.org  Sat Jan  7 23:45:47 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Sun, 08 Jan 2017 04:45:47 +0000
Subject: [issue29200] is it a bug in `functools._HashedSeq`
In-Reply-To: <1483840932.91.0.717045884042.issue29200@psf.upfronthosting.co.za>
Message-ID: <1483850747.39.0.121609678209.issue29200@psf.upfronthosting.co.za>


Raymond Hettinger added the comment:

To honor your report, I've added a test for this functionality.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29200>
_______________________________________

From report at bugs.python.org  Sat Jan  7 23:53:18 2017
From: report at bugs.python.org (Roundup Robot)
Date: Sun, 08 Jan 2017 04:53:18 +0000
Subject: [issue29200] is it a bug in `functools._HashedSeq`
In-Reply-To: <1483840932.91.0.717045884042.issue29200@psf.upfronthosting.co.za>
Message-ID: <20170108045315.32608.97476.F720D4A2@psf.io>


Roundup Robot added the comment:

New changeset 2e7e91785306 by Raymond Hettinger in branch 'default':
Issue #29200: Fix test to use self.assertEqual instead of py.test style tests
https://hg.python.org/cpython/rev/2e7e91785306

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29200>
_______________________________________

From report at bugs.python.org  Sun Jan  8 00:15:20 2017
From: report at bugs.python.org (Jeff Knupp)
Date: Sun, 08 Jan 2017 05:15:20 +0000
Subject: [issue29081] time.strptime() return wrong result
In-Reply-To: <1482833112.13.0.0168836521105.issue29081@psf.upfronthosting.co.za>
Message-ID: <1483852520.99.0.656379540822.issue29081@psf.upfronthosting.co.za>


Jeff Knupp added the comment:

I believe this is working as intended. Remember, the '%w' directive instructs strptime to consider 0 to be Sunday, while tm_wday considers 0 Monday. In 2016, the %W directive means that the first week (week #1) starts on Monday, January 4th. If you go 52 weeks forward from the 4th, you get to Monday, December 26th. By asking for day 0 (%w=0), you want the *Sunday* of the 52nd week *from the first Monday of the year*. Since Monday is day 0 of that week, you want the Sunday that is 6 days from the 26th, or Jan 1, 2017.

One can certainly argue that tm_yday is documented to return an integer between [0,366] and thus we should never see 367, but it's the correct value given your input. The only other choice would be to raise an exception, which definitely shouldn't happen since the values you entered clearly match the format string spec.

Perhaps the docs should be updated, but when you consider that %W goes from [0,53], tm_yday can go well past 366 and still represent a semantically valid value.

----------
nosy: +jeffknupp

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29081>
_______________________________________

From report at bugs.python.org  Sun Jan  8 00:19:56 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Sun, 08 Jan 2017 05:19:56 +0000
Subject: [issue29195] Get rid of supporting outdated wrong keyword arguments
 in re methods
In-Reply-To: <1483790693.93.0.397432841389.issue29195@psf.upfronthosting.co.za>
Message-ID: <1483852796.73.0.825929816451.issue29195@psf.upfronthosting.co.za>


Raymond Hettinger added the comment:

It's too bad that "pattern" lost out to the less informative "string".

+1 for finishing the work and applying this patch.

----------
nosy: +rhettinger

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29195>
_______________________________________

From report at bugs.python.org  Sun Jan  8 00:24:27 2017
From: report at bugs.python.org (Martin Panter)
Date: Sun, 08 Jan 2017 05:24:27 +0000
Subject: [issue28815] test_socket fails if /proc/modules is existent but not
 readable
In-Reply-To: <1480263378.99.0.648069549511.issue28815@psf.upfronthosting.co.za>
Message-ID: <1483853067.12.0.986775301351.issue28815@psf.upfronthosting.co.za>


Changes by Martin Panter <vadmium+py at gmail.com>:


----------
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28815>
_______________________________________

From report at bugs.python.org  Sun Jan  8 01:09:09 2017
From: report at bugs.python.org (Berker Peksag)
Date: Sun, 08 Jan 2017 06:09:09 +0000
Subject: =?utf-8?q?=5Bissue9182=5D_document_=E2=80=9C--=E2=80=9D_as_a_way_to_disti?=
 =?utf-8?q?nguish_option_w/_narg=3D=27+=27_from_positional_argument_in_arg?=
 =?utf-8?q?parse?=
In-Reply-To: <1278433481.81.0.201692715302.issue9182@psf.upfronthosting.co.za>
Message-ID: <1483855749.18.0.969045538319.issue9182@psf.upfronthosting.co.za>


Changes by Berker Peksag <berker.peksag at gmail.com>:


----------
nosy: +berker.peksag

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue9182>
_______________________________________

From report at bugs.python.org  Sun Jan  8 01:13:18 2017
From: report at bugs.python.org (Jiajun Huang)
Date: Sun, 08 Jan 2017 06:13:18 +0000
Subject: [issue29200] is it a bug in `functools._HashedSeq`
In-Reply-To: <20170108045315.32608.97476.F720D4A2@psf.io>
Message-ID: <CAEV_hz1s7cDdE4-ddGdgpnf3YgYm0t6DOp3QTqpcpJXqv_mRNQ@mail.gmail.com>


Jiajun Huang added the comment:

thanks for reply :)

2017?1?8?????Roundup Robot <report at bugs.python.org> ???

>
> Roundup Robot added the comment:
>
> New changeset 2e7e91785306 by Raymond Hettinger in branch 'default':
> Issue #29200: Fix test to use self.assertEqual instead of py.test style
> tests
> https://hg.python.org/cpython/rev/2e7e91785306
>
> ----------
>
> _______________________________________
> Python tracker <report at bugs.python.org <javascript:;>>
> <http://bugs.python.org/issue29200>
> _______________________________________
>

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29200>
_______________________________________

From report at bugs.python.org  Sun Jan  8 01:24:43 2017
From: report at bugs.python.org (Steve Dower)
Date: Sun, 08 Jan 2017 06:24:43 +0000
Subject: [issue29201] Python 3.6 not working within VS 2015
In-Reply-To: <1483849460.53.0.951526575508.issue29201@psf.upfronthosting.co.za>
Message-ID: <1483856683.06.0.893897760647.issue29201@psf.upfronthosting.co.za>


Steve Dower added the comment:

This is a bug in Visual Studio, not Python.

There is an RC release of the Python support with a fix for this bug is available at https://github.com/Microsoft/PTVS on the releases page, and an actual update will appear in notifications within a couple of weeks. VS 2017 is not affected by the bug and supports Python 3.6 well.

----------
resolution:  -> third party
stage:  -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29201>
_______________________________________

From report at bugs.python.org  Sun Jan  8 03:12:42 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Sun, 08 Jan 2017 08:12:42 +0000
Subject: [issue29034] Fix memory leak and use-after-free in path_converter
In-Reply-To: <1482316498.7.0.932585385435.issue29034@psf.upfronthosting.co.za>
Message-ID: <1483863162.66.0.684486923452.issue29034@psf.upfronthosting.co.za>


Xiang Zhang added the comment:

path_converter-new-3.patch fixes the wrong return value on successful cases.

----------
Added file: http://bugs.python.org/file46206/path_converter-new-3.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29034>
_______________________________________

From report at bugs.python.org  Sun Jan  8 03:15:04 2017
From: report at bugs.python.org (Milan Oberkirch)
Date: Sun, 08 Jan 2017 08:15:04 +0000
Subject: [issue29142] urllib: no_proxy variable values with leading dot not
 properly handled
In-Reply-To: <1483452589.12.0.177615671636.issue29142@psf.upfronthosting.co.za>
Message-ID: <1483863304.86.0.709150918646.issue29142@psf.upfronthosting.co.za>


Milan Oberkirch added the comment:

Thanks for the feedback!

I changed the patch to use lstrip so that the FQDN "example.com." would match the NOPROXY entry "example.com.". I don't think we get FQDNs, but that's just my guess.

I think ideally "example.com" would also match "example.com." but that's a different issue.

----------
Added file: http://bugs.python.org/file46207/ignore_dots-v3.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29142>
_______________________________________

From report at bugs.python.org  Sun Jan  8 03:42:32 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Sun, 08 Jan 2017 08:42:32 +0000
Subject: [issue29034] Fix memory leak and use-after-free in path_converter
In-Reply-To: <1482316498.7.0.932585385435.issue29034@psf.upfronthosting.co.za>
Message-ID: <1483864952.3.0.974962277013.issue29034@psf.upfronthosting.co.za>


Changes by Xiang Zhang <angwerzx at 126.com>:


Added file: http://bugs.python.org/file46208/path_converter-new-4.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29034>
_______________________________________

From report at bugs.python.org  Sun Jan  8 03:52:43 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sun, 08 Jan 2017 08:52:43 +0000
Subject: [issue29034] Fix memory leak and use-after-free in path_converter
In-Reply-To: <1482316498.7.0.932585385435.issue29034@psf.upfronthosting.co.za>
Message-ID: <1483865563.41.0.0289725372038.issue29034@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

The last patch LGTM. Thanks Xiang! At the end this issue has appeared much more complex that it was looked at the start.

----------
assignee:  -> xiang.zhang
stage: patch review -> commit review

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29034>
_______________________________________

From report at bugs.python.org  Sun Jan  8 04:50:36 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Sun, 08 Jan 2017 09:50:36 +0000
Subject: [issue29202] Improve dict iteration
Message-ID: <1483869036.69.0.874318106576.issue29202@psf.upfronthosting.co.za>


New submission from Raymond Hettinger:

I haven't had much of a chance to comb through all the code it detail, but there are least a few places that loop over the whole entry table (size is dk_nentries) when fewer iterations would suffice (stop once mp->ma_used entries have been seen).  Since the table is usually dense packed to the left, this will give fewer loops and fewer memory accesses.

----------
assignee: serhiy.storchaka
components: Interpreter Core
files: dict_shorter_iteration.diff
keywords: patch
messages: 284970
nosy: inada.naoki, rhettinger, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Improve dict iteration
type: performance
versions: Python 3.6, Python 3.7
Added file: http://bugs.python.org/file46209/dict_shorter_iteration.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29202>
_______________________________________

From report at bugs.python.org  Sun Jan  8 04:50:48 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Sun, 08 Jan 2017 09:50:48 +0000
Subject: [issue29202] Improve dict iteration
In-Reply-To: <1483869036.69.0.874318106576.issue29202@psf.upfronthosting.co.za>
Message-ID: <1483869048.53.0.265421263419.issue29202@psf.upfronthosting.co.za>


Changes by Raymond Hettinger <raymond.hettinger at gmail.com>:


----------
priority: normal -> low

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29202>
_______________________________________

From report at bugs.python.org  Sun Jan  8 05:01:29 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Sun, 08 Jan 2017 10:01:29 +0000
Subject: [issue29203] With PEP 468,
 the lru cache not longer needs to sort keyword args
Message-ID: <1483869689.96.0.118801671139.issue29203@psf.upfronthosting.co.za>


New submission from Raymond Hettinger:

Since the ordering of keyword arguments is now guaranteed, the LRU cache doesn't need to sort any longer.

The will give a small change in behavior that I don't care about.  A call f(a=1, b=2) would now be cached separately from f(b=2, a=1).  That won't arise often and isn't really different than the status quo where f(1, b=2) or f(1, 2) are already cached separately.  Overall it is a net win by saving the sorting step on every call.

----------
assignee: serhiy.storchaka
components: Library (Lib)
files: lru_468.diff
keywords: patch
messages: 284971
nosy: rhettinger, serhiy.storchaka
priority: normal
severity: normal
stage: patch review
status: open
title: With PEP 468, the lru cache not longer needs to sort keyword args
type: performance
versions: Python 3.7
Added file: http://bugs.python.org/file46210/lru_468.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29203>
_______________________________________

From report at bugs.python.org  Sun Jan  8 05:25:25 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sun, 08 Jan 2017 10:25:25 +0000
Subject: [issue29203] With PEP 468,
 the lru cache not longer needs to sort keyword args
In-Reply-To: <1483869689.96.0.118801671139.issue29203@psf.upfronthosting.co.za>
Message-ID: <1483871124.99.0.524505526853.issue29203@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

I think this is a bug and it should be fixed in 3.6. Currenly lru_cache breaks PEP 468 (Preserving Keyword Argument Order).

>>> from functools import lru_cache
>>> @lru_cache()
... def f(**kwargs):
...     return list(kwargs.items())
... 
>>> f(a=1, b=2)
[('a', 1), ('b', 2)]
>>> f(b=2, a=1)
[('a', 1), ('b', 2)]

C implementation should be changed too.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29203>
_______________________________________

From report at bugs.python.org  Sun Jan  8 05:35:38 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Sun, 08 Jan 2017 10:35:38 +0000
Subject: [issue29203] With PEP 468,
 the lru cache not longer needs to sort keyword args
In-Reply-To: <1483869689.96.0.118801671139.issue29203@psf.upfronthosting.co.za>
Message-ID: <1483871738.14.0.51721124865.issue29203@psf.upfronthosting.co.za>


Raymond Hettinger added the comment:

> I think this is a bug and it should be fixed in 3.6.

I concur.  Do you care to whip-up a patch (it is late here).

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29203>
_______________________________________

From report at bugs.python.org  Sun Jan  8 05:40:07 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sun, 08 Jan 2017 10:40:07 +0000
Subject: [issue29202] Improve dict iteration
In-Reply-To: <1483869036.69.0.874318106576.issue29202@psf.upfronthosting.co.za>
Message-ID: <1483872007.22.0.0639897768324.issue29202@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

I'm wondering why new dict implementation don't keep the array of items compact? Original Raymond's implementation did not preserve the order after deletions, but saved items in an array without gaps. This could simplify and speed up an iteration (no need to check values for NULL, and needed to iterate fewer elements), and could get rid of reallocations in often mutated dicts. I haven't found clear explanation of this.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29202>
_______________________________________

From report at bugs.python.org  Sun Jan  8 06:13:24 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sun, 08 Jan 2017 11:13:24 +0000
Subject: [issue29204] Add code deprecations in ElementTree
Message-ID: <1483874003.49.0.675614316595.issue29204@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka:

Some deprecated ElementTree features are deprecated only in the documentation or in Python implementation (that is virtually the same since C implementation is default). Proposed patch adds missed deprecations is code. It also makes warnings be ignored only in tests where they are expected. This is possible since converting doctests to unittests some time ago.

Added deprecations:

* Element.getchildren() and Element.getiterator() methods. They were deprecated in the documentation and in Python implementation in 2.7 and 3.2.

* The xml.etree.cElementTree module. Deprecated in the documentation in 3.3.

* The html argument of XMLParser. Deprecated in the documentation in 3.4.

Ned, is it appropriate to commit the patch (or its part) in 3.6? The discrepancy between Python and C implementation can be considered as a bug. What are your thoughts?

----------
components: Extension Modules, Library (Lib)
files: etree_deprecations.patch
keywords: patch
messages: 284975
nosy: ned.deily, serhiy.storchaka
priority: normal
severity: normal
stage: patch review
status: open
title: Add code deprecations in ElementTree
type: enhancement
versions: Python 3.7
Added file: http://bugs.python.org/file46211/etree_deprecations.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29204>
_______________________________________

From report at bugs.python.org  Sun Jan  8 06:13:41 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sun, 08 Jan 2017 11:13:41 +0000
Subject: [issue29204] Add code deprecations in ElementTree
In-Reply-To: <1483874003.49.0.675614316595.issue29204@psf.upfronthosting.co.za>
Message-ID: <1483874021.49.0.346020548877.issue29204@psf.upfronthosting.co.za>


Changes by Serhiy Storchaka <storchaka+cpython at gmail.com>:


----------
nosy: +eli.bendersky, scoder

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29204>
_______________________________________

From report at bugs.python.org  Sun Jan  8 06:13:45 2017
From: report at bugs.python.org (Martin Panter)
Date: Sun, 08 Jan 2017 11:13:45 +0000
Subject: [issue28997] test_readline.test_nonascii fails on Android
In-Reply-To: <1481965369.72.0.409169776064.issue28997@psf.upfronthosting.co.za>
Message-ID: <1483874025.39.0.721624640463.issue28997@psf.upfronthosting.co.za>


Martin Panter added the comment:

.
Thanks for the explanation. It sounds like the Readline library assumes an ASCII-only locale and sets its ?convert-meta? variable to ?on?. But Python assumes UTF-8 and inputs b"\xC3\xAB" to the terminal. Readline converts the input to two escape sequences: "\N{ESC}\x43" == "\N{ESC}C" (Alt + Capital C), which probably runs the ?capitalize-word? command, and "\N{ESC}\x2B" == "\N{ESC}+" (Alt + Plus), which presumably generates the bell character.

I don?t understand why you say Readline is ?correctly? using the C or Posix locale (ASCII), while my understanding is Python on Android always uses UTF-8 as the locale encoding. It seems there is an inconsistency with the locale or encodings being used.

Or is this just an obscure case that you choose not to support on Android, and therefore skip the test?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28997>
_______________________________________

From report at bugs.python.org  Sun Jan  8 06:28:46 2017
From: report at bugs.python.org (Stefan Behnel)
Date: Sun, 08 Jan 2017 11:28:46 +0000
Subject: [issue27973] urllib.urlretrieve() fails on second ftp transfer
In-Reply-To: <1473174596.52.0.952829730129.issue27973@psf.upfronthosting.co.za>
Message-ID: <1483874926.4.0.144873246347.issue27973@psf.upfronthosting.co.za>


Stefan Behnel added the comment:

This bug makes the installation of lxml fail on many systems (especially MacOS) when pip installs from sources, because it tries to download its library dependencies from FTP and crashes due to the FTP "error". I guess the current fix is to not use urllib for that and instead implement the FTP downloads separately. That's unfortunate.

----------
nosy: +scoder

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue27973>
_______________________________________

From report at bugs.python.org  Sun Jan  8 06:29:20 2017
From: report at bugs.python.org (Jelle Zijlstra)
Date: Sun, 08 Jan 2017 11:29:20 +0000
Subject: [issue29205] col_offset for AsyncFunctionDef AST nodes is wrong
Message-ID: <1483874960.49.0.58571949289.issue29205@psf.upfronthosting.co.za>


New submission from Jelle Zijlstra:

The col_offset attribute for ast.AsyncFunctionDef objects points to the "def" keyword, not to the "async" keyword that actually starts the node.

Test case:

In [18]: code = 'async def f(): pass'

In [19]: tree = ast.parse(code)

In [20]: tree.body[0]
Out[20]: <_ast.AsyncFunctionDef at 0x7f5cb6a58f60>

In [21]: tree.body[0].col_offset
Out[21]: 6

----------
components: Library (Lib)
messages: 284978
nosy: Jelle Zijlstra
priority: normal
severity: normal
status: open
title: col_offset for AsyncFunctionDef AST nodes is wrong
versions: Python 3.5, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29205>
_______________________________________

From report at bugs.python.org  Sun Jan  8 06:29:23 2017
From: report at bugs.python.org (John Lehmann)
Date: Sun, 08 Jan 2017 11:29:23 +0000
Subject: [issue29194] importlib reload fails for module supplied as argument
 to command line
In-Reply-To: <1483790533.68.0.771948929262.issue29194@psf.upfronthosting.co.za>
Message-ID: <1483874963.95.0.385921366654.issue29194@psf.upfronthosting.co.za>


John Lehmann added the comment:

You get a slightly different error using "-m".

I'll raise a new defect - with some background that may make the bug seem more reasonable.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29194>
_______________________________________

From report at bugs.python.org  Sun Jan  8 06:31:04 2017
From: report at bugs.python.org (Martin Panter)
Date: Sun, 08 Jan 2017 11:31:04 +0000
Subject: [issue29204] Add code deprecations in ElementTree
In-Reply-To: <1483874003.49.0.675614316595.issue29204@psf.upfronthosting.co.za>
Message-ID: <1483875064.52.0.00514671883915.issue29204@psf.upfronthosting.co.za>


Martin Panter added the comment:

Isn?t cElementTree useful and recommended in 2.7? It would be awkward to deprecate it in Python 3. But I guess the other cases should be okay to deprecate in 3.7.

----------
nosy: +martin.panter

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29204>
_______________________________________

From report at bugs.python.org  Sun Jan  8 06:35:30 2017
From: report at bugs.python.org (Jelle Zijlstra)
Date: Sun, 08 Jan 2017 11:35:30 +0000
Subject: [issue29205] col_offset for AsyncFunctionDef AST nodes is wrong
In-Reply-To: <1483874960.49.0.58571949289.issue29205@psf.upfronthosting.co.za>
Message-ID: <1483875330.77.0.796018892084.issue29205@psf.upfronthosting.co.za>


Jelle Zijlstra added the comment:

The col_offset is actually correct when there is a decorator:

In [26]: code = '@decorator\nasync def f(): pass'

In [27]: tree = ast.parse(code)

In [28]: tree.body[0].col_offset
Out[28]: 0

The same issue appears with async for and async with:

In [31]: code = '@decorator\nasync def f():\n    async for x in y: pass'

In [32]: tree = ast.parse(code)

In [34]: tree.body[0].body[0]
Out[34]: <_ast.AsyncFor at 0x7f5cb6a77198>

In [35]: tree.body[0].body[0].col_offset
Out[35]: 10

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29205>
_______________________________________

From report at bugs.python.org  Sun Jan  8 06:43:51 2017
From: report at bugs.python.org (John Lehmann)
Date: Sun, 08 Jan 2017 11:43:51 +0000
Subject: [issue29206] importlib reload fails for module supplied as argument
 to command line
Message-ID: <1483875831.65.0.670865149709.issue29206@psf.upfronthosting.co.za>


New submission from John Lehmann:

In testing the py3 port for the web framework web.py, I found a limitation of importlib.reload.

A module that was loaded via the command line cannot be reloaded with importlib.reload.

The basic mode of operation for web.py is to create a web application as a single module, say, "app.py" and run this with "python app.py".

When in development mode, on each page view, the modification time for each file backing each module is checked for changes.  If the file has changed, the file is reloaded.

This allows for an iterative development mode familiar to web developers since the glory days of writing VB pages in ASP.

The problem occurs when the file is loaded directly, or with "-m".

For example with the attached file:

  $ python reloader.py
  Traceback (most recent call last):
    File "reloader.py", line 31, in <module>
      reload_module("__main__")
    File "reloader.py", line 28, in reload_module
      importlib.reload(module)
    File "/usr/local/var/pyenv/versions/3.5.2/lib/python3.5/importlib/__init__.py", line 166, in reload
      _bootstrap._exec(spec, module)
    File "<frozen importlib._bootstrap>", line 607, in _exec
  AttributeError: 'NoneType' object has no attribute 'name'

And with -m:

  $ python -m reloader
  Traceback (most recent call last):
    File "/usr/local/var/pyenv/versions/3.5.2/lib/python3.5/runpy.py", line 184, in _run_module_as_main
      "__main__", mod_spec)
    File "/usr/local/var/pyenv/versions/3.5.2/lib/python3.5/runpy.py", line 85, in _run_code
      exec(code, run_globals)
    File "reloader.py", line 31, in <module>
      reload_module("__main__")
    File "reloader.py", line 28, in reload_module
      importlib.reload(module)
    File "/usr/local/var/pyenv/versions/3.5.2/lib/python3.5/importlib/__init__.py", line 147, in reload
      raise ImportError(msg.format(name), name=name)
  ImportError: module reloader not in sys.modules

Note that there are two different error messages, neither of which is great.

There are workarounds, but given that this works in previous version of python, it might be better to fix it.  Or to have an explicit error message that describes the problem more precisely:

  "Cannot reload __main__"

----------
files: reloader.py
messages: 284982
nosy: brett.cannon, j1o1h1n, r.david.murray
priority: normal
severity: normal
status: open
title: importlib reload fails for module supplied as argument to command line
type: behavior
versions: Python 3.5
Added file: http://bugs.python.org/file46212/reloader.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29206>
_______________________________________

From report at bugs.python.org  Sun Jan  8 06:47:10 2017
From: report at bugs.python.org (John Lehmann)
Date: Sun, 08 Jan 2017 11:47:10 +0000
Subject: [issue29206] importlib reload fails for module supplied as argument
 to command line
In-Reply-To: <1483875831.65.0.670865149709.issue29206@psf.upfronthosting.co.za>
Message-ID: <1483876030.15.0.80485959662.issue29206@psf.upfronthosting.co.za>


John Lehmann added the comment:

I may not have been clear as to how the problem seems when working with a web.py application:

* visit the web page, see something that needs fixing
* make the required change to app.py
* reload the webpage
* error occurs

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29206>
_______________________________________

From report at bugs.python.org  Sun Jan  8 06:52:03 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sun, 08 Jan 2017 11:52:03 +0000
Subject: [issue29203] With PEP 468,
 the lru cache not longer needs to sort keyword args
In-Reply-To: <1483869689.96.0.118801671139.issue29203@psf.upfronthosting.co.za>
Message-ID: <1483876323.9.0.27642030744.issue29203@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

Here is a patch that fixes also C implementation and has tests.

----------
type: performance -> behavior
versions: +Python 3.6
Added file: http://bugs.python.org/file46213/lru_cache_pep468.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29203>
_______________________________________

From report at bugs.python.org  Sun Jan  8 06:56:39 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sun, 08 Jan 2017 11:56:39 +0000
Subject: [issue29204] Add code deprecations in ElementTree
In-Reply-To: <1483874003.49.0.675614316595.issue29204@psf.upfronthosting.co.za>
Message-ID: <1483876599.9.0.569013584523.issue29204@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

Yes, I have a doubt about this too.

Perhaps it can be just removed. The idiomatic code in Python 2 is:

try:
    import xml.etree.cElementTree as ET
except ImportError:
    import xml.etree.ElementTree as ET

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29204>
_______________________________________

From report at bugs.python.org  Sun Jan  8 07:00:55 2017
From: report at bugs.python.org (Stefan Behnel)
Date: Sun, 08 Jan 2017 12:00:55 +0000
Subject: [issue27973] urllib.urlretrieve() fails on second ftp transfer
In-Reply-To: <1473174596.52.0.952829730129.issue27973@psf.upfronthosting.co.za>
Message-ID: <1483876855.32.0.795445622995.issue27973@psf.upfronthosting.co.za>


Stefan Behnel added the comment:

Actually, it seems that calling urlcleanup() is sufficient as a work-around.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue27973>
_______________________________________

From report at bugs.python.org  Sun Jan  8 07:22:25 2017
From: report at bugs.python.org (Stefan Behnel)
Date: Sun, 08 Jan 2017 12:22:25 +0000
Subject: [issue29204] Add code deprecations in ElementTree
In-Reply-To: <1483874003.49.0.675614316595.issue29204@psf.upfronthosting.co.za>
Message-ID: <1483878145.04.0.769295612651.issue29204@psf.upfronthosting.co.za>


Stefan Behnel added the comment:

I'm ok with the deprecations.

Regarding the cElementTree module, this is a bit problematic. The idiomatic import has lost its use in Py2.5 when ET and cET were added to the stdlib, so code that was written for Py2.5 or later (e.g. because it uses generators) might no longer have that cascade. On the other hand, issuing a warning for the module would also hit this import cascade, even though the code would work just fine without cElementTree. One argument speaks for deprecation, the other for removal.

However, cElementTree is redundant now, so it should be removed eventually. And since that removal would break some code anyway, I'd be ok with just removing it without prior import warnings. People can then decide whether they want to fix their code by adding the well-known import cascade (and not get annoying warnings for it) or by switching entirely to plain ET and not looking back.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29204>
_______________________________________

From report at bugs.python.org  Sun Jan  8 07:24:20 2017
From: report at bugs.python.org (INADA Naoki)
Date: Sun, 08 Jan 2017 12:24:20 +0000
Subject: [issue29202] Improve dict iteration
In-Reply-To: <1483869036.69.0.874318106576.issue29202@psf.upfronthosting.co.za>
Message-ID: <1483878260.45.0.404118191475.issue29202@psf.upfronthosting.co.za>


INADA Naoki added the comment:

> I'm wondering why new dict implementation don't keep the array of items compact? Original Raymond's implementation did not preserve the order after deletions, but saved items in an array without gaps. This could simplify and speed up an iteration (no need to check values for NULL, and needed to iterate fewer elements), and could get rid of reallocations in often mutated dicts. I haven't found clear explanation of this.

As far as I remember, I choose it because:

1. easy to explain, easy to discussion.

"keep insertion order" is easier than "keep insertion order unless deletion".

I want to start discussion based on most simple behavior.
But my patch was reviewed by core developers in sprint, when right before 3.6b1.

2. I want to share same implementation with OrderedDict, like PyPy.

3. Make patch compact.

If we compact dk_entries when deletion, we need another dk_indices compaction.
Before sprint, my patch wasn't reviewed for months.  I was afraid that my patch wasn't reviewed by 3.6b1.


For now, "namespace dict keeps insertion order" is language spec.
`del` shouldn't break insertion order.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29202>
_______________________________________

From report at bugs.python.org  Sun Jan  8 07:31:45 2017
From: report at bugs.python.org (INADA Naoki)
Date: Sun, 08 Jan 2017 12:31:45 +0000
Subject: [issue29202] Improve dict iteration
In-Reply-To: <1483869036.69.0.874318106576.issue29202@psf.upfronthosting.co.za>
Message-ID: <1483878705.58.0.516963300315.issue29202@psf.upfronthosting.co.za>


INADA Naoki added the comment:

patch LGTM.

> Since the table is usually dense packed to the left, this will give fewer loops and fewer memory accesses.

Since dk_nentries is "right end of active entries", this patch doesn't
affect for "dence packed to the left" dict.
But it can affect when dk_entries is sparce, and there are many deleted entries on the right end.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29202>
_______________________________________

From report at bugs.python.org  Sun Jan  8 07:38:41 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sun, 08 Jan 2017 12:38:41 +0000
Subject: [issue29202] Improve dict iteration
In-Reply-To: <1483869036.69.0.874318106576.issue29202@psf.upfronthosting.co.za>
Message-ID: <1483879121.12.0.393235992573.issue29202@psf.upfronthosting.co.za>


Changes by Serhiy Storchaka <storchaka+cpython at gmail.com>:


Added file: http://bugs.python.org/file46214/dict_shorter_iteration.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29202>
_______________________________________

From report at bugs.python.org  Sun Jan  8 07:45:02 2017
From: report at bugs.python.org (INADA Naoki)
Date: Sun, 08 Jan 2017 12:45:02 +0000
Subject: [issue29202] Improve dict iteration
In-Reply-To: <1483869036.69.0.874318106576.issue29202@psf.upfronthosting.co.za>
Message-ID: <1483879502.56.0.201496102574.issue29202@psf.upfronthosting.co.za>


INADA Naoki added the comment:

While this isn't bugfix, I'm +1 to commit this to 3.6 branch to reduce
future maintenance cost.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29202>
_______________________________________

From report at bugs.python.org  Sun Jan  8 07:55:55 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sun, 08 Jan 2017 12:55:55 +0000
Subject: [issue29202] Improve dict iteration
In-Reply-To: <1483869036.69.0.874318106576.issue29202@psf.upfronthosting.co.za>
Message-ID: <1483880155.63.0.262304619709.issue29202@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

In general the idea LGTM. It it is slightly dangerous if dict structure is broken. j is incremented only when value != NULL, and if ma_used is not correct, this can cause reading out of the end of an array. Of course this should never happen. But if there is a chance to add some assertions or additional checks in debug mode, it would be nice. Unless this complicate the code to much.

The i counter is not used in some loops. It can be eliminated.

----------
assignee: serhiy.storchaka -> rhettinger
stage:  -> patch review

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29202>
_______________________________________

From report at bugs.python.org  Sun Jan  8 08:06:48 2017
From: report at bugs.python.org (Terry J. Reedy)
Date: Sun, 08 Jan 2017 13:06:48 +0000
Subject: [issue27603] Migrate IDLE context menu items to shell extension
In-Reply-To: <1469332799.74.0.70339682021.issue27603@psf.upfronthosting.co.za>
Message-ID: <1483880808.56.0.711043065445.issue27603@psf.upfronthosting.co.za>


Changes by Terry J. Reedy <tjreedy at udel.edu>:


----------
assignee: terry.reedy -> 

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue27603>
_______________________________________

From report at bugs.python.org  Sun Jan  8 08:11:16 2017
From: report at bugs.python.org (ita1024)
Date: Sun, 08 Jan 2017 13:11:16 +0000
Subject: [issue29174] 'NoneType' object is not callable in subprocess.py
In-Reply-To: <1483688938.47.0.507597836624.issue29174@psf.upfronthosting.co.za>
Message-ID: <1483881076.12.0.469536758118.issue29174@psf.upfronthosting.co.za>


ita1024 added the comment:

> The code in test.py is not realistic.

My application uses a process pool that gets freed when my application terminates.

> You might have more influence with a realistic use case.

I am reporting this because the users of my application are complaining.

> Victor has committed a fix for the ?exception ignored? problem, so assuming it works

I am looking forward to seeing that. Thanks Victor!

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29174>
_______________________________________

From report at bugs.python.org  Sun Jan  8 09:03:48 2017
From: report at bugs.python.org (Jelle Zijlstra)
Date: Sun, 08 Jan 2017 14:03:48 +0000
Subject: [issue29205] col_offset for AsyncFunctionDef AST nodes is wrong
In-Reply-To: <1483874960.49.0.58571949289.issue29205@psf.upfronthosting.co.za>
Message-ID: <1483884228.47.0.0931374775456.issue29205@psf.upfronthosting.co.za>


Changes by Jelle Zijlstra <jelle.zijlstra at gmail.com>:


----------
keywords: +patch
Added file: http://bugs.python.org/file46215/issue29205.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29205>
_______________________________________

From report at bugs.python.org  Sun Jan  8 10:31:07 2017
From: report at bugs.python.org (Roundup Robot)
Date: Sun, 08 Jan 2017 15:31:07 +0000
Subject: [issue29034] Fix memory leak and use-after-free in path_converter
In-Reply-To: <1482316498.7.0.932585385435.issue29034@psf.upfronthosting.co.za>
Message-ID: <20170108153104.27780.66392.A0C89C96@psf.io>


Roundup Robot added the comment:

New changeset 553eedb8b247 by Xiang Zhang in branch '3.6':
Issue #29034: Fix memory leak and use-after-free in path_converter.
https://hg.python.org/cpython/rev/553eedb8b247

New changeset 08042f0dbb67 by Xiang Zhang in branch 'default':
Issue #29034: Merge 3.6.
https://hg.python.org/cpython/rev/08042f0dbb67

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29034>
_______________________________________

From report at bugs.python.org  Sun Jan  8 10:52:16 2017
From: report at bugs.python.org (petevine)
Date: Sun, 08 Jan 2017 15:52:16 +0000
Subject: [issue29207] make install fails on ARM
Message-ID: <1483890736.55.0.166449192505.issue29207@psf.upfronthosting.co.za>


New submission from petevine:

I've just tried installing 2.7.13 on my ARM machine but after a successful build, `make install` fails without any hint what's wrong:

Compiling /usr/lib/python2.7/xmllib.py ...
Compiling /usr/lib/python2.7/xmlrpclib.py ...
Compiling /usr/lib/python2.7/zipfile.py ...
make: *** [libinstall] Error 1

Python 3.5.2 installation was successful on the same machine a little earlier. At the very least, a hint should be provided on how to get more verbose info.

----------
components: Build
messages: 284994
nosy: petevine
priority: normal
severity: normal
status: open
title: make install fails on ARM
type: crash
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29207>
_______________________________________

From report at bugs.python.org  Sun Jan  8 11:03:39 2017
From: report at bugs.python.org (Xavier de Gaye)
Date: Sun, 08 Jan 2017 16:03:39 +0000
Subject: [issue29180] skip tests that raise PermissionError in test_os
 (non-root user on Android)
In-Reply-To: <1483715692.75.0.979129057464.issue29180@psf.upfronthosting.co.za>
Message-ID: <1483891419.18.0.567423052717.issue29180@psf.upfronthosting.co.za>


Xavier de Gaye added the comment:

> If test_os is the only user of the helper, I'd say let's keep it in Lib/test/test_os.py for now.

I agree with that concern.

support.os_link() is also used by a patch in issue 29181, and I am planning to modify the changes made by issue 28759 to also use it so that tests calling os.link() are skipped based on the fact that they raise PermissionError instead of based on the fact that they are being run by an Android non-root user as is currently the case.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29180>
_______________________________________

From report at bugs.python.org  Sun Jan  8 11:50:12 2017
From: report at bugs.python.org (Dustin Spicuzza)
Date: Sun, 08 Jan 2017 16:50:12 +0000
Subject: [issue29208] BlockingIOError during system startup
Message-ID: <1483894212.45.0.948118557952.issue29208@psf.upfronthosting.co.za>


New submission from Dustin Spicuzza:

I haven't dug into this very deeply yet, so I'm seeking some clarity on this issue before doing so.

According to my reading of PEP 522 (https://www.python.org/dev/peps/pep-0522/), BlockingIOError will be raised in security sensitive APIs, but not when just importing the random module. However, it seems that this is not actually the case:

BlockingIOError: [Errno 11] Resource temporarily unavailable 
  
 The above exception was the direct cause of the following exception: 
  
 Traceback (most recent call last): 
   File "/home/lvuser/py/robot.py", line 15, in <module> 
     import wpilib 
   File "/usr/local/lib/python3.6/site-packages/wpilib/__init__.py", line 72, in <module> 
     from ._impl.main import run 
   File "/usr/local/lib/python3.6/site-packages/wpilib/_impl/main.py", line 7, in <module> 
     from pkg_resources import iter_entry_points 
   File "/usr/local/lib/python3.6/site-packages/pkg_resources/__init__.py", line 36, in <module> 
     import email.parser 
   File "/usr/local/lib/python3.6/email/parser.py", line 12, in <module> 
     from email.feedparser import FeedParser, BytesFeedParser 
   File "/usr/local/lib/python3.6/email/feedparser.py", line 27, in <module> 
     from email._policybase import compat32 
   File "/usr/local/lib/python3.6/email/_policybase.py", line 9, in <module> 
     from email.utils import _has_surrogates 
   File "/usr/local/lib/python3.6/email/utils.py", line 28, in <module> 
     import random 
   File "/usr/local/lib/python3.6/random.py", line 742, in <module> 
     _inst = Random() 
 SystemError: <class 'random.Random'> returned a result with an error set 


Background: I'm the maintainer of the RobotPy project, which allows students to use Python to control their robots in the FIRST Robotics competition. The robot controller we use is an NI roboRIO, which is powered by an ARM cortex-a9.

This error was encountered when the robot is booting up with a python program. Presumably the program is started early enough in the boot process where the urandom blocking issue mentioned in the PEP occurs.

I'd love some help figuring out what's going on here, and the best way to fix it. Thanks!

I'm also tracking this issue at https://github.com/robotpy/robotpy-wpilib/issues/243

----------
components: Library (Lib)
messages: 284996
nosy: virtuald
priority: normal
severity: normal
status: open
title: BlockingIOError during system startup
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29208>
_______________________________________

From report at bugs.python.org  Sun Jan  8 11:59:31 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sun, 08 Jan 2017 16:59:31 +0000
Subject: [issue29034] Fix memory leak and use-after-free in path_converter
In-Reply-To: <1482316498.7.0.932585385435.issue29034@psf.upfronthosting.co.za>
Message-ID: <1483894771.84.0.963831237294.issue29034@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

Is 3.5 free from all these bugs?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29034>
_______________________________________

From report at bugs.python.org  Sun Jan  8 12:03:53 2017
From: report at bugs.python.org (Dustin Spicuzza)
Date: Sun, 08 Jan 2017 17:03:53 +0000
Subject: [issue29208] BlockingIOError during system startup
In-Reply-To: <1483894212.45.0.948118557952.issue29208@psf.upfronthosting.co.za>
Message-ID: <1483895033.92.0.456878128395.issue29208@psf.upfronthosting.co.za>


Dustin Spicuzza added the comment:

Crap, after reading more carefully, I see that PEP was rejected in favor of PEP 524.

According to that PEP, then I must implement the wait_for_system_rng() function? It's a bit weird, because I'm not explicitly using the random module and don't care to use it, I'm using pkg_resources... which is importing the email parser which imports random which creates a random instance which causes this bug.

It seems to me then that the stdlib should be modified to avoid importing random unless they actually need it, otherwise this gets triggered. Or am I missing something here?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29208>
_______________________________________

From report at bugs.python.org  Sun Jan  8 12:13:47 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Sun, 08 Jan 2017 17:13:47 +0000
Subject: [issue29034] Fix memory leak and use-after-free in path_converter
In-Reply-To: <1482316498.7.0.932585385435.issue29034@psf.upfronthosting.co.za>
Message-ID: <1483895627.93.0.261339286087.issue29034@psf.upfronthosting.co.za>


Xiang Zhang added the comment:

> Is 3.5 free from all these bugs?

path_converter is much simpler in 3.5 so I think it's free of these bugs. :-)

----------
resolution:  -> fixed
stage: commit review -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29034>
_______________________________________

From report at bugs.python.org  Sun Jan  8 12:56:26 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sun, 08 Jan 2017 17:56:26 +0000
Subject: [issue29209] Remove old-deprecated ElementTree features
Message-ID: <1483898185.03.0.0187159979308.issue29209@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka:

Proposed patch removes old-deprecated ElementTree features.

* Methods Element.getchildren(), Element.getiterator() and ElementTree.getiterator() deprecated in 2.7 and 3.2.
 Use list(elem) or iteration instead of getchildren(), methods iter() instead of getiterator(). 

* The html argument of XMLParser deprecated in 3.4.
 The rest of arguments are keyword-only now (passing them as keywords was recommended in the documentatin).

* The support of the doctype() method of XMLParser subclasses and its default implementation. Deprecated in 3.2. Define the doctype() method on a custom TreeBuilder target instead.

* The xml.etree.cElementTree module deprecated in 3.3.

Unfortunately some of these deprecations are in the documentation only or in Python implementatation, but not in C implementatation. Perhaps missed warnings should be added first (see issue29204). But if commit the patch from issue29204 in 3.6, perhaps deprecated features could be removed in 3.7.

----------
components: Extension Modules, Library (Lib), XML
files: etree_remove_deprecated.patch
keywords: patch
messages: 285000
nosy: eli.bendersky, ned.deily, scoder, serhiy.storchaka
priority: normal
severity: normal
stage: patch review
status: open
title: Remove old-deprecated ElementTree features
type: enhancement
versions: Python 3.7
Added file: http://bugs.python.org/file46216/etree_remove_deprecated.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29209>
_______________________________________

From report at bugs.python.org  Sun Jan  8 12:56:53 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sun, 08 Jan 2017 17:56:53 +0000
Subject: [issue29209] Remove old-deprecated ElementTree features
In-Reply-To: <1483898185.03.0.0187159979308.issue29209@psf.upfronthosting.co.za>
Message-ID: <1483898213.81.0.46262793601.issue29209@psf.upfronthosting.co.za>


Changes by Serhiy Storchaka <storchaka+cpython at gmail.com>:


----------
dependencies: +Add code deprecations in ElementTree

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29209>
_______________________________________

From report at bugs.python.org  Sun Jan  8 13:03:41 2017
From: report at bugs.python.org (petevine)
Date: Sun, 08 Jan 2017 18:03:41 +0000
Subject: [issue29207] make install fails on ARM
In-Reply-To: <1483890736.55.0.166449192505.issue29207@psf.upfronthosting.co.za>
Message-ID: <1483898621.91.0.272189516878.issue29207@psf.upfronthosting.co.za>


petevine added the comment:

In case a clue might be gleaned right away from `make test`, I'm posting the final part of the output here:

[394/401/1] test_xmlrpc
../Python-2.7.13/Lib/threading.py:846: DeprecationWarning: sys.exc_clear() not supported in 3.x; use except clauses
  self.__exc_clear()
[395/401/1] test_xpickle
[396/401/1] test_xrange
[397/401/1] test_zipfile
[398/401/1] test_zipfile64
test_zipfile64 skipped -- test requires loads of disk-space bytes and a long time to run
[399/401/1] test_zipimport
[400/401/1] test_zipimport_support
[401/401/1] test_zlib
359 tests OK.
1 test failed:
    test_gdb
1 test altered the execution environment:
    test_import
40 tests skipped:
    test_aepack test_al test_applesingle test_bsddb test_bsddb185
    test_bsddb3 test_cd test_cl test_codecmaps_cn test_codecmaps_hk
    test_codecmaps_jp test_codecmaps_kr test_codecmaps_tw test_curses
    test_dl test_gl test_idle test_imgfile test_kqueue
    test_linuxaudiodev test_macos test_macostools test_msilib
    test_ossaudiodev test_scriptpackages test_smtpnet
    test_socketserver test_startfile test_sunaudiodev test_tcl
    test_timeout test_tk test_ttk_guionly test_ttk_textonly
    test_turtle test_urllib2net test_urllibnet test_winreg
    test_winsound test_zipfile64
8 skips unexpected on linux2:
    test_bsddb test_bsddb3 test_idle test_tcl test_tk test_ttk_guionly
    test_ttk_textonly test_turtle
make: *** [test] Error 1

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29207>
_______________________________________

From report at bugs.python.org  Sun Jan  8 13:22:27 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sun, 08 Jan 2017 18:22:27 +0000
Subject: [issue29210] Remove the support of the exclude argument in
 tarfile.TarFile.add()
Message-ID: <1483899747.86.0.0120992057455.issue29210@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka:

Proposed patch removes the support of the exclude argument in tarfile.TarFile.add(). It was deprecated in 2.7 and 3.2. The filter argument that supersedes it is available in all maintained releases.

----------
components: Library (Lib)
files: remove_tarfile_add_exclude.patch
keywords: patch
messages: 285002
nosy: lars.gustaebel, serhiy.storchaka
priority: normal
severity: normal
stage: patch review
status: open
title: Remove the support of the exclude argument in tarfile.TarFile.add()
type: enhancement
versions: Python 3.7
Added file: http://bugs.python.org/file46217/remove_tarfile_add_exclude.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29210>
_______________________________________

From report at bugs.python.org  Sun Jan  8 14:00:30 2017
From: report at bugs.python.org (Brett Cannon)
Date: Sun, 08 Jan 2017 19:00:30 +0000
Subject: [issue29206] importlib reload fails for module supplied as argument
 to command line
In-Reply-To: <1483875831.65.0.670865149709.issue29206@psf.upfronthosting.co.za>
Message-ID: <1483902030.37.0.0940134020462.issue29206@psf.upfronthosting.co.za>


Brett Cannon added the comment:

The error from using -m stems from runpy not aliasing the module as both "__main__" and "reloader". I suspect the reason is that having a module under two names gets really messy because if you update just one of those modules you leave the  other one around and then you have skewed the results.

As for your use-case, be aware that importlib.reload() is not designed for it. You will still have objects floating around in the interpreter that are using the original module and so you're not going to get all instances of objects automatically updated to the new code. The only way to guarantee usage of the new code is to restart the interpreter.

I get why you want this, John, but Python is simply not structured to support what you're after. Closing as "won't fix".

----------
nosy: +ncoghlan
resolution:  -> wont fix
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29206>
_______________________________________

From report at bugs.python.org  Sun Jan  8 14:04:52 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sun, 08 Jan 2017 19:04:52 +0000
Subject: [issue28167] remove platform.linux_distribution()
In-Reply-To: <1473933007.75.0.131793274646.issue28167@psf.upfronthosting.co.za>
Message-ID: <1483902292.32.0.859858743634.issue28167@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

But platform.dist() can be removed. It was deprecated in 2.6.

----------
nosy: +serhiy.storchaka

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28167>
_______________________________________

From report at bugs.python.org  Sun Jan  8 16:03:59 2017
From: report at bugs.python.org (Xavier de Gaye)
Date: Sun, 08 Jan 2017 21:03:59 +0000
Subject: [issue28759] access to mkfifo,
 mknod and hard links is controled by SELinux MAC on Android
In-Reply-To: <1479715975.06.0.930254771903.issue28759@psf.upfronthosting.co.za>
Message-ID: <1483909439.99.0.405488103137.issue28759@psf.upfronthosting.co.za>


Xavier de Gaye added the comment:

Re-opening the issue for a more generic change that skips the tests when PermissionError is raised instead of when they are run by an Android non-root user as this is done currently after the previously pushed changes.

----------
resolution: fixed -> 
stage: resolved -> patch review
status: closed -> open
Added file: http://bugs.python.org/file46218/without_android_not_root.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28759>
_______________________________________

From report at bugs.python.org  Sun Jan  8 16:14:25 2017
From: report at bugs.python.org (Andrew Dalke)
Date: Sun, 08 Jan 2017 21:14:25 +0000
Subject: [issue29211] assertRaises with exceptions re-raised from a generator
 kills generator
Message-ID: <1483910065.0.0.586195151306.issue29211@psf.upfronthosting.co.za>


New submission from Andrew Dalke:

The unittest assertRaises/assertRaisesRegex implementation calls traceback.clear_frames() because of issue9815 ("assertRaises as a context manager keeps tracebacks and frames alive").

However, if the traceback is from an exception created in a generator, caught, and re-raised outside of the generator, then the clear_frames() will cause the generator to raise a StopIteration exception the next time it is used.

Here is a reproducible where I create a generator and wrap it inside of an object API:

def simple_gen():
    yield 1, None
    try:
        1/0
    except ZeroDivisionError as err:
        yield None, err
    yield 3, None

class Spam:
    def __init__(self):
        self.gen = simple_gen()
    def get_next(self):
        value, err = next(self.gen)
        if err is not None:
            raise err
        return value

I can test this without unittest using the following:

def simple_test():
    spam = Spam()
    assert spam.get_next() == 1
    try:
        spam.get_next()
    except ZeroDivisionError:
        pass
    else:
        raise AssertionError
    assert spam.get_next() == 3
    print("simple test passed")

simple_test()


This prints "simple test passed", as expected.

The unittest implementation is simpler:

import unittest

class TestGen(unittest.TestCase):
    def test_gen(self):
        spam = Spam()
        self.assertEqual(spam.get_next(), 1)
        with self.assertRaises(ZeroDivisionError):
            spam.get_next()
        self.assertEqual(spam.get_next(), 3)

unittest.main()

but it reports an unexpected error:

======================================================================
ERROR: test_gen (__main__.TestGen)
----------------------------------------------------------------------
Traceback (most recent call last):
 File "clear.py", line 40, in test_gen
   self.assertEqual(spam.get_next(), 3)
 File "clear.py", line 13, in get_next
   value, err = next(self.gen)
StopIteration

----------------------------------------------------------------------
Ran 1 test in 0.000s

FAILED (errors=1)

I have tracked it down to the call to traceback.clear_frames(tb) in unittest/case.py. The following ClearFrames context manager will call traceback.clear_frames() if requested. The test code uses ClearFrames to demonstrate that the call to clear_frames() is what causes the unexpected StopIteration exception:


import traceback

class ClearFrames:
   def __init__(self, clear_frames):
       self.clear_frames = clear_frames
   def __enter__(self):
       return self

   def __exit__(self, exc_type, exc_value, tb):
       assert exc_type is ZeroDivisionError, exc_type
       if self.clear_frames:
           traceback.clear_frames(tb)  # This is the only difference between the tests.
       return True

# This is essentially the same test case as before, but structured using
# a context manager that either does or does not clear the traceback frames.
def clear_test(clear_frames):
    spam = Spam()
    assert spam.get_next() == 1
    with ClearFrames(clear_frames):
        spam.get_next()
    try:
        assert spam.get_next() == 3
    except StopIteration:
        print(" ... got StopIteration")
        return
    print(" ... clear_test passed")

print("\nDo not clear frames")
clear_test(False)
print("\nClear frames")
clear_test(True)


The output from this test is:

Do not clear frames
 ... clear_test passed

Clear frames
 ... got StopIteration

There are only a dozen or so tests in my code which are affected by this. (These are from a test suite which I am porting from 2.7 to 3.5.) I can easily re-write them to avoid using assertRaisesRegex.

I have no suggestion for a longer-term solution.

----------
components: Library (Lib)
messages: 285006
nosy: dalke
priority: normal
severity: normal
status: open
title: assertRaises with exceptions re-raised from a generator kills generator
type: behavior
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29211>
_______________________________________

From report at bugs.python.org  Sun Jan  8 16:20:02 2017
From: report at bugs.python.org (Martin Panter)
Date: Sun, 08 Jan 2017 21:20:02 +0000
Subject: [issue29208] BlockingIOError during system startup
In-Reply-To: <1483894212.45.0.948118557952.issue29208@psf.upfronthosting.co.za>
Message-ID: <1483910402.86.0.327009779042.issue29208@psf.upfronthosting.co.za>


Martin Panter added the comment:

It looks like the logic for handling an error seeding from urandom is reversed: <https://hg.python.org/cpython/rev/45fc0c83ed42#l6.66>. Random_seed_urandom() actually returns -1 if is an exception set, and 0 if it was successful.

The result would be a normal working urandom setup ignores urandom and seeds from the time and PID. Not a big deal, since the old code always seeded from a (lower resolution) time anyway. But if urandom failed, we get the SystemError.

The fix should be simple, in Modules/_randommodule.c change the random_seed() function to read

if (arg == NULL || arg == Py_None) {
    if (random_seed_urandom(self) < 0) { // was >= 0!
        PyErr_Clear();
        
        /* Reading system entropy failed, fall back on the worst entropy:
           use the current time and process identifier. */
        random_seed_time_pid(self);
    }
    Py_RETURN_NONE;
}

----------
keywords: +3.6regression
nosy: +haypo, martin.panter
type:  -> behavior

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29208>
_______________________________________

From report at bugs.python.org  Sun Jan  8 16:24:43 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sun, 08 Jan 2017 21:24:43 +0000
Subject: [issue28759] access to mkfifo,
 mknod and hard links is controled by SELinux MAC on Android
In-Reply-To: <1479715975.06.0.930254771903.issue28759@psf.upfronthosting.co.za>
Message-ID: <1483910683.26.0.909834077318.issue28759@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

I think it is worth to add special helper in test.support for using as a context manager:

    with helper():
        os.mkfifo(filename)

Then you could change its implementation without changing the testing code. For example:

    @contextmanager
    def helper():
        try:
            yield
        except PermissionError as e:
            raise unittest.SkipTest(str(e))

or

    @contextmanager
    def helper():
        if android_not_root:
            raise unittest.SkipTest("operation not allowed, non root user")
        yield

----------
nosy: +serhiy.storchaka

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28759>
_______________________________________

From report at bugs.python.org  Sun Jan  8 16:48:17 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sun, 08 Jan 2017 21:48:17 +0000
Subject: [issue29195] Get rid of supporting outdated wrong keyword arguments
 in re methods
In-Reply-To: <1483790693.93.0.397432841389.issue29195@psf.upfronthosting.co.za>
Message-ID: <1483912097.04.0.0519924853557.issue29195@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

"pattern" is not more informative, it is just wrong. The argument of the match() method is not a pattern, it is a string with which a pattern is matched.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29195>
_______________________________________

From report at bugs.python.org  Sun Jan  8 18:07:08 2017
From: report at bugs.python.org (David Vo)
Date: Sun, 08 Jan 2017 23:07:08 +0000
Subject: [issue29208] BlockingIOError during system startup
In-Reply-To: <1483894212.45.0.948118557952.issue29208@psf.upfronthosting.co.za>
Message-ID: <1483916828.12.0.953060485394.issue29208@psf.upfronthosting.co.za>


David Vo added the comment:

It looks like this was already fixed in issue 29085.

----------
nosy: +auscompgeek

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29208>
_______________________________________

From report at bugs.python.org  Sun Jan  8 18:12:35 2017
From: report at bugs.python.org (Dustin Spicuzza)
Date: Sun, 08 Jan 2017 23:12:35 +0000
Subject: [issue29208] BlockingIOError during system startup
In-Reply-To: <1483894212.45.0.948118557952.issue29208@psf.upfronthosting.co.za>
Message-ID: <1483917155.85.0.937363899252.issue29208@psf.upfronthosting.co.za>


Dustin Spicuzza added the comment:

I'll try applying that fix tomorrow and see if the issue is addressed.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29208>
_______________________________________

From report at bugs.python.org  Sun Jan  8 19:12:10 2017
From: report at bugs.python.org (Jelle Zijlstra)
Date: Mon, 09 Jan 2017 00:12:10 +0000
Subject: [issue29198] AsyncGenerator is missing from typing
In-Reply-To: <1483825061.03.0.644790904886.issue29198@psf.upfronthosting.co.za>
Message-ID: <1483920730.89.0.817299235058.issue29198@psf.upfronthosting.co.za>


Changes by Jelle Zijlstra <jelle.zijlstra at gmail.com>:


Added file: http://bugs.python.org/file46219/asyncgenerator3.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29198>
_______________________________________

From report at bugs.python.org  Sun Jan  8 19:12:30 2017
From: report at bugs.python.org (Jelle Zijlstra)
Date: Mon, 09 Jan 2017 00:12:30 +0000
Subject: [issue29198] AsyncGenerator is missing from typing
In-Reply-To: <1483825061.03.0.644790904886.issue29198@psf.upfronthosting.co.za>
Message-ID: <1483920750.25.0.623529381039.issue29198@psf.upfronthosting.co.za>


Changes by Jelle Zijlstra <jelle.zijlstra at gmail.com>:


Added file: http://bugs.python.org/file46220/typingindentation.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29198>
_______________________________________

From report at bugs.python.org  Sun Jan  8 19:14:06 2017
From: report at bugs.python.org (Martin Panter)
Date: Mon, 09 Jan 2017 00:14:06 +0000
Subject: [issue29208] BlockingIOError during system startup
In-Reply-To: <1483894212.45.0.948118557952.issue29208@psf.upfronthosting.co.za>
Message-ID: <1483920846.71.0.972108271946.issue29208@psf.upfronthosting.co.za>


Martin Panter added the comment:

David you are right, thanks. 

Dustin: the exact patch you want is revision 0a55e039d25f.

----------
resolution:  -> duplicate
status: open -> closed
superseder:  -> Python 3.6 on Windows doesn't seed Random() well enough

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29208>
_______________________________________

From report at bugs.python.org  Sun Jan  8 20:29:38 2017
From: report at bugs.python.org (Roundup Robot)
Date: Mon, 09 Jan 2017 01:29:38 +0000
Subject: [issue29203] With PEP 468,
 the lru cache not longer needs to sort keyword args
In-Reply-To: <1483869689.96.0.118801671139.issue29203@psf.upfronthosting.co.za>
Message-ID: <20170109012931.53514.78001.75E3BDF4@psf.io>


Roundup Robot added the comment:

New changeset 48c750c26710 by Raymond Hettinger in branch '3.6':
Issue #29203:  functools.lru_cache() now respects PEP 468
https://hg.python.org/cpython/rev/48c750c26710

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29203>
_______________________________________

From report at bugs.python.org  Sun Jan  8 21:04:38 2017
From: report at bugs.python.org (Roundup Robot)
Date: Mon, 09 Jan 2017 02:04:38 +0000
Subject: [issue29203] With PEP 468,
 the lru cache not longer needs to sort keyword args
In-Reply-To: <1483869689.96.0.118801671139.issue29203@psf.upfronthosting.co.za>
Message-ID: <20170109020435.52704.91529.C3CFC9B8@psf.io>


Roundup Robot added the comment:

New changeset cc47d385512b by Raymond Hettinger in branch 'default':
Complete the merge for issue #29203
https://hg.python.org/cpython/rev/cc47d385512b

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29203>
_______________________________________

From report at bugs.python.org  Sun Jan  8 21:17:37 2017
From: report at bugs.python.org (Nick Coghlan)
Date: Mon, 09 Jan 2017 02:17:37 +0000
Subject: [issue29206] importlib reload fails for module supplied as argument
 to command line
In-Reply-To: <1483875831.65.0.670865149709.issue29206@psf.upfronthosting.co.za>
Message-ID: <1483928257.06.0.84288970574.issue29206@psf.upfronthosting.co.za>


Nick Coghlan added the comment:

Since reload re-uses the existing namespace, having two names is less messy when they're just aliases for the same module object (it still has all the usual cache validity problems of any reload operation, but it doesn't have the extra challenges of two different module objects derived from the same source code).

As such, with -m, adding:

    import __main__
    sys.modules[__main__.__spec__.name] = __main__

is sufficient to get this working again, and will work for any of the PEP 451 based versions (i.e. 3.4+). (I'm open to an RFE to get runpy to do this by default in 3.7+ - it's an idea that has come up several times, and I think it will ultimately be less surprising than the current behaviour of allowing two entirely distinct copies of the module to be loaded)

The direct execution case is a bit different, as that's genuinely missing a __spec__ entry, and needs to be told how to reload itself:

    import __main__
    main_file = __main__.__file__
    main_name = os.path.splitext(os.path.basename(main_file))[0]
    __main__.__spec__ = importlib.util.spec_from_file_location(main_name, main_file)
    sys.modules[__main__.__spec__.name] = __main__

The two cases can be distinguished at runtime by whether or not __main__.__spec__ was already set.

Since this isn't really something we encourage people to do in general, but does remain possible for frameworks that want to support live reloading of __main__, I'm OK with requiring that extra framework-provided scaffolding in 3.4+.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29206>
_______________________________________

From report at bugs.python.org  Sun Jan  8 21:52:58 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Mon, 09 Jan 2017 02:52:58 +0000
Subject: [issue29142] urllib: no_proxy variable values with leading dot not
 properly handled
In-Reply-To: <1483452589.12.0.177615671636.issue29142@psf.upfronthosting.co.za>
Message-ID: <1483930378.26.0.499017836479.issue29142@psf.upfronthosting.co.za>


Xiang Zhang added the comment:

LGTM.

----------
assignee:  -> xiang.zhang
stage: patch review -> commit review

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29142>
_______________________________________

From report at bugs.python.org  Sun Jan  8 22:16:04 2017
From: report at bugs.python.org (Roundup Robot)
Date: Mon, 09 Jan 2017 03:16:04 +0000
Subject: [issue29145] failing overflow checks in replace_*
In-Reply-To: <1483468602.77.0.231507148809.issue29145@psf.upfronthosting.co.za>
Message-ID: <20170109031600.32475.1621.F7E01507@psf.io>


Roundup Robot added the comment:

New changeset 29e505526bdd by Xiang Zhang in branch '2.7':
Issue #29145: Fix overflow checks in string, bytearray and unicode.
https://hg.python.org/cpython/rev/29e505526bdd

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29145>
_______________________________________

From report at bugs.python.org  Sun Jan  8 22:33:34 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Mon, 09 Jan 2017 03:33:34 +0000
Subject: [issue29145] failing overflow checks in replace_*
In-Reply-To: <1483468602.77.0.231507148809.issue29145@psf.upfronthosting.co.za>
Message-ID: <1483932814.1.0.677181010825.issue29145@psf.upfronthosting.co.za>


Xiang Zhang added the comment:

I committed the patch to 2.7. The new patch is for 3.x.

----------
stage: patch review -> commit review
versions: +Python 3.5, Python 3.6, Python 3.7
Added file: http://bugs.python.org/file46221/overflow-checks-3x.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29145>
_______________________________________

From report at bugs.python.org  Sun Jan  8 22:57:42 2017
From: report at bugs.python.org (Roundup Robot)
Date: Mon, 09 Jan 2017 03:57:42 +0000
Subject: [issue29142] urllib: no_proxy variable values with leading dot not
 properly handled
In-Reply-To: <1483452589.12.0.177615671636.issue29142@psf.upfronthosting.co.za>
Message-ID: <20170109035739.52607.38450.0D93F3BD@psf.io>


Roundup Robot added the comment:

New changeset cb4f73be9486 by Xiang Zhang in branch '2.7':
Issue #29142: Fix suffixes in no_proxy handling in urllib.
https://hg.python.org/cpython/rev/cb4f73be9486

New changeset 350c5a1f9cfd by Xiang Zhang in branch '3.5':
Issue #29142: Fix suffixes in no_proxy handling in urllib.
https://hg.python.org/cpython/rev/350c5a1f9cfd

New changeset e90efd9f203e by Xiang Zhang in branch '3.6':
Issue #29142: Merge 3.5.
https://hg.python.org/cpython/rev/e90efd9f203e

New changeset 3954f8f6be3d by Xiang Zhang in branch 'default':
Issue #29142: Merge 3.6.
https://hg.python.org/cpython/rev/3954f8f6be3d

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29142>
_______________________________________

From report at bugs.python.org  Sun Jan  8 23:24:18 2017
From: report at bugs.python.org (Martin Panter)
Date: Mon, 09 Jan 2017 04:24:18 +0000
Subject: [issue1621] Do not assume signed integer overflow behavior
In-Reply-To: <1197593027.35.0.00314874350765.issue1621@psf.upfronthosting.co.za>
Message-ID: <1483935858.22.0.248416368401.issue1621@psf.upfronthosting.co.za>


Changes by Martin Panter <vadmium+py at gmail.com>:


----------
dependencies: +failing overflow checks in replace_*

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue1621>
_______________________________________

From report at bugs.python.org  Sun Jan  8 23:38:26 2017
From: report at bugs.python.org (Martin Panter)
Date: Mon, 09 Jan 2017 04:38:26 +0000
Subject: [issue29145] failing overflow checks in replace_*
In-Reply-To: <1483468602.77.0.231507148809.issue29145@psf.upfronthosting.co.za>
Message-ID: <1483936705.99.0.571806000195.issue29145@psf.upfronthosting.co.za>


Martin Panter added the comment:

FTR I thought the consensus was not to backport these fixes unless there was a demonstrated problem: <https://bugs.python.org/issue1621#msg144499>, though personally, I would be in favour of backporting in many cases.

Regarding str.join() in unicode.c, see also my unicode.patch in that bug. I included a test case that works on 32-bit platforms, which you may use. (From memory, you probably need to disable -fwrapv and maybe use -ftrapv or the undefined behaviour sanitizer to make the test fail.)

----------
nosy: +martin.panter

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29145>
_______________________________________

From report at bugs.python.org  Mon Jan  9 00:16:03 2017
From: report at bugs.python.org (desbma)
Date: Mon, 09 Jan 2017 05:16:03 +0000
Subject: [issue29212] Python 3.6 logging thread name regression with
 concurrent.future threads
Message-ID: <1483938963.09.0.315595404976.issue29212@psf.upfronthosting.co.za>


New submission from desbma:

Logging statement using 'threadName' from concurrent.futures threads produce the wrong output with Python 3.6.

The attached program with Python 3.5.X outputs:
MainThread From main thread
Thread-1 From worker thread

But with 3.6, it outputs:
MainThread From main thread
<concurrent.futures.thread.ThreadPoolExecutor object at 0x7f96711ac278>_0 From worker thread

----------
files: test.py
messages: 285021
nosy: desbma
priority: normal
severity: normal
status: open
title: Python 3.6 logging thread name regression with concurrent.future threads
type: behavior
versions: Python 3.6
Added file: http://bugs.python.org/file46222/test.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29212>
_______________________________________

From report at bugs.python.org  Mon Jan  9 00:36:34 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Mon, 09 Jan 2017 05:36:34 +0000
Subject: [issue29145] failing overflow checks in replace_*
In-Reply-To: <1483468602.77.0.231507148809.issue29145@psf.upfronthosting.co.za>
Message-ID: <1483940194.9.0.46866403255.issue29145@psf.upfronthosting.co.za>


Xiang Zhang added the comment:

> FTR I thought the consensus was not to backport these fixes unless there was a demonstrated problem: <https://bugs.python.org/issue1621#msg144499>, though personally, I would be in favour of backporting in many cases.

Sorry not know that. :-( Another benefit of the change is it actually backports the codes from 3.x so it's easy for later maintainence and it doesn't add additional work.

> I included a test case that works on 32-bit platforms, which you may use.

Thanks. But I don't get such an environment to test so I prefer not to add it now. I adjust my patch according to yours. :-)

----------
Added file: http://bugs.python.org/file46223/overflow-checks-3x-2.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29145>
_______________________________________

From report at bugs.python.org  Mon Jan  9 00:37:34 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Mon, 09 Jan 2017 05:37:34 +0000
Subject: [issue29142] urllib: no_proxy variable values with leading dot not
 properly handled
In-Reply-To: <1483452589.12.0.177615671636.issue29142@psf.upfronthosting.co.za>
Message-ID: <1483940254.89.0.0496921084986.issue29142@psf.upfronthosting.co.za>


Xiang Zhang added the comment:

Thanks Milan and Thomas!

----------
resolution:  -> fixed
stage: commit review -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29142>
_______________________________________

From report at bugs.python.org  Mon Jan  9 02:20:47 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Mon, 09 Jan 2017 07:20:47 +0000
Subject: [issue29200] is it a bug in `functools._HashedSeq`
In-Reply-To: <1483840932.91.0.717045884042.issue29200@psf.upfronthosting.co.za>
Message-ID: <1483946447.18.0.443678585866.issue29200@psf.upfronthosting.co.za>


Changes by Serhiy Storchaka <storchaka+cpython at gmail.com>:


----------
stage:  -> resolved

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29200>
_______________________________________

From report at bugs.python.org  Mon Jan  9 02:21:45 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Mon, 09 Jan 2017 07:21:45 +0000
Subject: [issue29203] With PEP 468,
 the lru cache not longer needs to sort keyword args
In-Reply-To: <1483869689.96.0.118801671139.issue29203@psf.upfronthosting.co.za>
Message-ID: <1483946505.51.0.0854356923613.issue29203@psf.upfronthosting.co.za>


Changes by Raymond Hettinger <raymond.hettinger at gmail.com>:


----------
resolution:  -> fixed
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29203>
_______________________________________

From report at bugs.python.org  Mon Jan  9 02:27:42 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Mon, 09 Jan 2017 07:27:42 +0000
Subject: [issue29203] With PEP 468,
 the lru cache not longer needs to sort keyword args
In-Reply-To: <1483869689.96.0.118801671139.issue29203@psf.upfronthosting.co.za>
Message-ID: <1483946862.87.0.403426263166.issue29203@psf.upfronthosting.co.za>


Changes by Serhiy Storchaka <storchaka+cpython at gmail.com>:


----------
stage: patch review -> resolved

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29203>
_______________________________________

From report at bugs.python.org  Mon Jan  9 02:31:55 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Mon, 09 Jan 2017 07:31:55 +0000
Subject: [issue28969] lru_cache is not threadsafe
In-Reply-To: <1481712564.52.0.530922236321.issue28969@psf.upfronthosting.co.za>
Message-ID: <1483947115.94.0.325836146612.issue28969@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

With Antoine's suggestion lru_cache-dict-pop-simpler-3.5.patch no longer has an advantage. Just ignore it.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28969>
_______________________________________

From report at bugs.python.org  Mon Jan  9 03:05:10 2017
From: report at bugs.python.org (desbma)
Date: Mon, 09 Jan 2017 08:05:10 +0000
Subject: [issue29212] Python 3.6 logging thread name regression with
 concurrent.future threads
In-Reply-To: <1483938963.09.0.315595404976.issue29212@psf.upfronthosting.co.za>
Message-ID: <1483949110.48.0.341689611827.issue29212@psf.upfronthosting.co.za>


desbma added the comment:

The bug seem to have been introduced by https://hg.python.org/cpython/rev/1002a1bdc5b1

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29212>
_______________________________________

From report at bugs.python.org  Mon Jan  9 03:10:41 2017
From: report at bugs.python.org (Roundup Robot)
Date: Mon, 09 Jan 2017 08:10:41 +0000
Subject: [issue29190] Avoid possible errors in comparing strings
In-Reply-To: <1483773006.32.0.276673146687.issue29190@psf.upfronthosting.co.za>
Message-ID: <20170109081038.7200.5723.5960AE43@psf.io>


Roundup Robot added the comment:

New changeset 337461574c90 by Serhiy Storchaka in branch '3.5':
Issue #29190: Fixed possible errors in comparing strings in the pickle module.
https://hg.python.org/cpython/rev/337461574c90

New changeset 9fcff936f61f by Serhiy Storchaka in branch '3.6':
Issue #29190: Fixed possible errors in comparing strings in the pickle module.
https://hg.python.org/cpython/rev/9fcff936f61f

New changeset f477c715076c by Serhiy Storchaka in branch 'default':
Issue #29190: Fixed possible errors in comparing strings in the pickle module.
https://hg.python.org/cpython/rev/f477c715076c

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29190>
_______________________________________

From report at bugs.python.org  Mon Jan  9 03:21:17 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Mon, 09 Jan 2017 08:21:17 +0000
Subject: [issue29190] Avoid possible errors in comparing strings
In-Reply-To: <1483773006.32.0.276673146687.issue29190@psf.upfronthosting.co.za>
Message-ID: <1483950077.65.0.887968476343.issue29190@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

In the particular case of getround() in _decimal.c, seems the worst case is raising TypeError instead of MemoryError in pretty rare circumstances. This is not critically bad, there are a lot of other places where the initial exception is silently replaced by less specific exception. But the code *looks* fragile.

----------
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29190>
_______________________________________

From report at bugs.python.org  Mon Jan  9 03:25:52 2017
From: report at bugs.python.org (desbma)
Date: Mon, 09 Jan 2017 08:25:52 +0000
Subject: [issue29212] Python 3.6 logging thread name regression with
 concurrent.future threads
In-Reply-To: <1483938963.09.0.315595404976.issue29212@psf.upfronthosting.co.za>
Message-ID: <1483950352.73.0.418792705289.issue29212@psf.upfronthosting.co.za>


desbma added the comment:

Here is a patch that restores the previous behavior and update test to catch bogus naming.

----------
keywords: +patch
Added file: http://bugs.python.org/file46224/issue29212_1.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29212>
_______________________________________

From report at bugs.python.org  Mon Jan  9 03:31:39 2017
From: report at bugs.python.org (desbma)
Date: Mon, 09 Jan 2017 08:31:39 +0000
Subject: [issue29212] Python 3.6 logging thread name regression with
 concurrent.future threads
In-Reply-To: <1483938963.09.0.315595404976.issue29212@psf.upfronthosting.co.za>
Message-ID: <1483950699.65.0.509252675127.issue29212@psf.upfronthosting.co.za>


Changes by desbma <dutch109 at gmail.com>:


----------
components: +Library (Lib)
versions: +Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29212>
_______________________________________

From report at bugs.python.org  Mon Jan  9 04:42:45 2017
From: report at bugs.python.org (Stefan Krah)
Date: Mon, 09 Jan 2017 09:42:45 +0000
Subject: [issue29190] Avoid possible errors in comparing strings
In-Reply-To: <1483950077.65.0.887968476343.issue29190@psf.upfronthosting.co.za>
Message-ID: <20170109094230.GA2738@bytereef.org>


Stefan Krah added the comment:

On Mon, Jan 09, 2017 at 08:21:17AM +0000, Serhiy Storchaka wrote:
> In the particular case of getround() in _decimal.c, seems the worst case is raising TypeError instead of MemoryError in pretty rare circumstances. This is not critically bad, there are a lot of other places where the initial exception is silently replaced by less specific exception. But the code *looks* fragile.

No, it does not.  It is obvious to a human that -1 <==> "not equal".

If Coccinelle does not understand that, well ...

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29190>
_______________________________________

From report at bugs.python.org  Mon Jan  9 04:59:52 2017
From: report at bugs.python.org (STINNER Victor)
Date: Mon, 09 Jan 2017 09:59:52 +0000
Subject: [issue28167] remove platform.linux_distribution()
In-Reply-To: <1473933007.75.0.131793274646.issue28167@psf.upfronthosting.co.za>
Message-ID: <1483955992.93.0.334755144505.issue28167@psf.upfronthosting.co.za>


STINNER Victor added the comment:

+1 to remove platform.linux_distribution().

Linux distributions are moving much faster than the Python stdlib, so using the distro module which is hosted on PyPI is much simpler. In 2017, it became very easy to have dependencies, pip became the de factor standard.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28167>
_______________________________________

From report at bugs.python.org  Mon Jan  9 05:08:17 2017
From: report at bugs.python.org (STINNER Victor)
Date: Mon, 09 Jan 2017 10:08:17 +0000
Subject: [issue29085] Python 3.6 on Windows doesn't seed Random() well enough
In-Reply-To: <1482860364.41.0.203895591313.issue29085@psf.upfronthosting.co.za>
Message-ID: <1483956497.31.0.924897863191.issue29085@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Oh, sorry, I introduced the bug in Python 3.6 with the PEP 524 (os.urandom() now blocks on Linux). Too bad that there is no simple way to write an unit test for that.

> ... but only on Windows on Python 3.6

With the bug, or when the fix when _PyOS_URandomNonblock() fails, Random.seed() uses:

* Reading system entropy failed, fall back on the worst entropy:
  use the current time and process identifier. */
random_seed_time_pid(self);

It's just that on Windows, the system clock has a resolution around 15 ms, whereas it has a resolution better than 1 us on Linux. So it's just that calling Random.seed() usually takes longer than the resolution of the system clock on Linux :-) Not really that the bug is specific to Windows.

Thanks for the fix Benjamin!

----------
nosy: +haypo

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29085>
_______________________________________

From report at bugs.python.org  Mon Jan  9 05:22:23 2017
From: report at bugs.python.org (Roundup Robot)
Date: Mon, 09 Jan 2017 10:22:23 +0000
Subject: [issue29157] random.c: Prefer getrandom() over getentropy() to
 support glibc 2.24 on Linux
In-Reply-To: <1483551181.22.0.980001779001.issue29157@psf.upfronthosting.co.za>
Message-ID: <20170109102220.53365.40744.5D10FC9B@psf.io>


Roundup Robot added the comment:

New changeset 8125d9a8152b by Victor Stinner in branch '3.5':
Issue #29157: Prefer getrandom() over getentropy()
https://hg.python.org/cpython/rev/8125d9a8152b

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29157>
_______________________________________

From report at bugs.python.org  Mon Jan  9 05:22:23 2017
From: report at bugs.python.org (Roundup Robot)
Date: Mon, 09 Jan 2017 10:22:23 +0000
Subject: [issue29188] Backport random.c from Python 3.5 to Python 2.7
In-Reply-To: <1483746834.46.0.642378238515.issue29188@psf.upfronthosting.co.za>
Message-ID: <20170109102220.53365.64890.BE91F38A@psf.io>


Roundup Robot added the comment:

New changeset 13a39142c047 by Victor Stinner in branch '2.7':
Don't use getentropy() on Linux
https://hg.python.org/cpython/rev/13a39142c047

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29188>
_______________________________________

From report at bugs.python.org  Mon Jan  9 05:26:37 2017
From: report at bugs.python.org (STINNER Victor)
Date: Mon, 09 Jan 2017 10:26:37 +0000
Subject: [issue29188] Backport random.c from Python 3.5 to Python 2.7
In-Reply-To: <1483746834.46.0.642378238515.issue29188@psf.upfronthosting.co.za>
Message-ID: <1483957597.6.0.19653376891.issue29188@psf.upfronthosting.co.za>


STINNER Victor added the comment:

> This is VERY far from our historical policy for backports.  Python 2.7 is supposed to be getting more stable over time (that is one of its chief virtues).  We don't want to risk the kind of mini-catastrophe that got published in 3.6 (issue29085).

I don't consider that the issue #29085 is a catastrophe and it's just a bug which was already fixed.

Moreover, Python 2.7 and 3.5 don't have _PyOS_URandomNonblock() function and so the _random module is not impacted by this issue.


> If you want to push for this, there needs to be a thorough discussion on python-dev (there are tons of possible backports that could be made if the rationale was "I would prefer to use the same code on all maintained versions").

Sorry, I suffered from the previous discussion about random numbers. I don't want to reopen a new discussion, people would become crazy again.

I just fixed Python/random.c in support glibc 2.24 that's all.

If someone wants the cool getrandom() function/syscall on Python 2.7, please open a new issue. It doesn't really enhance the security, it's just a matter of avoid a file descriptor.

----------
resolution:  -> fixed
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29188>
_______________________________________

From report at bugs.python.org  Mon Jan  9 05:33:32 2017
From: report at bugs.python.org (STINNER Victor)
Date: Mon, 09 Jan 2017 10:33:32 +0000
Subject: [issue29157] random.c: Prefer getrandom() over getentropy() to
 support glibc 2.24 on Linux
In-Reply-To: <1483551181.22.0.980001779001.issue29157@psf.upfronthosting.co.za>
Message-ID: <1483958012.04.0.699000177232.issue29157@psf.upfronthosting.co.za>


STINNER Victor added the comment:

> It seems like Python 3.5 is close to a release, I prefer to wait after the release to fix this issue. (...) @Larry: Do you want this change in Python 3.5.3? The change is quite large.

Ah, I was wrong: 3.5 was already open for the next 3.5.4 release, so I pushed my change. Again, I don't think that supporting the glibc 2.24 in Python 3.5 is a bug important enough to post-pone a release.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29157>
_______________________________________

From report at bugs.python.org  Mon Jan  9 05:56:13 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Mon, 09 Jan 2017 10:56:13 +0000
Subject: [issue28870] Reduce stack consumption of
 PyObject_CallFunctionObjArgs() and like
In-Reply-To: <1480891813.94.0.54795440575.issue28870@psf.upfronthosting.co.za>
Message-ID: <1483959373.07.0.0519606247878.issue28870@psf.upfronthosting.co.za>


Changes by Xiang Zhang <angwerzx at 126.com>:


----------
nosy: +xiang.zhang

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28870>
_______________________________________

From report at bugs.python.org  Mon Jan  9 06:19:05 2017
From: report at bugs.python.org (Marco Buttu)
Date: Mon, 09 Jan 2017 11:19:05 +0000
Subject: [issue29133] Minor inaccuracy in shlex.shlex punctuation_chars example
In-Reply-To: <1483362012.78.0.704413629859.issue29133@psf.upfronthosting.co.za>
Message-ID: <1483960745.59.0.662363618907.issue29133@psf.upfronthosting.co.za>


Marco Buttu added the comment:

Here is a 3rd patch with the Berker's suggestion. I just limited the output to 79 characters per line, and made use of the +NORMALIZE_WHITESPACE option  (to normalize the newline inside the output).

----------
Added file: http://bugs.python.org/file46225/issue29133_3rd.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29133>
_______________________________________

From report at bugs.python.org  Mon Jan  9 07:16:31 2017
From: report at bugs.python.org (Roundup Robot)
Date: Mon, 09 Jan 2017 12:16:31 +0000
Subject: [issue28701] Replace PyUnicode_CompareWithASCIIString with
 _PyUnicode_EqualToASCIIString
In-Reply-To: <1479239405.55.0.31899621759.issue28701@psf.upfronthosting.co.za>
Message-ID: <20170109121628.19960.14281.ECD45EF4@psf.io>


Roundup Robot added the comment:

New changeset 35334a4d41aa by Stefan Krah in branch '3.5':
Issue #28701: Revert part of 5bdc8e1a50c8 for the following reasons:
https://hg.python.org/cpython/rev/35334a4d41aa

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28701>
_______________________________________

From report at bugs.python.org  Mon Jan  9 07:27:20 2017
From: report at bugs.python.org (Cornelius Diekmann)
Date: Mon, 09 Jan 2017 12:27:20 +0000
Subject: [issue29070] Integration tests for pty.spawn on Linux and all other
 platforms
In-Reply-To: <1482684225.17.0.467768789806.issue29070@psf.upfronthosting.co.za>
Message-ID: <1483964840.38.0.955935171615.issue29070@psf.upfronthosting.co.za>


Cornelius Diekmann added the comment:

Dear Martin, now I understand your intention. I merged my test suite with Chris's fix and documented our insights. SmallPtyTests contains regression tests for this issue.

While testing, I found a subtle change in behavior introduced by Chris's patch: It is no longer possible to get a broken pipe easily. I made a small adjustment to preserve this existing behavior:  In the _copy loop, I changed the order in which master_fd and STDIN_FILENO are copied to preserve the BrokenPipeError.


I tuned the slow tests, the complete test suite now needs less than 1 second on my system. I updated the documentation to note that the module is no longer supposed to be Linux-only.


> I realized that PtyWhiteBoxIntegrationTermiosTest is a reasonable test for the termios module, so it is beneficial. Though it may have been simpler to write it using pty.openpty(), avoiding an extra thread that just copies data between other threads.

I agree completely. However, pty.openpty() does not depend on the _copy() loop which is modified by Chris's patch. To add higher test coverage for the changes we introduce by merging Chris's fix, I decided to stick to pty.spawn().


I tested on Linux 4.4, Linux 3.13, FreeBSD 11, MacOS 10.11.6 and all tests are green. If anything goes wrong on the testbot army, here is the fallback strategy:
[I don't expect anything to go wrong, but I'm rather prepared than sorry]
If the test_pty does not finish within one minute on a system, please kill it. It would be nice to know on which platform it failed, so we can whitelist the module in the tests and the documentation for only the supported platforms.

----------
Added file: http://bugs.python.org/file46226/pty.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29070>
_______________________________________

From report at bugs.python.org  Mon Jan  9 07:46:28 2017
From: report at bugs.python.org (Masayuki Yamamoto)
Date: Mon, 09 Jan 2017 12:46:28 +0000
Subject: [issue25658] PyThread assumes pthread_key_t is an integer,
 which is against POSIX
In-Reply-To: <1447861816.88.0.632584646231.issue25658@psf.upfronthosting.co.za>
Message-ID: <1483965988.68.0.219523980196.issue25658@psf.upfronthosting.co.za>


Masayuki Yamamoto added the comment:

After Erik posted PEP 539 draft, we've discussed features of new API on the Python-ideas [*]. As the result of discussion, features of new API have been changed below points. Thus, I updated patch based on the result.

1. API uses opaque type because to cover the key details.
2. The key has the state that is whether initialized, and a function is added to check the state.
3. When key creation and deletion, API silently skips function in unnecessary case (i.e. valid key never be overwritten).
4. The test is added to check the key state that after calling API.

[*] https://mail.python.org/pipermail/python-ideas/2016-December/043983.html

----------
Added file: http://bugs.python.org/file46227/pythread-tss-2.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue25658>
_______________________________________

From report at bugs.python.org  Mon Jan  9 07:52:45 2017
From: report at bugs.python.org (Xavier de Gaye)
Date: Mon, 09 Jan 2017 12:52:45 +0000
Subject: [issue28759] access to mkfifo,
 mknod and hard links is controled by SELinux MAC on Android
In-Reply-To: <1479715975.06.0.930254771903.issue28759@psf.upfronthosting.co.za>
Message-ID: <1483966365.45.0.517834159092.issue28759@psf.upfronthosting.co.za>


Xavier de Gaye added the comment:

Thanks for this interesting suggestion Serhiy. I propose to use supported_operation() as the name of the helper() generator.

I will update the patches accordingly in issues #29180, #29181 and #29184, and re-open issue 28764.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28759>
_______________________________________

From report at bugs.python.org  Mon Jan  9 07:59:08 2017
From: report at bugs.python.org (Xavier de Gaye)
Date: Mon, 09 Jan 2017 12:59:08 +0000
Subject: [issue28759] access to mkfifo,
 mknod and hard links is controled by SELinux MAC on Android
In-Reply-To: <1479715975.06.0.930254771903.issue28759@psf.upfronthosting.co.za>
Message-ID: <1483966748.72.0.250783416584.issue28759@psf.upfronthosting.co.za>


Xavier de Gaye added the comment:

> re-open issue 28764.

Oops, no issue 28764 is not relevant here.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28759>
_______________________________________

From report at bugs.python.org  Mon Jan  9 08:34:34 2017
From: report at bugs.python.org (STINNER Victor)
Date: Mon, 09 Jan 2017 13:34:34 +0000
Subject: [issue29070] Integration tests for pty.spawn on Linux and all other
 platforms
In-Reply-To: <1482684225.17.0.467768789806.issue29070@psf.upfronthosting.co.za>
Message-ID: <1483968874.02.0.464078250435.issue29070@psf.upfronthosting.co.za>


Changes by STINNER Victor <victor.stinner at gmail.com>:


----------
nosy:  -haypo

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29070>
_______________________________________

From report at bugs.python.org  Mon Jan  9 08:40:51 2017
From: report at bugs.python.org (=?utf-8?b?0JrQvtC90YHRgtCw0L3RgtC40L0g0JLQvtC70LrQvtCy?=)
Date: Mon, 09 Jan 2017 13:40:51 +0000
Subject: [issue27931] Email parse IndexError <""@wiarcom.com>
In-Reply-To: <1472743779.07.0.918992764498.issue27931@psf.upfronthosting.co.za>
Message-ID: <1483969251.88.0.708866638466.issue27931@psf.upfronthosting.co.za>


?????????? ?????? added the comment:

ping

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue27931>
_______________________________________

From report at bugs.python.org  Mon Jan  9 09:12:18 2017
From: report at bugs.python.org (Josh Wilson)
Date: Mon, 09 Jan 2017 14:12:18 +0000
Subject: [issue29213] python -m venv activate.bat has weird mix of line endings
Message-ID: <1483971138.02.0.292455070123.issue29213@psf.upfronthosting.co.za>


New submission from Josh Wilson:

The activate.bat file generated by python -m venv somevirtualenv seems to have a mix of line ending styles.  Sometimes using Carriage Return (CR) and Line Feed (LF) and other times using only CR.

This seems to cause unexpected behavior when trying to modify the file to set additional environment variables.  Since this is a Windows-specific file it seems like it should use Windows-style line endings of CRLF

Deactivate.bat has the same issue.

----------
components: Windows
messages: 285043
nosy: Josh Wilson, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: python -m venv activate.bat has weird mix of line endings
type: behavior
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29213>
_______________________________________

From report at bugs.python.org  Mon Jan  9 09:17:01 2017
From: report at bugs.python.org (=?utf-8?b?0JzQsNGA0Log0JrQvtGA0LXQvdCx0LXRgNCz?=)
Date: Mon, 09 Jan 2017 14:17:01 +0000
Subject: [issue29214] Standard open() does not allow to specify file
 permissions.
Message-ID: <1483971421.1.0.906896747438.issue29214@psf.upfronthosting.co.za>


New submission from ???? ?????????:

1. Syscall open() allows that.
2. This is important to prevent race-conditions between creating a file with default permissions and calling fchmod().

----------
components: Library (Lib)
messages: 285044
nosy: mmarkk
priority: normal
severity: normal
status: open
title: Standard open() does not allow to specify file permissions.
versions: Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29214>
_______________________________________

From report at bugs.python.org  Mon Jan  9 09:17:34 2017
From: report at bugs.python.org (=?utf-8?b?0JzQsNGA0Log0JrQvtGA0LXQvdCx0LXRgNCz?=)
Date: Mon, 09 Jan 2017 14:17:34 +0000
Subject: [issue29214] Standard open() does not allow to specify file
 permissions.
In-Reply-To: <1483971421.1.0.906896747438.issue29214@psf.upfronthosting.co.za>
Message-ID: <1483971454.33.0.586492632491.issue29214@psf.upfronthosting.co.za>


Changes by ???? ????????? <socketpair at gmail.com>:


----------
type:  -> behavior

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29214>
_______________________________________

From report at bugs.python.org  Mon Jan  9 09:17:44 2017
From: report at bugs.python.org (=?utf-8?b?0JzQsNGA0Log0JrQvtGA0LXQvdCx0LXRgNCz?=)
Date: Mon, 09 Jan 2017 14:17:44 +0000
Subject: [issue29214] Standard open() does not allow to specify file
 permissions.
In-Reply-To: <1483971421.1.0.906896747438.issue29214@psf.upfronthosting.co.za>
Message-ID: <1483971464.8.0.645395744419.issue29214@psf.upfronthosting.co.za>


Changes by ???? ????????? <socketpair at gmail.com>:


----------
versions: +Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29214>
_______________________________________

From report at bugs.python.org  Mon Jan  9 09:28:49 2017
From: report at bugs.python.org (Sander Vrijders)
Date: Mon, 09 Jan 2017 14:28:49 +0000
Subject: [issue29215] pyport.h uses non C90-style comment
Message-ID: <1483972129.08.0.0486776479108.issue29215@psf.upfronthosting.co.za>


New submission from Sander Vrijders:

I am writing C90 compliant code and am wrapping my C headers with SWIG, which includes python3.6m/pyport.h. I've told my compiler to report all warnings as errors. Compilation fails with the following error:

```
/usr/include/python3.6m/pyport.h:40:1: error: C++ style comments are not allowed in ISO C90
 // long long is required. Ensure HAVE_LONG_LONG is defined for compatibility.
```

I believe the comment was added by the patch of a previous bug (see https://hg.python.org/cpython/rev/4745d801cae2). The solution would be to change this comment to a C90 comment.

----------
messages: 285045
nosy: Sander Vrijders
priority: normal
severity: normal
status: open
title: pyport.h uses non C90-style comment
type: compile error
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29215>
_______________________________________

From report at bugs.python.org  Mon Jan  9 09:30:57 2017
From: report at bugs.python.org (Christian Heimes)
Date: Mon, 09 Jan 2017 14:30:57 +0000
Subject: [issue29214] Standard open() does not allow to specify file
 permissions.
In-Reply-To: <1483971421.1.0.906896747438.issue29214@psf.upfronthosting.co.za>
Message-ID: <1483972257.37.0.911405761534.issue29214@psf.upfronthosting.co.za>


Changes by Christian Heimes <lists at cheimes.de>:


----------
nosy: +christian.heimes

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29214>
_______________________________________

From report at bugs.python.org  Mon Jan  9 09:39:20 2017
From: report at bugs.python.org (STINNER Victor)
Date: Mon, 09 Jan 2017 14:39:20 +0000
Subject: [issue29214] Standard open() does not allow to specify file
 permissions.
In-Reply-To: <1483971421.1.0.906896747438.issue29214@psf.upfronthosting.co.za>
Message-ID: <1483972760.57.0.588557521728.issue29214@psf.upfronthosting.co.za>


STINNER Victor added the comment:

You can actually specify permission:

fd = os.open("document.txt", os.O_WRONLY | os.O_CREAT, 0o777)
fp = open(fd, "wb")
with fp:
   ...

You can also use the dir_fd parameter for even better security ;-)

Is your request for the standard open() function, rather than os.open()?

Note: io.FileIO(), the underlying object when calling the Python open(), calls the C open() function with 0o666 for the permission mode.

----------
nosy: +haypo

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29214>
_______________________________________

From report at bugs.python.org  Mon Jan  9 09:46:11 2017
From: report at bugs.python.org (Christian Heimes)
Date: Mon, 09 Jan 2017 14:46:11 +0000
Subject: [issue29214] Standard open() does not allow to specify file
 permissions.
In-Reply-To: <1483971421.1.0.906896747438.issue29214@psf.upfronthosting.co.za>
Message-ID: <1483973171.63.0.296799603211.issue29214@psf.upfronthosting.co.za>


Christian Heimes added the comment:

I would be nice to have a public API to set permissions of files for io.open(), too.

Note to your note: the actual permission is influenced by the process' global umask. If you are concerned about security, you should set the umask to 0o077 early.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29214>
_______________________________________

From report at bugs.python.org  Mon Jan  9 10:47:44 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Mon, 09 Jan 2017 15:47:44 +0000
Subject: [issue29215] pyport.h uses non C90-style comment
In-Reply-To: <1483972129.08.0.0486776479108.issue29215@psf.upfronthosting.co.za>
Message-ID: <1483976864.34.0.533010478692.issue29215@psf.upfronthosting.co.za>


Xiang Zhang added the comment:

That's a change in 3.6, see https://docs.python.org/3/whatsnew/3.6.html#build-and-c-api-changes.

----------
nosy: +xiang.zhang

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29215>
_______________________________________

From report at bugs.python.org  Mon Jan  9 11:11:42 2017
From: report at bugs.python.org (Sander Vrijders)
Date: Mon, 09 Jan 2017 16:11:42 +0000
Subject: [issue29215] pyport.h uses non C90-style comment
In-Reply-To: <1483972129.08.0.0486776479108.issue29215@psf.upfronthosting.co.za>
Message-ID: <1483978302.17.0.03927567159.issue29215@psf.upfronthosting.co.za>


Sander Vrijders added the comment:

Okay, a bit conflicted about the following though. The changelog refers to PEP7. This document indeed states that Python versions greater than or equal to 3.6 use C89 with several select C99 features, such as C++-style line comments. But then below that it has a rule that says never use C++ style // one-line comments. That seems contradictory.

FYI, I mean not complaining about stdint.h, gcc (and clang) also support those with the C90 switch turned on. 

Or should I just assume that all headers are henceforth C99 and any wrapping code should also be C99? Though it seems a bit overkill to kill all compatibility for a single comment line. If I use a C89 comment on my system the build works perfectly.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29215>
_______________________________________

From report at bugs.python.org  Mon Jan  9 11:21:36 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Mon, 09 Jan 2017 16:21:36 +0000
Subject: [issue29215] pyport.h uses non C90-style comment
In-Reply-To: <1483972129.08.0.0486776479108.issue29215@psf.upfronthosting.co.za>
Message-ID: <1483978896.76.0.750591409802.issue29215@psf.upfronthosting.co.za>


Xiang Zhang added the comment:

I found that conflict once but lost it then. :-(

> Or should I just assume that all headers are henceforth C99 and any wrapping code should also be C99?

I am not sure. :-( Nosy other experts. :-)

----------
nosy: +haypo, serhiy.storchaka

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29215>
_______________________________________

From report at bugs.python.org  Mon Jan  9 11:31:16 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Mon, 09 Jan 2017 16:31:16 +0000
Subject: [issue29215] pyport.h uses non C90-style comment
In-Reply-To: <1483972129.08.0.0486776479108.issue29215@psf.upfronthosting.co.za>
Message-ID: <1483979476.62.0.386158812726.issue29215@psf.upfronthosting.co.za>


Changes by Serhiy Storchaka <storchaka+cpython at gmail.com>:


----------
nosy: +benjamin.peterson

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29215>
_______________________________________

From report at bugs.python.org  Mon Jan  9 11:38:22 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Mon, 09 Jan 2017 16:38:22 +0000
Subject: [issue29216] Space saving step for the LRU cache
Message-ID: <1483979902.02.0.507353430639.issue29216@psf.upfronthosting.co.za>


New submission from Raymond Hettinger:

Sync with the space savings optimization in the pure Python code.

I'm unsure whether the best way is to check for an exact tuple or whether to just check an exact int/str.

----------
assignee: serhiy.storchaka
components: Library (Lib)
files: functools_sync.diff
keywords: patch
messages: 285051
nosy: rhettinger, serhiy.storchaka
priority: low
severity: normal
stage: patch review
status: open
title: Space saving step for the LRU cache
type: resource usage
versions: Python 3.7
Added file: http://bugs.python.org/file46228/functools_sync.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29216>
_______________________________________

From report at bugs.python.org  Mon Jan  9 11:40:05 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Mon, 09 Jan 2017 16:40:05 +0000
Subject: [issue29216] Space saving step for the LRU cache
In-Reply-To: <1483979902.02.0.507353430639.issue29216@psf.upfronthosting.co.za>
Message-ID: <1483980005.91.0.206498138662.issue29216@psf.upfronthosting.co.za>


Changes by Raymond Hettinger <raymond.hettinger at gmail.com>:


Added file: http://bugs.python.org/file46229/functools_sync2.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29216>
_______________________________________

From report at bugs.python.org  Mon Jan  9 11:48:44 2017
From: report at bugs.python.org (Roundup Robot)
Date: Mon, 09 Jan 2017 16:48:44 +0000
Subject: [issue29133] Minor inaccuracy in shlex.shlex punctuation_chars example
In-Reply-To: <1483362012.78.0.704413629859.issue29133@psf.upfronthosting.co.za>
Message-ID: <20170109164838.93968.59336.1FF98EC5@psf.io>


Roundup Robot added the comment:

New changeset af5b34b2d169 by Vinay Sajip in branch '3.6':
Fixes #29133: clarified shlex documentation.
https://hg.python.org/cpython/rev/af5b34b2d169

New changeset e3d820c0c884 by Vinay Sajip in branch 'default':
Closes #29133: merged update from 3.6.
https://hg.python.org/cpython/rev/e3d820c0c884

----------
nosy: +python-dev
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29133>
_______________________________________

From report at bugs.python.org  Mon Jan  9 11:55:38 2017
From: report at bugs.python.org (Roundup Robot)
Date: Mon, 09 Jan 2017 16:55:38 +0000
Subject: [issue29177] skip tests of test_logging when bind() raises
 PermissionError (non-root user on Android)
In-Reply-To: <1483701262.25.0.0295401910206.issue29177@psf.upfronthosting.co.za>
Message-ID: <20170109165531.52790.86282.48BB230A@psf.io>


Roundup Robot added the comment:

New changeset 6bf563472ccd by Vinay Sajip in branch '3.6':
Fixes #29177: Improved resilience of logging tests which use socket servers.
https://hg.python.org/cpython/rev/6bf563472ccd

New changeset 3f324d5df0c0 by Vinay Sajip in branch 'default':
Closes #29177: Merged fix from 3.6.
https://hg.python.org/cpython/rev/3f324d5df0c0

----------
nosy: +python-dev
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29177>
_______________________________________

From report at bugs.python.org  Mon Jan  9 12:01:34 2017
From: report at bugs.python.org (=?utf-8?b?0JzQsNGA0Log0JrQvtGA0LXQvdCx0LXRgNCz?=)
Date: Mon, 09 Jan 2017 17:01:34 +0000
Subject: [issue29214] Standard open() does not allow to specify file
 permissions.
In-Reply-To: <1483971421.1.0.906896747438.issue29214@psf.upfronthosting.co.za>
Message-ID: <1483981294.15.0.609192464056.issue29214@psf.upfronthosting.co.za>


???? ????????? added the comment:

1. Yes, I'm reporting problem about standard open(), not os.open().
2. Yes, I know about umask. But unfortunatelly, umask affects whole process, including all threads, so it is not possible to force specific permissions without of race conditions with other threds.
3. Calling C's `open()` with `0666` MUST be documented (with mentioning of umask, which also affects these bits). This is `MUST` because `man 2 open` says that there is no default for `mode` argument, and it must be specified when O_CREAT / O_TMPFILE is specified.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29214>
_______________________________________

From report at bugs.python.org  Mon Jan  9 12:10:23 2017
From: report at bugs.python.org (STINNER Victor)
Date: Mon, 09 Jan 2017 17:10:23 +0000
Subject: [issue28870] Reduce stack consumption of
 PyObject_CallFunctionObjArgs() and like
In-Reply-To: <1480891813.94.0.54795440575.issue28870@psf.upfronthosting.co.za>
Message-ID: <1483981823.01.0.318409235293.issue28870@psf.upfronthosting.co.za>


STINNER Victor added the comment:

I modified Serhiy's stack_overflow.py of #28858:
* re-run each test 10 tests and show the maximum depth
* only test: ['test_python_call', 'test_python_getitem', 'test_python_iterator']

Maximum number of Python calls before a crash.

(*) Reference (unpatched): 560 bytes/call

test_python_call 7172
test_python_getitem 6232
test_python_iterator 5344
=> total: 18 838

(1) no_small_stack.patch: 368 bytes/call

test_python_call 7172 (=)
test_python_getitem 6544 (+312)
test_python_iterator 5572 (+228)
=> total: 19 288

(2) less_stack.patch: 384 bytes/call

test_python_call 7272 (+100)
test_python_getitem 6384 (+152)
test_python_iterator 5456 (+112)
=> total: 19 112

(3) subfunc.patch: 496 bytes

test_python_call 7272 (+100)
test_python_getitem 6712 (+480)
test_python_iterator 6020 (+678)
=> total: 20 004

(4) alloca.patch: 528 bytes/call

test_python_call 7272 (+100)
test_python_getitem 6464 (+232)
test_python_iterator 5752 (+408)
=> total: 19 488

Patched sorted by bytes/call, from best to worst: no_small_stack.patch (368) > less_stack.patch (384) > subfunc.patch (496) > alloca.patch (528) > reference (560).

Patched sorted by number of calls before crash: subfunc.patch (20 004) > alloca.patch (19 488) > no_small_stack.patch (19 288) > less_stack.patch (19 112) > reference (18 838).

I expected a correlation between the measure bytes/call measured by testcapi_stacksize.patch and the number of calls before a crash, but I fail to see an obvious correlation :-/

Maybe the compiler is smarter than what I would expect and emits efficient code to be able to use less stack memory?

Maybe the Linux kernel does weird things which makes the behaviour on stack-overflow non-obvious :-)

At least, I would expect that no_small_stack.patch would be the clear winner, since it has the smallest usage of C stack.

----------
Added file: http://bugs.python.org/file46230/stack_overflow_28870.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28870>
_______________________________________

From report at bugs.python.org  Mon Jan  9 12:13:57 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Mon, 09 Jan 2017 17:13:57 +0000
Subject: [issue29216] Space saving step for the LRU cache
In-Reply-To: <1483979902.02.0.507353430639.issue29216@psf.upfronthosting.co.za>
Message-ID: <1483982037.43.0.625807575084.issue29216@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

The first patch fails with e.g. tuple subclasses.

The optimization is correct only for types not comparable with tuples. Tuple subclasses are comparable, and classes with custom __eq__ can be comparable. Thus the optimization should be used only for exact builtin types that are known not comparable with tuples. Of course it makes sense only for hashable types.

First than commit something like the second patch I would gather statistics. What are the most used types of the single positional argument of lru-cached functions? Is it worth to apply the optimization for bytes or None? Checking type in C is cheaper than in Python, we can use more types.

----------
assignee: serhiy.storchaka -> rhettinger

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29216>
_______________________________________

From report at bugs.python.org  Mon Jan  9 12:28:49 2017
From: report at bugs.python.org (STINNER Victor)
Date: Mon, 09 Jan 2017 17:28:49 +0000
Subject: [issue28870] Reduce stack consumption of
 PyObject_CallFunctionObjArgs() and like
In-Reply-To: <1480891813.94.0.54795440575.issue28870@psf.upfronthosting.co.za>
Message-ID: <1483982929.57.0.104679788872.issue28870@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Impact of the _PY_FASTCALL_SMALL_STACK constant:

* _PY_FASTCALL_SMALL_STACK=1: 528 bytes/call

test_python_call 7376
test_python_getitem 6544
test_python_iterator 5572
=> total: 19 492

* _PY_FASTCALL_SMALL_STACK=3: 528 bytes/call

test_python_call 7272
test_python_getitem 6464
test_python_iterator 5512
=> total: 19 248

* _PY_FASTCALL_SMALL_STACK=5 (current value): 560 bytes/call

test_python_call 7172
test_python_getitem 6232
test_python_iterator 5344
=> total: 19 636

* _PY_FASTCALL_SMALL_STACK=10: 592 bytes/call

test_python_call 6984
test_python_getitem 5952
test_python_iterator 5132
=> total: 18 068

Increasing _PY_FASTCALL_SMALL_STACK has a clear effect on the total. Total decreases when _PY_FASTCALL_SMALL_STACK increases.


---

no_small_stack.patch with _PY_FASTCALL_SMALL_STACK=3: 368 bytes/call

test_python_call 7272
test_python_getitem 6628
test_python_iterator 5632
=> total: 19 532

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28870>
_______________________________________

From report at bugs.python.org  Mon Jan  9 12:28:51 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Mon, 09 Jan 2017 17:28:51 +0000
Subject: [issue29216] Space saving step for the LRU cache
In-Reply-To: <1483979902.02.0.507353430639.issue29216@psf.upfronthosting.co.za>
Message-ID: <1483982931.8.0.297866865477.issue29216@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

Ah, there is a wart in the optimization in Python implementation. If call a cached function with multiple integer arguments and with equivalent float arguments, the second call will return a cached result. But if call with single integer and float arguments, both calls will be cached separately.

>>> import functools
>>> @functools.lru_cache()
... def f(*args):
...     return args
... 
>>> f(1, None)
(1, None)
>>> f(1.0, None)
(1, None)
>>> f.cache_info()
CacheInfo(hits=1, misses=1, maxsize=128, currsize=1)
>>> f.cache_clear()
>>> f(1)
(1,)
>>> f(1.0)
(1.0,)
>>> f.cache_info()
CacheInfo(hits=0, misses=2, maxsize=128, currsize=2)

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29216>
_______________________________________

From report at bugs.python.org  Mon Jan  9 12:33:16 2017
From: report at bugs.python.org (Eric Snow)
Date: Mon, 09 Jan 2017 17:33:16 +0000
Subject: [issue29154] 5 failures in test_doctest: ModuleNotFoundError: No
 module named 'IPython'
In-Reply-To: <1483544927.48.0.740005145144.issue29154@psf.upfronthosting.co.za>
Message-ID: <1483983196.05.0.564752508643.issue29154@psf.upfronthosting.co.za>


Eric Snow added the comment:

There shouldn't be anything in CPython that depends on IPython.  I'd recommend checking for $PYTHONSTARTUP, in site.py, and for .pth files in directories on sys.path.  Also look for .py files in the current directory that are shadowing stdlib modules.  Any of these could introduce the attempt to import IPython.  Also, do you get the same error if you directly run the doctest-failing code?

----------
nosy: +eric.snow

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29154>
_______________________________________

From report at bugs.python.org  Mon Jan  9 12:45:03 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Mon, 09 Jan 2017 17:45:03 +0000
Subject: [issue28870] Reduce stack consumption of
 PyObject_CallFunctionObjArgs() and like
In-Reply-To: <1480891813.94.0.54795440575.issue28870@psf.upfronthosting.co.za>
Message-ID: <1483983903.78.0.502341651195.issue28870@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

I'm not sure that the result of pyobjectl_callfunctionobjargs_stacksize() has direct relation to stack consumption in test_python_call, test_python_getitem and test_python_iterator. Try to measure the stack consumption in these cases. This can be done with _testcapi helper that just returns the value of stack pointer. Run all three tests with fixed level of recursion and measure the difference between stack pointers.

Would be nice also measure a performance effect of the patches.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28870>
_______________________________________

From report at bugs.python.org  Mon Jan  9 12:55:37 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Mon, 09 Jan 2017 17:55:37 +0000
Subject: [issue29214] Standard open() does not allow to specify file
 permissions.
In-Reply-To: <1483971421.1.0.906896747438.issue29214@psf.upfronthosting.co.za>
Message-ID: <1483984537.72.0.463942556499.issue29214@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

io.open() is high-level function. It handles buffering, encoding, newline translating. It is even higher-level than C's fopen().

Syscall open() is low-level. Python os.open() is an interface to this low-level feature.

There is a connection between low and high level -- io.open() accepts a file descriptor returned by os.open(). You also can provide the opener argument.

I don't think io.open() needs the support of mode and dir_fd arguments and all possible O_* flags. They are low-level features.

----------
nosy: +serhiy.storchaka

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29214>
_______________________________________

From report at bugs.python.org  Mon Jan  9 13:33:23 2017
From: report at bugs.python.org (Socob)
Date: Mon, 09 Jan 2017 18:33:23 +0000
Subject: [issue28414] SSL match_hostname fails for internationalized domain
 names
In-Reply-To: <1476172956.13.0.85556243538.issue28414@psf.upfronthosting.co.za>
Message-ID: <1483986803.76.0.961715102085.issue28414@psf.upfronthosting.co.za>


Changes by Socob <206a8535 at opayq.com>:


----------
nosy: +Socob

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28414>
_______________________________________

From report at bugs.python.org  Mon Jan  9 13:33:47 2017
From: report at bugs.python.org (Socob)
Date: Mon, 09 Jan 2017 18:33:47 +0000
Subject: [issue17305] IDNA2008 encoding missing
In-Reply-To: <1361928766.42.0.728949411958.issue17305@psf.upfronthosting.co.za>
Message-ID: <1483986827.79.0.390830557605.issue17305@psf.upfronthosting.co.za>


Changes by Socob <206a8535 at opayq.com>:


----------
nosy: +Socob

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue17305>
_______________________________________

From report at bugs.python.org  Mon Jan  9 13:39:29 2017
From: report at bugs.python.org (INADA Naoki)
Date: Mon, 09 Jan 2017 18:39:29 +0000
Subject: [issue28969] lru_cache is not threadsafe
In-Reply-To: <1481712564.52.0.530922236321.issue28969@psf.upfronthosting.co.za>
Message-ID: <1483987169.12.0.305603445719.issue28969@psf.upfronthosting.co.za>


INADA Naoki added the comment:

LGTM

----------
nosy: +inada.naoki

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28969>
_______________________________________

From report at bugs.python.org  Mon Jan  9 14:29:20 2017
From: report at bugs.python.org (=?utf-8?b?0JzQsNGA0Log0JrQvtGA0LXQvdCx0LXRgNCz?=)
Date: Mon, 09 Jan 2017 19:29:20 +0000
Subject: [issue29214] Standard open() does not allow to specify file
 permissions.
In-Reply-To: <1483971421.1.0.906896747438.issue29214@psf.upfronthosting.co.za>
Message-ID: <1483990160.18.0.478397685808.issue29214@psf.upfronthosting.co.za>


???? ????????? added the comment:

Permissions -- are very important thing. As I think, such high-level functions should not hide that important functionality from end-users.

Also, it is not difficult to make a patch (as I think).

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29214>
_______________________________________

From report at bugs.python.org  Mon Jan  9 14:33:04 2017
From: report at bugs.python.org (Xavier de Gaye)
Date: Mon, 09 Jan 2017 19:33:04 +0000
Subject: [issue28759] access to mkfifo,
 mknod and hard links is controled by SELinux MAC on Android
In-Reply-To: <1479715975.06.0.930254771903.issue28759@psf.upfronthosting.co.za>
Message-ID: <1483990384.01.0.679619871977.issue28759@psf.upfronthosting.co.za>


Xavier de Gaye added the comment:

New patch following Serhiy's suggestion in msg285008.

----------
Added file: http://bugs.python.org/file46231/issue28759-01.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28759>
_______________________________________

From report at bugs.python.org  Mon Jan  9 14:35:34 2017
From: report at bugs.python.org (Xavier de Gaye)
Date: Mon, 09 Jan 2017 19:35:34 +0000
Subject: [issue29180] skip tests that raise PermissionError in test_os
 (non-root user on Android)
In-Reply-To: <1483715692.75.0.979129057464.issue29180@psf.upfronthosting.co.za>
Message-ID: <1483990534.69.0.481197706763.issue29180@psf.upfronthosting.co.za>


Xavier de Gaye added the comment:

New patch following Serhiy's suggestion in msg285008.

----------
Added file: http://bugs.python.org/file46232/issue29180_01.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29180>
_______________________________________

From report at bugs.python.org  Mon Jan  9 14:39:50 2017
From: report at bugs.python.org (Xavier de Gaye)
Date: Mon, 09 Jan 2017 19:39:50 +0000
Subject: [issue29181] skip tests that raise PermissionError in test_tarfile
 (non-root user on Android)
In-Reply-To: <1483720295.96.0.810571843517.issue29181@psf.upfronthosting.co.za>
Message-ID: <1483990790.21.0.984664555252.issue29181@psf.upfronthosting.co.za>


Xavier de Gaye added the comment:

New patch following Serhiy's suggestion in msg285008.

----------
Added file: http://bugs.python.org/file46233/issue29181_01.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29181>
_______________________________________

From report at bugs.python.org  Mon Jan  9 14:41:31 2017
From: report at bugs.python.org (Xavier de Gaye)
Date: Mon, 09 Jan 2017 19:41:31 +0000
Subject: [issue29184] skip tests of test_socketserver when bind() raises
 PermissionError (non-root user on Android)
In-Reply-To: <1483732137.12.0.493215743488.issue29184@psf.upfronthosting.co.za>
Message-ID: <1483990891.09.0.836707269815.issue29184@psf.upfronthosting.co.za>


Xavier de Gaye added the comment:

New patch following Serhiy's suggestion in msg285008.

----------
Added file: http://bugs.python.org/file46234/issue29184_01.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29184>
_______________________________________

From report at bugs.python.org  Mon Jan  9 15:11:36 2017
From: report at bugs.python.org (Brett Cannon)
Date: Mon, 09 Jan 2017 20:11:36 +0000
Subject: [issue29214] Standard open() does not allow to specify file
 permissions.
In-Reply-To: <1483971421.1.0.906896747438.issue29214@psf.upfronthosting.co.za>
Message-ID: <1483992696.06.0.144255714721.issue29214@psf.upfronthosting.co.za>


Brett Cannon added the comment:

I agree with Serhiy that I don't think this is necessary for io.open() when os.open() supports this. But this really can't be discussed further until someone provides a clear design proposal on how to make this work with io.open().

----------
nosy: +brett.cannon
type: behavior -> enhancement
versions:  -Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29214>
_______________________________________

From report at bugs.python.org  Mon Jan  9 15:22:38 2017
From: report at bugs.python.org (=?utf-8?b?0JzQsNGA0Log0JrQvtGA0LXQvdCx0LXRgNCz?=)
Date: Mon, 09 Jan 2017 20:22:38 +0000
Subject: [issue29214] Standard open() does not allow to specify file
 permissions.
In-Reply-To: <1483971421.1.0.906896747438.issue29214@psf.upfronthosting.co.za>
Message-ID: <1483993358.34.0.602361801895.issue29214@psf.upfronthosting.co.za>


???? ????????? added the comment:

I expect this:

open(...., perms=0o644)

Why not?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29214>
_______________________________________

From report at bugs.python.org  Mon Jan  9 15:23:45 2017
From: report at bugs.python.org (=?utf-8?b?0JzQsNGA0Log0JrQvtGA0LXQvdCx0LXRgNCz?=)
Date: Mon, 09 Jan 2017 20:23:45 +0000
Subject: [issue29214] Standard open() does not allow to specify file
 permissions.
In-Reply-To: <1483971421.1.0.906896747438.issue29214@psf.upfronthosting.co.za>
Message-ID: <1483993425.64.0.972953865099.issue29214@psf.upfronthosting.co.za>


???? ????????? added the comment:

so, io.open() should just pass that value down to implementation. implementation should use this value instead of hard-coded 0o666.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29214>
_______________________________________

From report at bugs.python.org  Mon Jan  9 15:42:37 2017
From: report at bugs.python.org (George King)
Date: Mon, 09 Jan 2017 20:42:37 +0000
Subject: [issue29057] Compiler failure on Mac OS X - sys/random.h
In-Reply-To: <1482514649.72.0.460227366509.issue29057@psf.upfronthosting.co.za>
Message-ID: <1483994557.83.0.886053138712.issue29057@psf.upfronthosting.co.za>


George King added the comment:

I am encountering this problem on macOS 10.12.2, with Xcode 8.2.1 (latest).

I have tried building from the following cpython branches today (using the github fork):

2.7: 13a39142c047
In file included from ../../Python/random.c:7:
/usr/include/sys/random.h:37:32: error: unknown type name 'u_int'
void read_random(void* buffer, u_int numBytes);


3.5: 35334a4d41aa
3.6: 6bf563472ccd
Both 3.x branches report a different error:

../../Python/pytime.c:567:11: warning: implicit declaration of function 'clock_gettime' is invalid in C99 [-Wimplicit-function-declaration]
    err = clock_gettime(CLOCK_REALTIME, &ts);
          ^
../../Python/pytime.c:567:25: error: use of undeclared identifier 'CLOCK_REALTIME'
    err = clock_gettime(CLOCK_REALTIME, &ts);
                        ^
../../Python/pytime.c:581:13: warning: implicit declaration of function 'clock_getres' is invalid in C99 [-Wimplicit-function-declaration]
        if (clock_getres(CLOCK_REALTIME, &res) == 0)
            ^
../../Python/pytime.c:581:26: error: use of undeclared identifier 'CLOCK_REALTIME'
        if (clock_getres(CLOCK_REALTIME, &res) == 0)
                         ^

----------
nosy: +George.King

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29057>
_______________________________________

From report at bugs.python.org  Mon Jan  9 15:43:41 2017
From: report at bugs.python.org (George King)
Date: Mon, 09 Jan 2017 20:43:41 +0000
Subject: [issue29057] Compiler failure on Mac OS X - sys/random.h
In-Reply-To: <1482514649.72.0.460227366509.issue29057@psf.upfronthosting.co.za>
Message-ID: <1483994621.8.0.73449678681.issue29057@psf.upfronthosting.co.za>


George King added the comment:

(I meant the github mirror: github.com/python/cpython)

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29057>
_______________________________________

From report at bugs.python.org  Mon Jan  9 15:53:27 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Mon, 09 Jan 2017 20:53:27 +0000
Subject: [issue29214] Standard open() does not allow to specify file
 permissions.
In-Reply-To: <1483971421.1.0.906896747438.issue29214@psf.upfronthosting.co.za>
Message-ID: <1483995207.79.0.185295068823.issue29214@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

open(..., opener=partial(os.open, mode=0o644))

Not every combination of functions deserves a new parameter of a builtin.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29214>
_______________________________________

From report at bugs.python.org  Mon Jan  9 16:16:16 2017
From: report at bugs.python.org (Francis Deslauriers)
Date: Mon, 09 Jan 2017 21:16:16 +0000
Subject: [issue28909] Adding LTTng-UST tracing support
In-Reply-To: <1481227474.94.0.337411886613.issue28909@psf.upfronthosting.co.za>
Message-ID: <1483996576.83.0.189298119213.issue28909@psf.upfronthosting.co.za>


Francis Deslauriers added the comment:

Thanks ?ukasz,
I will add tests following the example of the DTrace tests and update the Doc/howto/instrumentation.rst file to include this new information.

LTTng can be used  on all major Linux distros (Ubuntu, Debian, Fedora, etc.) either from distribution packages or compiled from source. It runs on kernels that came out after 2.6.27.

You are right, I should have explained why I changed the provider name. I changed it be as specific as possible to avoid potential name clash with other Python interpreters.
I will send an updated patch that keeps the old name given that it was already released.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28909>
_______________________________________

From report at bugs.python.org  Mon Jan  9 16:22:31 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Mon, 09 Jan 2017 21:22:31 +0000
Subject: [issue29162] pyshell.py: name 'sys' is not defined
In-Reply-To: <1483579208.82.0.533923327462.issue29162@psf.upfronthosting.co.za>
Message-ID: <1483996951.24.0.999756493513.issue29162@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

I agree that incidental names should be eliminated from import. Could you open a new issue for this?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29162>
_______________________________________

From report at bugs.python.org  Mon Jan  9 16:53:13 2017
From: report at bugs.python.org (Ned Deily)
Date: Mon, 09 Jan 2017 21:53:13 +0000
Subject: [issue29057] Compiler failure on Mac OS X - sys/random.h
In-Reply-To: <1482514649.72.0.460227366509.issue29057@psf.upfronthosting.co.za>
Message-ID: <1483998793.53.0.574045148336.issue29057@psf.upfronthosting.co.za>


Ned Deily added the comment:

George, did you do totally clean builds, including rerunning ./configure?  If so, what ./configure and make commands did you use?  Using the current hg repo, I don't see the failures you are seeing.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29057>
_______________________________________

From report at bugs.python.org  Mon Jan  9 17:27:19 2017
From: report at bugs.python.org (=?utf-8?b?0JzQsNGA0Log0JrQvtGA0LXQvdCx0LXRgNCz?=)
Date: Mon, 09 Jan 2017 22:27:19 +0000
Subject: [issue29214] Standard open() does not allow to specify file
 permissions.
In-Reply-To: <1483971421.1.0.906896747438.issue29214@psf.upfronthosting.co.za>
Message-ID: <1484000839.0.0.602273374487.issue29214@psf.upfronthosting.co.za>


???? ????????? added the comment:

Such construction is not so easy. Especially for beginners. Not everyone even uses context managers to work with files. They will try to use os.chmod(). More clever will use os.fchmod(fileobj.fileno()). And in rare case someone asks about race condition between threads. So, not implementing that is error prone, and also forces users to make hidden bugs.

Why not to follow the principle of Least Surprise ?

In you way, we did not have to implement a lot of things, that we have now. Like tempfile, shutil and so on. They may be implemented using complex combination of simplier items. Answer: these things make Python easy, secure and (insert good words here by yourself).

Why are you afraid to add one very simple thing ?

Next, what if I want to use, say, 'r+b', in that case I should remember combination of O_XXXXX constants like O_TRUNC|O_CREAT|O_APPEND and so on.. It's great that Python allows such things! really great! but how many people want to write that in their code?

(Sorry for long text, but I really can not understand)

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29214>
_______________________________________

From report at bugs.python.org  Mon Jan  9 17:34:30 2017
From: report at bugs.python.org (Martin Panter)
Date: Mon, 09 Jan 2017 22:34:30 +0000
Subject: [issue29145] failing overflow checks in replace_*
In-Reply-To: <1483468602.77.0.231507148809.issue29145@psf.upfronthosting.co.za>
Message-ID: <1484001270.08.0.909200328702.issue29145@psf.upfronthosting.co.za>


Martin Panter added the comment:

Both fixes (join and replace) look good to me. However I don?t think it is necessary to change the exception message in 3.5 or 3.6.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29145>
_______________________________________

From report at bugs.python.org  Mon Jan  9 17:43:26 2017
From: report at bugs.python.org (David Muller)
Date: Mon, 09 Jan 2017 22:43:26 +0000
Subject: [issue29217] Documentation for uuid has wrong description for variant
Message-ID: <1484001806.43.0.207679570942.issue29217@psf.upfronthosting.co.za>


New submission from David Muller:

The documentation's description for uuid.variant says that its value is one of several integer constants, but those constants are actually strings.

----------
assignee: docs at python
components: Documentation
messages: 285079
nosy: TigerhawkT3, docs at python
priority: normal
severity: normal
status: open
title: Documentation for uuid has wrong description for variant
type: behavior
versions: Python 2.7, Python 3.3, Python 3.4, Python 3.5, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29217>
_______________________________________

From report at bugs.python.org  Mon Jan  9 18:01:13 2017
From: report at bugs.python.org (Eric N. Vander Weele)
Date: Mon, 09 Jan 2017 23:01:13 +0000
Subject: [issue29218] distutils: Remove unused install_misc class
Message-ID: <1484002873.56.0.81761190557.issue29218@psf.upfronthosting.co.za>


New submission from Eric N. Vander Weele:

This class hasn't been used for quite some time.  Seems safe to remove.

----------
components: Build, Distutils
files: distutils-remove-install_misc-1.patch
keywords: patch
messages: 285080
nosy: dstufft, eric.araujo, ericvw, gward
priority: normal
severity: normal
status: open
title: distutils: Remove unused install_misc class
type: enhancement
versions: Python 3.7
Added file: http://bugs.python.org/file46235/distutils-remove-install_misc-1.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29218>
_______________________________________

From report at bugs.python.org  Mon Jan  9 18:29:05 2017
From: report at bugs.python.org (Anthony Sottile)
Date: Mon, 09 Jan 2017 23:29:05 +0000
Subject: [issue20094] intermitent failures with test_dbm
In-Reply-To: <1388335598.36.0.994619281921.issue20094@psf.upfronthosting.co.za>
Message-ID: <1484004545.35.0.505103279463.issue20094@psf.upfronthosting.co.za>


Anthony Sottile added the comment:

I'm seeing this same failure in python3.5 on 16.04 about 20% of the time:

```
$ python3.5 -m test -v test_dbm
== CPython 3.5.2 (default, Nov 17 2016, 17:05:23) [GCC 5.4.0 20160609]
==   Linux-4.4.0-57-generic-x86_64-with-Ubuntu-16.04-xenial little-endian
==   hash algorithm: siphash24 64bit
==   /tmp/test_python_8169
Testing with flags: sys.flags(debug=0, inspect=0, interactive=0, optimize=0, dont_write_bytecode=0, no_user_site=0, no_site=0, ignore_environment=0, verbose=0, bytes_warning=0, quiet=0, hash_randomization=1, isolated=0)
[1/1] test_dbm
test_keys (test.test_dbm.WhichDBTestCase) ... ok
test_whichdb (test.test_dbm.WhichDBTestCase) ... ok
test_whichdb_ndbm (test.test_dbm.WhichDBTestCase) ... BDB0004 fop_read_meta: @test_8169_tmp_ndbm.db: unexpected file type or format
ok
test_anydbm_access (test.test_dbm.TestCase-dbm.gnu) ... ok
test_anydbm_creation (test.test_dbm.TestCase-dbm.gnu) ... ok
test_anydbm_creation_n_file_exists_with_invalid_contents (test.test_dbm.TestCase-dbm.gnu) ... ok
test_anydbm_keys (test.test_dbm.TestCase-dbm.gnu) ... ok
test_anydbm_modification (test.test_dbm.TestCase-dbm.gnu) ... ok
test_anydbm_not_existing (test.test_dbm.TestCase-dbm.gnu) ... ok
test_anydbm_read (test.test_dbm.TestCase-dbm.gnu) ... ok
test_error (test.test_dbm.TestCase-dbm.gnu) ... ok
test_anydbm_access (test.test_dbm.TestCase-dbm.ndbm) ... ok
test_anydbm_creation (test.test_dbm.TestCase-dbm.ndbm) ... ERROR
BDB3028 @test_8169_tmp.db: unable to flush: No such file or directory
test_anydbm_creation_n_file_exists_with_invalid_contents (test.test_dbm.TestCase-dbm.ndbm) ... ok
test_anydbm_keys (test.test_dbm.TestCase-dbm.ndbm) ... ok
test_anydbm_modification (test.test_dbm.TestCase-dbm.ndbm) ... ERROR
BDB3028 @test_8169_tmp.db: unable to flush: No such file or directory
test_anydbm_not_existing (test.test_dbm.TestCase-dbm.ndbm) ... ok
test_anydbm_read (test.test_dbm.TestCase-dbm.ndbm) ... ERROR
test_error (test.test_dbm.TestCase-dbm.ndbm) ... ok
test_anydbm_access (test.test_dbm.TestCase-dbm.dumb) ... ok
test_anydbm_creation (test.test_dbm.TestCase-dbm.dumb) ... ok
test_anydbm_creation_n_file_exists_with_invalid_contents (test.test_dbm.TestCase-dbm.dumb) ... ok
test_anydbm_keys (test.test_dbm.TestCase-dbm.dumb) ... ok
test_anydbm_modification (test.test_dbm.TestCase-dbm.dumb) ... ok
test_anydbm_not_existing (test.test_dbm.TestCase-dbm.dumb) ... ok
test_anydbm_read (test.test_dbm.TestCase-dbm.dumb) ... ok
test_error (test.test_dbm.TestCase-dbm.dumb) ... ok

======================================================================
ERROR: test_anydbm_creation (test.test_dbm.TestCase-dbm.ndbm)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3.5/test/test_dbm.py", line 74, in test_anydbm_creation
    self.read_helper(f)
  File "/usr/lib/python3.5/test/test_dbm.py", line 115, in read_helper
    self.assertEqual(self._dict[key], f[key.encode("ascii")])
KeyError: b'0'

======================================================================
ERROR: test_anydbm_modification (test.test_dbm.TestCase-dbm.ndbm)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3.5/test/test_dbm.py", line 89, in test_anydbm_modification
    self.read_helper(f)
  File "/usr/lib/python3.5/test/test_dbm.py", line 115, in read_helper
    self.assertEqual(self._dict[key], f[key.encode("ascii")])
KeyError: b'0'

======================================================================
ERROR: test_anydbm_read (test.test_dbm.TestCase-dbm.ndbm)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3.5/test/test_dbm.py", line 95, in test_anydbm_read
    self.read_helper(f)
  File "/usr/lib/python3.5/test/test_dbm.py", line 115, in read_helper
    self.assertEqual(self._dict[key], f[key.encode("ascii")])
KeyError: b'0'

----------------------------------------------------------------------
Ran 27 tests in 0.093s

FAILED (errors=3)
test test_dbm failed
1 test failed:
    test_dbm
```

Under 3.6 (built from source), it seems to fail 100% of the time

```
$ ./python -m test test_dbm -v
== CPython 3.6.0 (default, Jan 9 2017, 15:00:38) [GCC 5.4.0 20160609]
==   Linux-4.4.0-57-generic-x86_64-with-debian-stretch-sid little-endian
==   hash algorithm: siphash24 64bit
==  cwd: /home/asottile/workspace/cpython/build/test_python_8246
==  encodings: locale=UTF-8, FS=utf-8
Testing with flags: sys.flags(debug=0, inspect=0, interactive=0, optimize=0, dont_write_bytecode=0, no_user_site=0, no_site=0, ignore_environment=0, verbose=0, bytes_warning=0, quiet=0, hash_randomization=1, isolated=0)
Run tests sequentially
0:00:00 [1/1] test_dbm
test_keys (test.test_dbm.WhichDBTestCase) ... ok
test_whichdb (test.test_dbm.WhichDBTestCase) ... ok
test_whichdb_ndbm (test.test_dbm.WhichDBTestCase) ... BDB0004 fop_read_meta: @test_8246_tmp_ndbm.db: unexpected file type or format
ok
test_anydbm_access (test.test_dbm.TestCase-dbm.gnu) ... ok
test_anydbm_creation (test.test_dbm.TestCase-dbm.gnu) ... ok
test_anydbm_creation_n_file_exists_with_invalid_contents (test.test_dbm.TestCase-dbm.gnu) ... ok
test_anydbm_keys (test.test_dbm.TestCase-dbm.gnu) ... ok
test_anydbm_modification (test.test_dbm.TestCase-dbm.gnu) ... ok
test_anydbm_not_existing (test.test_dbm.TestCase-dbm.gnu) ... ok
test_anydbm_read (test.test_dbm.TestCase-dbm.gnu) ... ok
test_error (test.test_dbm.TestCase-dbm.gnu) ... ok
test_anydbm_access (test.test_dbm.TestCase-dbm.ndbm) ... ok
test_anydbm_creation (test.test_dbm.TestCase-dbm.ndbm) ... ERROR
BDB3028 @test_8246_tmp.db: unable to flush: No such file or directory
test_anydbm_creation_n_file_exists_with_invalid_contents (test.test_dbm.TestCase-dbm.ndbm) ... ok
test_anydbm_keys (test.test_dbm.TestCase-dbm.ndbm) ... ok
test_anydbm_modification (test.test_dbm.TestCase-dbm.ndbm) ... ERROR
BDB3028 @test_8246_tmp.db: unable to flush: No such file or directory
test_anydbm_not_existing (test.test_dbm.TestCase-dbm.ndbm) ... ok
test_anydbm_read (test.test_dbm.TestCase-dbm.ndbm) ... ERROR
test_error (test.test_dbm.TestCase-dbm.ndbm) ... ok
test_anydbm_access (test.test_dbm.TestCase-dbm.dumb) ... ok
test_anydbm_creation (test.test_dbm.TestCase-dbm.dumb) ... ok
test_anydbm_creation_n_file_exists_with_invalid_contents (test.test_dbm.TestCase-dbm.dumb) ... ok
test_anydbm_keys (test.test_dbm.TestCase-dbm.dumb) ... ok
test_anydbm_modification (test.test_dbm.TestCase-dbm.dumb) ... ok
test_anydbm_not_existing (test.test_dbm.TestCase-dbm.dumb) ... ok
test_anydbm_read (test.test_dbm.TestCase-dbm.dumb) ... ok
test_error (test.test_dbm.TestCase-dbm.dumb) ... ok

======================================================================
ERROR: test_anydbm_creation (test.test_dbm.TestCase-dbm.ndbm)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/asottile/workspace/cpython/Lib/test/test_dbm.py", line 73, in test_anydbm_creation
    self.read_helper(f)
  File "/home/asottile/workspace/cpython/Lib/test/test_dbm.py", line 114, in read_helper
    self.assertEqual(self._dict[key], f[key.encode("ascii")])
KeyError: b'0'

======================================================================
ERROR: test_anydbm_modification (test.test_dbm.TestCase-dbm.ndbm)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/asottile/workspace/cpython/Lib/test/test_dbm.py", line 88, in test_anydbm_modification
    self.read_helper(f)
  File "/home/asottile/workspace/cpython/Lib/test/test_dbm.py", line 114, in read_helper
    self.assertEqual(self._dict[key], f[key.encode("ascii")])
KeyError: b'0'

======================================================================
ERROR: test_anydbm_read (test.test_dbm.TestCase-dbm.ndbm)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/asottile/workspace/cpython/Lib/test/test_dbm.py", line 94, in test_anydbm_read
    self.read_helper(f)
  File "/home/asottile/workspace/cpython/Lib/test/test_dbm.py", line 114, in read_helper
    self.assertEqual(self._dict[key], f[key.encode("ascii")])
KeyError: b'0'

----------------------------------------------------------------------
Ran 27 tests in 0.084s

FAILED (errors=3)
test test_dbm failed
test_dbm failed

1 test failed:
    test_dbm

Total duration: 112 ms
Tests result: FAILURE
```

----------
nosy: +asottile
versions: +Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue20094>
_______________________________________

From report at bugs.python.org  Mon Jan  9 18:37:13 2017
From: report at bugs.python.org (Anthony Sottile)
Date: Mon, 09 Jan 2017 23:37:13 +0000
Subject: [issue20094] intermitent failures with test_dbm
In-Reply-To: <1388335598.36.0.994619281921.issue20094@psf.upfronthosting.co.za>
Message-ID: <1484005033.07.0.581516767507.issue20094@psf.upfronthosting.co.za>


Anthony Sottile added the comment:

Stepping through the code, it seems under ndbm it is creating a file with a '.db' extension:

```
(Pdb) list
 47  	             'g': b'intended',
 48  	             }
 49  	
 50  	    def init_db(self):
 51  	        import pdb; pdb.set_trace()
 52  ->	        with dbm.open(_fname, 'n') as f:
 53  	            for k in self._dict:
 54  	                f[k.encode("ascii")] = self._dict[k]
 55  	
 56  	    def keys_helper(self, f):
 57  	        keys = sorted(k.decode("ascii") for k in f.keys())
(Pdb) n
> /home/asottile/workspace/cpython/Lib/test/test_dbm.py(53)init_db()
-> for k in self._dict:
(Pdb) os.listdir('.')
['@test_8371_tmp.db']
(Pdb) _fname
'@test_8371_tmp'
```

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue20094>
_______________________________________

From report at bugs.python.org  Mon Jan  9 18:48:20 2017
From: report at bugs.python.org (Anthony Sottile)
Date: Mon, 09 Jan 2017 23:48:20 +0000
Subject: [issue20094] intermitent failures with test_dbm
In-Reply-To: <1388335598.36.0.994619281921.issue20094@psf.upfronthosting.co.za>
Message-ID: <1484005700.09.0.441719737951.issue20094@psf.upfronthosting.co.za>


Anthony Sottile added the comment:

That doesn't seem to be the problem though, that occurs in both the successful and failure case

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue20094>
_______________________________________

From report at bugs.python.org  Mon Jan  9 19:34:20 2017
From: report at bugs.python.org (Eric N. Vander Weele)
Date: Tue, 10 Jan 2017 00:34:20 +0000
Subject: [issue29218] distutils: Remove unused install_misc class
In-Reply-To: <1484002873.56.0.81761190557.issue29218@psf.upfronthosting.co.za>
Message-ID: <1484008460.91.0.52738528885.issue29218@psf.upfronthosting.co.za>


Changes by Eric N. Vander Weele <ericvw at gmail.com>:


----------
components:  -Build

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29218>
_______________________________________

From report at bugs.python.org  Mon Jan  9 19:45:51 2017
From: report at bugs.python.org (paul j3)
Date: Tue, 10 Jan 2017 00:45:51 +0000
Subject: [issue14392] type=bool doesn't raise error in argparse.Action
In-Reply-To: <1332486208.67.0.940302931961.issue14392@psf.upfronthosting.co.za>
Message-ID: <1484009151.82.0.118143780082.issue14392@psf.upfronthosting.co.za>


Changes by paul j3 <ajipanca at gmail.com>:


----------
nosy: +paul.j3

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue14392>
_______________________________________

From report at bugs.python.org  Mon Jan  9 19:52:33 2017
From: report at bugs.python.org (paul j3)
Date: Tue, 10 Jan 2017 00:52:33 +0000
Subject: =?utf-8?q?=5Bissue9182=5D_document_=E2=80=9C--=E2=80=9D_as_a_way_to_disti?=
 =?utf-8?q?nguish_option_w/_narg=3D=27+=27_from_positional_argument_in_arg?=
 =?utf-8?q?parse?=
In-Reply-To: <1278433481.81.0.201692715302.issue9182@psf.upfronthosting.co.za>
Message-ID: <1484009553.67.0.319780852582.issue9182@psf.upfronthosting.co.za>


Changes by paul j3 <ajipanca at gmail.com>:


----------
nosy: +paul.j3

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue9182>
_______________________________________

From report at bugs.python.org  Mon Jan  9 19:54:07 2017
From: report at bugs.python.org (paul j3)
Date: Tue, 10 Jan 2017 00:54:07 +0000
Subject: [issue14102] argparse: add ability to create a man page
In-Reply-To: <1330031760.56.0.8938390885.issue14102@psf.upfronthosting.co.za>
Message-ID: <1484009647.35.0.485689214452.issue14102@psf.upfronthosting.co.za>


Changes by paul j3 <ajipanca at gmail.com>:


----------
nosy: +paul.j3

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue14102>
_______________________________________

From report at bugs.python.org  Mon Jan  9 20:38:07 2017
From: report at bugs.python.org (Ilya Kulakov)
Date: Tue, 10 Jan 2017 01:38:07 +0000
Subject: [issue24699] TemporaryDirectory is cleaned up twice
In-Reply-To: <1437691863.88.0.795764975498.issue24699@psf.upfronthosting.co.za>
Message-ID: <1484012287.49.0.714132257484.issue24699@psf.upfronthosting.co.za>


Changes by Ilya Kulakov <kulakov.ilya at gmail.com>:


----------
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24699>
_______________________________________

From report at bugs.python.org  Mon Jan  9 20:43:30 2017
From: report at bugs.python.org (Ilya Kulakov)
Date: Tue, 10 Jan 2017 01:43:30 +0000
Subject: [issue29219] TracebackException(capture_locals=True) may fail with
 RecursionError
Message-ID: <1484012610.3.0.9642464783.issue29219@psf.upfronthosting.co.za>


New submission from Ilya Kulakov:

I'm using Python 3.5.2 to be precise. I have code that is roughly equivalent to:

    import sys
    import traceback

    def handle_exception(exc_type, exc_value, exc_traceback):
        traceback.TracebackException(exc_type, exc_value, exc_traceback, capture_locals=True)

    sys.excepthook = handle_exception

For one of the clients the following error happened:

Traceback (most recent call last):
  File "...", line 222, in ...
  File "...", line 160, in ...
  File "...", line 878, in ...
  File ":/traceback.py", line 463, in __init__
  File ":/traceback.py", line 474, in __init__
  File ":/traceback.py", line 352, in extract
  File ":/traceback.py", line 257, in __init__
  File ":/traceback.py", line 257, in <genexpr>
  File ":/ctypes/__init__.py", line 354, in __repr__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
  File ":/ctypes/__init__.py", line 360, in __getattr__
  File ":/ctypes/__init__.py", line 365, in __getitem__
RecursionError: maximum recursion depth exceeded

----------
components: Library (Lib)
messages: 285084
nosy: Ilya.Kulakov
priority: normal
severity: normal
status: open
title: TracebackException(capture_locals=True) may fail with RecursionError
type: crash
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29219>
_______________________________________

From report at bugs.python.org  Mon Jan  9 21:27:04 2017
From: report at bugs.python.org (Anthony Sottile)
Date: Tue, 10 Jan 2017 02:27:04 +0000
Subject: [issue28700] test_dbm failure: KeyError: b'0' (intermittent in 3.5,
 reliable in 3.6)
In-Reply-To: <1479238038.8.0.290390662143.issue28700@psf.upfronthosting.co.za>
Message-ID: <1484015224.68.0.242752710231.issue28700@psf.upfronthosting.co.za>


Changes by Anthony Sottile <asottile at umich.edu>:


----------
nosy: +Anthony Sottile

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28700>
_______________________________________

From report at bugs.python.org  Mon Jan  9 21:47:19 2017
From: report at bugs.python.org (Mark Blakeney)
Date: Tue, 10 Jan 2017 02:47:19 +0000
Subject: [issue29220] Python 3.6 regression/change using
 logging.addLevelName() to clear a name
Message-ID: <1484016439.6.0.79629021761.issue29220@psf.upfronthosting.co.za>


New submission from Mark Blakeney:

I have code which does a logging.addLevelName(logging.INFO, '') so that the logging level name is not displayed in INFO messages, but is in all other levels. I have been running this code fine since many versions of Python 2 through to 3.5.

Now running with python 3.6.0-1 (on Arch Linux), the name is output in messages as "Level 20". Changing the empty string '' in the call above to a space ' ' is my current work-around.

----------
components: Library (Lib)
messages: 285085
nosy: markb
priority: normal
severity: normal
status: open
title: Python 3.6 regression/change using logging.addLevelName() to clear a name
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29220>
_______________________________________

From report at bugs.python.org  Mon Jan  9 22:05:14 2017
From: report at bugs.python.org (Roundup Robot)
Date: Tue, 10 Jan 2017 03:05:14 +0000
Subject: [issue29145] failing overflow checks in replace_*
In-Reply-To: <1483468602.77.0.231507148809.issue29145@psf.upfronthosting.co.za>
Message-ID: <20170110030510.104398.66430.34625E81@psf.io>


Roundup Robot added the comment:

New changeset 09b1cdac74de by Xiang Zhang in branch '3.5':
Issue #29145: Fix overflow checks in str.replace() and str.join().
https://hg.python.org/cpython/rev/09b1cdac74de

New changeset d966ccda9f17 by Xiang Zhang in branch '3.6':
Issue #29145: Merge 3.5.
https://hg.python.org/cpython/rev/d966ccda9f17

New changeset f61a0e8ec022 by Xiang Zhang in branch 'default':
Issue #29145: Merge 3.6.
https://hg.python.org/cpython/rev/f61a0e8ec022

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29145>
_______________________________________

From report at bugs.python.org  Mon Jan  9 22:08:46 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Tue, 10 Jan 2017 03:08:46 +0000
Subject: [issue29145] failing overflow checks in replace_*
In-Reply-To: <1483468602.77.0.231507148809.issue29145@psf.upfronthosting.co.za>
Message-ID: <1484017726.04.0.119406668758.issue29145@psf.upfronthosting.co.za>


Xiang Zhang added the comment:

Thanks you all. :-?

----------
resolution:  -> fixed
stage: commit review -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29145>
_______________________________________

From report at bugs.python.org  Mon Jan  9 22:32:54 2017
From: report at bugs.python.org (Eryk Sun)
Date: Tue, 10 Jan 2017 03:32:54 +0000
Subject: [issue24699] TemporaryDirectory is cleaned up twice
In-Reply-To: <1437691863.88.0.795764975498.issue24699@psf.upfronthosting.co.za>
Message-ID: <1484019174.52.0.471672567906.issue24699@psf.upfronthosting.co.za>


Changes by Eryk Sun <eryksun at gmail.com>:


----------
resolution:  -> out of date
stage:  -> resolved

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24699>
_______________________________________

From report at bugs.python.org  Mon Jan  9 22:32:55 2017
From: report at bugs.python.org (Roundup Robot)
Date: Tue, 10 Jan 2017 03:32:55 +0000
Subject: [issue29217] Documentation for uuid has wrong description for variant
In-Reply-To: <1484001806.43.0.207679570942.issue29217@psf.upfronthosting.co.za>
Message-ID: <20170110033252.103882.65874.E7B4A085@psf.io>


Roundup Robot added the comment:

New changeset 0d4e0a736688 by Xiang Zhang in branch '2.7':
Issue #29217: Fix the wrong type description of UUID.variant.
https://hg.python.org/cpython/rev/0d4e0a736688

New changeset 1f8e8e16e996 by Xiang Zhang in branch '3.5':
Issue #29217: Fix the wrong type description of UUID.variant.
https://hg.python.org/cpython/rev/1f8e8e16e996

New changeset aabb9360ff93 by Xiang Zhang in branch '3.6':
Issue #29217: Merge 3.5.
https://hg.python.org/cpython/rev/aabb9360ff93

New changeset a30cdf366c02 by Xiang Zhang in branch 'default':
Issue #29217: Merge 3.6.
https://hg.python.org/cpython/rev/a30cdf366c02

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29217>
_______________________________________

From report at bugs.python.org  Mon Jan  9 22:34:28 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Tue, 10 Jan 2017 03:34:28 +0000
Subject: [issue29217] Documentation for uuid has wrong description for variant
In-Reply-To: <1484001806.43.0.207679570942.issue29217@psf.upfronthosting.co.za>
Message-ID: <1484019268.31.0.46780657659.issue29217@psf.upfronthosting.co.za>


Xiang Zhang added the comment:

Thanks for your report David. :-) I simply remove the type description since I think the type of the constants doesn't matter here.

----------
nosy: +xiang.zhang
resolution:  -> fixed
stage:  -> resolved
status: open -> closed
versions:  -Python 3.3, Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29217>
_______________________________________

From report at bugs.python.org  Mon Jan  9 22:41:40 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Tue, 10 Jan 2017 03:41:40 +0000
Subject: [issue29220] Python 3.6 regression/change using
 logging.addLevelName() to clear a name
In-Reply-To: <1484016439.6.0.79629021761.issue29220@psf.upfronthosting.co.za>
Message-ID: <1484019700.95.0.492756676309.issue29220@psf.upfronthosting.co.za>


Changes by Xiang Zhang <angwerzx at 126.com>:


----------
nosy: +vinay.sajip

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29220>
_______________________________________

From report at bugs.python.org  Mon Jan  9 23:18:47 2017
From: report at bugs.python.org (Anthony Scopatz)
Date: Tue, 10 Jan 2017 04:18:47 +0000
Subject: [issue29221] ABC Recursion Error on isinstance() with less than
 recursion limit class hierarchy depth
Message-ID: <1484021927.49.0.387886726852.issue29221@psf.upfronthosting.co.za>


New submission from Anthony Scopatz:

Classes that have an abstract base class somewhere in their hierarchy have a significantly reduced depth with respect to the recursion limit. In the attached minimal example, the class hierarchy is only able to be 245 deep past the ABC before a recursion error, rather than the expected 1000. 

Also disconcerting is that this recursion error is triggered by unrelated objects, namely an isinstance() check. This means that the error can happen at any point in the interpreter simply because the offending class exists. You don't need to call isinstance() with the offending class.

This is likely due to the way the way that ABCMeta.__subclasscheck__(). This issue can be avoided by either:

1. Not having a deep-ish hierarchy, or 
2. Calling the trigger, isinstance(), each time a new class is created (inside of the loop).

Option (2) works because it tricks ABCMeta into putting each subclass class into its internal cache of subclasses. This fix is undesirable in general because what triggers the error, in general may not be known and can cross package boundaries.

Note: I only tested this on Python v3.4 and v3.5, but it presumably affects all currently supported versions of Python.

----------
components: Library (Lib)
files: abc_rec_fail.py
messages: 285090
nosy: Anthony Scopatz
priority: normal
severity: normal
status: open
title: ABC Recursion Error on isinstance() with less than recursion limit class hierarchy depth
type: behavior
versions: Python 2.7, Python 3.3, Python 3.4, Python 3.5, Python 3.6, Python 3.7
Added file: http://bugs.python.org/file46236/abc_rec_fail.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29221>
_______________________________________

From report at bugs.python.org  Tue Jan 10 00:53:47 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Tue, 10 Jan 2017 05:53:47 +0000
Subject: [issue29221] ABC Recursion Error on isinstance() with less than
 recursion limit class hierarchy depth
In-Reply-To: <1484021927.49.0.387886726852.issue29221@psf.upfronthosting.co.za>
Message-ID: <1484027627.03.0.933476103418.issue29221@psf.upfronthosting.co.za>


Changes by Serhiy Storchaka <storchaka+cpython at gmail.com>:


----------
nosy: +serhiy.storchaka
versions:  -Python 3.3, Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29221>
_______________________________________

From report at bugs.python.org  Tue Jan 10 01:07:28 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Tue, 10 Jan 2017 06:07:28 +0000
Subject: [issue29219] TracebackException(capture_locals=True) may fail with
 RecursionError
In-Reply-To: <1484012610.3.0.9642464783.issue29219@psf.upfronthosting.co.za>
Message-ID: <1484028448.5.0.108526674353.issue29219@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

Could you please provide a script that reproduces an issue?

Seems the repr of uninitialized CDLL instance is used. It causes an infinite recursion in attempt to resolve not set private attributes (_name, _handle, _FuncPtr).

----------
components: +ctypes
nosy: +amaury.forgeotdarc, belopolsky, meador.inge, serhiy.storchaka
type: crash -> behavior
versions: +Python 2.7, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29219>
_______________________________________

From report at bugs.python.org  Tue Jan 10 01:20:18 2017
From: report at bugs.python.org (Nick Coghlan)
Date: Tue, 10 Jan 2017 06:20:18 +0000
Subject: [issue26632] @public - an __all__ decorator
In-Reply-To: <1458786797.31.0.0749407715279.issue26632@psf.upfronthosting.co.za>
Message-ID: <1484029218.77.0.502500915511.issue26632@psf.upfronthosting.co.za>


Nick Coghlan added the comment:

Chiming in so Barry & Zach can take this feedback into account for any future proposal:

- I think the specific term "public" has too much baggage from other languages, especially in the sense that it implies that "private" is the default. In reality, all attributes in Python are public, with __all__ serving as executable documentation and a constraint on wildcard exports

- the notion of an automatically supplied __all__ object that defaults to reporting [name for name in dir(module) if not name.startswith("_")] has a lot more to recommend it (as it encodes the default wildcard export behaviour directly into the language implementation, rather than requiring that module analysis tools implement that default themselves)

- given "__all__" as the base name, the method name for class and function registration could just be the generic "__all__.register" (although "__all__.export" would also work, since it's the counterpart of "import *")

- using an object namespace would allow for other registration methods if that seemed appropriate

Even then, I'm at best +0 on the proposal, but I also don't make heavy use of code development helpers (IDEs, etc)

----------
nosy: +ncoghlan

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue26632>
_______________________________________

From report at bugs.python.org  Tue Jan 10 01:32:23 2017
From: report at bugs.python.org (Vex Woo)
Date: Tue, 10 Jan 2017 06:32:23 +0000
Subject: [issue29222] Python3 interactive shell hangs on
Message-ID: <1484029943.51.0.404854509621.issue29222@psf.upfronthosting.co.za>


New submission from Vex Woo:

I'v tested the following steps against python3.5 and python3.6. Python shell hangs on.

$ python3.6
Python 3.6.0 (v3.6.0:41df79263a11, Dec 22 2016, 17:23:13)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import itertools
>>> counter = itertools.count()
>>> next(counter)
0
>>> next(counter)
1
>>> 'count' in counter




^C^C^C^C^C^C^C^C



^C^C^C


^D
^Z
[1]  + 47254 suspended  python3.6
~ ->> fg
[1]  + 47254 continued  python3.6


^C^C^Z
[1]  + 47254 suspended  python3.6

----------
components: Build
messages: 285093
nosy: Nixawk
priority: normal
severity: normal
status: open
title: Python3 interactive shell hangs on
type: behavior
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29222>
_______________________________________

From report at bugs.python.org  Tue Jan 10 01:40:44 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Tue, 10 Jan 2017 06:40:44 +0000
Subject: [issue29222] Python3 interactive shell hangs on
In-Reply-To: <1484029943.51.0.404854509621.issue29222@psf.upfronthosting.co.za>
Message-ID: <1484030444.35.0.96820452503.issue29222@psf.upfronthosting.co.za>


Changes by Serhiy Storchaka <storchaka+cpython at gmail.com>:


----------
nosy: +rhettinger

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29222>
_______________________________________

From report at bugs.python.org  Tue Jan 10 01:54:33 2017
From: report at bugs.python.org (R. David Murray)
Date: Tue, 10 Jan 2017 06:54:33 +0000
Subject: [issue29222] Python3 interactive shell hangs on
In-Reply-To: <1484029943.51.0.404854509621.issue29222@psf.upfronthosting.co.za>
Message-ID: <1484031273.12.0.892138352007.issue29222@psf.upfronthosting.co.za>


R. David Murray added the comment:

This is an instance of the the problem discussed in Issue 26351.

----------
nosy: +r.david.murray
resolution:  -> duplicate
stage:  -> resolved
superseder:  -> Occasionally check for Ctrl-C in long-running operations like sum

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29222>
_______________________________________

From report at bugs.python.org  Tue Jan 10 02:15:32 2017
From: report at bugs.python.org (Benjamin Peterson)
Date: Tue, 10 Jan 2017 07:15:32 +0000
Subject: [issue29215] pyport.h uses non C90-style comment
In-Reply-To: <1483972129.08.0.0486776479108.issue29215@psf.upfronthosting.co.za>
Message-ID: <1484032532.84.0.335317982287.issue29215@psf.upfronthosting.co.za>


Benjamin Peterson added the comment:

C99 is required to build or include Python headers since 3.6. PEP 7 could be clearer. The bullet about C99 is meant to override for 3.6 other directives such as the comment prohibition.

----------
resolution:  -> not a bug
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29215>
_______________________________________

From report at bugs.python.org  Tue Jan 10 02:39:32 2017
From: report at bugs.python.org (Vinay Sajip)
Date: Tue, 10 Jan 2017 07:39:32 +0000
Subject: [issue29220] Python 3.6 regression/change using
 logging.addLevelName() to clear a name
In-Reply-To: <1484016439.6.0.79629021761.issue29220@psf.upfronthosting.co.za>
Message-ID: <1484033972.36.0.750290476011.issue29220@psf.upfronthosting.co.za>


Vinay Sajip added the comment:

This change was due to fixing #27937. Since yours is not a common use case, and since the workaround works for you, I propose not to change things back.

----------
assignee:  -> vinay.sajip
status: open -> pending

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29220>
_______________________________________

From report at bugs.python.org  Tue Jan 10 02:41:04 2017
From: report at bugs.python.org (Eryk Sun)
Date: Tue, 10 Jan 2017 07:41:04 +0000
Subject: [issue29222] Python3 interactive shell hangs on
In-Reply-To: <1484029943.51.0.404854509621.issue29222@psf.upfronthosting.co.za>
Message-ID: <1484034064.7.0.782384286572.issue29222@psf.upfronthosting.co.za>


Eryk Sun added the comment:

As a workaround, the platform's default Ctrl+C handler should allow killing the process:

    >>> signal.signal(signal.SIGINT, signal.SIG_DFL)
    <built-in function default_int_handler>

    >>> counter = itertools.count()
    >>> 'count' in counter
    ^C

But it's nowhere near as useful as a KeyboardInterrupt exception, if something ever gets implemented to resolve issue 26351.

----------
nosy: +eryksun
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29222>
_______________________________________

From report at bugs.python.org  Tue Jan 10 02:44:20 2017
From: report at bugs.python.org (Mark Blakeney)
Date: Tue, 10 Jan 2017 07:44:20 +0000
Subject: [issue29220] Python 3.6 regression/change using
 logging.addLevelName() to clear a name
In-Reply-To: <1484016439.6.0.79629021761.issue29220@psf.upfronthosting.co.za>
Message-ID: <1484034260.73.0.938104023906.issue29220@psf.upfronthosting.co.za>


Mark Blakeney added the comment:

Well my work-around is not ideal as I get an extra embedded space in those messages. I'm surprised this is "not a common use case"? It doesn't really worry me but it is a 3.6 incompatibility change which I would think is always far better to avoid.

----------
status: pending -> open

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29220>
_______________________________________

From report at bugs.python.org  Tue Jan 10 03:48:40 2017
From: report at bugs.python.org (Martin Panter)
Date: Tue, 10 Jan 2017 08:48:40 +0000
Subject: [issue15657] Error in Python 3 docs for PyMethodDef
In-Reply-To: <1344970966.7.0.115664419243.issue15657@psf.upfronthosting.co.za>
Message-ID: <1484038120.3.0.225079902091.issue15657@psf.upfronthosting.co.za>


Martin Panter added the comment:

.
The documentation did not get merged properly into 3.6+. And even in 3.5, under METH_KEYWORDS, I propose to change ?The flag is typically combined with METH_VARARGS? to ?The flag must be combined . . .?.

The remaining issue15657_36.diff patch looks out of date. There is a _PyCFunction_FastCallDict() function that appears to also need adjusting.

But instead of changing the value of METH_KEYWORDS, wouldn?t it be safer to just accept METH_KEYWORDS (= 2) on its own as a valid value? This is what Python 2 does. Essentially, revert the first hunk of

https://hg.python.org/cpython/diff/b7bfa780a882/Objects/methodobject.c

but without METH_OLDARGS, whose value was zero anyway.

BTW, the statement did not need to be removed in Python 2, but IMO it?s fine now without the statment. The statement was added in revision 1564c6839e6b.

----------
nosy: +martin.panter
versions: +Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue15657>
_______________________________________

From report at bugs.python.org  Tue Jan 10 03:59:36 2017
From: report at bugs.python.org (Mark Dickinson)
Date: Tue, 10 Jan 2017 08:59:36 +0000
Subject: [issue29137] Fix fpectl-induced ABI breakage
In-Reply-To: <1483402510.43.0.646850547969.issue29137@psf.upfronthosting.co.za>
Message-ID: <1484038776.91.0.298175654011.issue29137@psf.upfronthosting.co.za>


Changes by Mark Dickinson <dickinsm at gmail.com>:


----------
nosy: +mark.dickinson

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29137>
_______________________________________

From report at bugs.python.org  Tue Jan 10 04:16:10 2017
From: report at bugs.python.org (Armin Rigo)
Date: Tue, 10 Jan 2017 09:16:10 +0000
Subject: [issue28885] Python 3.5.2 strange-behavior issues (from PyPy)
In-Reply-To: <1481025768.53.0.205646799997.issue28885@psf.upfronthosting.co.za>
Message-ID: <1484039770.34.0.207333433758.issue28885@psf.upfronthosting.co.za>


Armin Rigo added the comment:

(S6) 'xxx' % b'foo' == 'xxx'
     b'xxx' % b'foo' raises TypeError

The first case is because PyMapping_Check() is true on b'foo', so it works like 'xxx' % {...}, which always just returns 'xxx'.  The second case is because _PyBytes_Format() contains more special cases, for bytes and bytearray, which are not present in PyUnicode_Format().

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28885>
_______________________________________

From report at bugs.python.org  Tue Jan 10 04:43:55 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Tue, 10 Jan 2017 09:43:55 +0000
Subject: [issue28885] Python 3.5.2 strange-behavior issues (from PyPy)
In-Reply-To: <1481025768.53.0.205646799997.issue28885@psf.upfronthosting.co.za>
Message-ID: <1484041435.25.0.00624444229542.issue28885@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

Armin, it would help if you report all cases as separate issues. Some of them are already resolved, some can be closed with "won't fix" resolution, others need special discussions. This issue can be left as a metaissue.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28885>
_______________________________________

From report at bugs.python.org  Tue Jan 10 05:15:21 2017
From: report at bugs.python.org (Xavier de Gaye)
Date: Tue, 10 Jan 2017 10:15:21 +0000
Subject: [issue28997] test_readline.test_nonascii fails on Android
In-Reply-To: <1481965369.72.0.409169776064.issue28997@psf.upfronthosting.co.za>
Message-ID: <1484043321.18.0.342964146119.issue28997@psf.upfronthosting.co.za>


Xavier de Gaye added the comment:

Trying to track the 'char * text' pointer that is returned to Python by readline in rlhandler() and using gdb 'set follow-fork-mode child' and 'set detach-on-fork off' to stop (with a breakpoint in rlhandler) in the child spawned by run_pty() in test_nonascii, unfortunately this fails with gdb crashing in a debug session on Android. So I use the attached readline.patch with good old printf instead. The results:

(1) UTF-8 locale:
    command: LANG=en_US.UTF-8 python -m test -v -m test_nonascii test_readline
    stderr output:
        char *text: "[?nserted]|t?xt[after]"
(2) C locale
    command: python -m test -v -m test_nonascii test_readline
    stderr output:
        char *text: "[?nsexrted]|t"
(3) C locale and a change in 'script' to not use "\xEB" in 'macro':
    s/macro = "|t\xEB[after]"/macro = "|te[after]"/
    command: python -m test -v -m test_nonascii test_readline
    stderr output:
        char *text: "[?nserted]|tex[after]"

Case (2) confirms your comment in msg283544: "the Ctrl+A macro seems to have only inserted the two ASCII characters "|t" and has stopped at the non-ASCII character "\xEB".
So readline with the C locale on Android:
* Accepts eight-bits characters in rl_pre_input_hook().
* Discards all the characters after the first eight-bits character in macro expansion.
* Discards eight-bits characters on input.
It seems reasonable to consider that readline is broken with the C locale on Android and to skip the test in that case whether using your last patch Martin, or when test.support.is_android is True.


I don't know if this is relevant, but on archlinux (not on debian), when I run bash and therefore readline, with the C locale (on xterm or on the linux console, using my french keyboard) I have the following results:

[xavier at bilboquet ~]$ LANG= bash
[xavier at bilboquet ~]$                   # first prompt
bash: $'\303': command not found
[xavier at bilboquet ~]$ A                 # second prompt
bash: $'\303A\251': command not found
[xavier at bilboquet                       # third prompt
[xavier at bilboquet ~]$


On the first prompt I type '?' followed by the <backspace> key and the <return> key.
On the second prompt I type '?' followed by <left-arrow>, followed by 'A' and <return>.
On the third prompt I type 4 times '?' followed by 8 times <backspace> and <return>, note how the prompt has been eaten by the backspaces.

----------
Added file: http://bugs.python.org/file46237/readline.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28997>
_______________________________________

From report at bugs.python.org  Tue Jan 10 05:21:20 2017
From: report at bugs.python.org (Xavier de Gaye)
Date: Tue, 10 Jan 2017 10:21:20 +0000
Subject: [issue28997] test_readline.test_nonascii fails on Android
In-Reply-To: <1481965369.72.0.409169776064.issue28997@psf.upfronthosting.co.za>
Message-ID: <1484043680.1.0.472260091982.issue28997@psf.upfronthosting.co.za>


Changes by Xavier de Gaye <xdegaye at gmail.com>:


----------
assignee: xdegaye -> 

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28997>
_______________________________________

From report at bugs.python.org  Tue Jan 10 05:51:18 2017
From: report at bugs.python.org (Evan)
Date: Tue, 10 Jan 2017 10:51:18 +0000
Subject: [issue29213] python -m venv activate.bat has weird mix of line endings
In-Reply-To: <1483971138.02.0.292455070123.issue29213@psf.upfronthosting.co.za>
Message-ID: <1484045478.57.0.981386044681.issue29213@psf.upfronthosting.co.za>


Evan added the comment:

The scripts are being read in with mode 'rb', decoded, modified, then written back out with mode 'w'. This means the EOL conversion doesn't happen on the way in, but does happen on the way out, and so '\r\n' becomes '\r\r\n'. One fix would be to always use binary mode and encode the contents back to bytes instead. However since there are no longer any binary files in venv/Scripts/*, perhaps a better fix is to remove the binary handling entirely and always use text mode?

----------
nosy: +evan_, vinay.sajip
versions: +Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29213>
_______________________________________

From report at bugs.python.org  Tue Jan 10 06:40:26 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 10 Jan 2017 11:40:26 +0000
Subject: [issue28870] Reduce stack consumption of
 PyObject_CallFunctionObjArgs() and like
In-Reply-To: <1480891813.94.0.54795440575.issue28870@psf.upfronthosting.co.za>
Message-ID: <1484048426.89.0.270711812709.issue28870@psf.upfronthosting.co.za>


STINNER Victor added the comment:

testcapi_stack_pointer.patch: add _testcapi.stack_pointer() function.

----------
Added file: http://bugs.python.org/file46238/testcapi_stack_pointer.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28870>
_______________________________________

From report at bugs.python.org  Tue Jan 10 06:45:51 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 10 Jan 2017 11:45:51 +0000
Subject: [issue28870] Reduce stack consumption of
 PyObject_CallFunctionObjArgs() and like
In-Reply-To: <1480891813.94.0.54795440575.issue28870@psf.upfronthosting.co.za>
Message-ID: <1484048751.13.0.322532690598.issue28870@psf.upfronthosting.co.za>


STINNER Victor added the comment:

stack_overflow_28870-sp.py: script using testcapi_stack_pointer.patch to compute the usage of the C stack. Results of this script.

(*) Reference

test_python_call: 7175 calls before crash, stack: 1168 bytes/call
test_python_getitem: 6235 calls before crash, stack: 1344 bytes/call
test_python_iterator: 5344 calls before crash, stack: 1568 bytes/call

=> total: 18754 calls, 4080 bytes

(1) no_small_stack.patch

test_python_call: 7175 calls before crash, stack: 1168 bytes/call
test_python_getitem: 6547 calls before crash, stack: 1280 bytes/call
test_python_iterator: 5572 calls before crash, stack: 1504 bytes/call

=> total: 19294 calls, 3952 bytes

test_python_call is clearly not impacted by no_small_stack.patch.

test_python_call loops on method_call():

method_call()
=> _PyObject_Call_Prepend()
=> _PyObject_FastCallDict()
=> _PyFunction_FastCallDict()
=> _PyEval_EvalCodeWithName()
=> PyEval_EvalFrameEx()
=> _PyEval_EvalFrameDefault()
=> call_function()
=> _PyObject_FastCallKeywords()
=> slot_tp_call()
=> PyObject_Call()
=> method_call()
=> (...)

_PyObject_Call_Prepend() is in the middle of the chain. This function uses a "small stack" of _PY_FASTCALL_SMALL_STACK "PyObject*" items. We can clearly see the impact of modifying _PY_FASTCALL_SMALL_STACK on the maximum number of 
test_python_call calls before crash in msg285057.

----------
Added file: http://bugs.python.org/file46239/stack_overflow_28870-sp.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28870>
_______________________________________

From report at bugs.python.org  Tue Jan 10 06:55:08 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 10 Jan 2017 11:55:08 +0000
Subject: [issue28870] Reduce stack consumption of
 PyObject_CallFunctionObjArgs() and like
In-Reply-To: <1480891813.94.0.54795440575.issue28870@psf.upfronthosting.co.za>
Message-ID: <1484049308.0.0.127206848607.issue28870@psf.upfronthosting.co.za>


STINNER Victor added the comment:

no_small_stack-2.patch: Remove all "small_stack" buffers.

Reference

test_python_call: 7175 calls before crash, stack: 1168 bytes/call
test_python_getitem: 6235 calls before crash, stack: 1344 bytes/call
test_python_iterator: 5344 calls before crash, stack: 1568 bytes/call

=> total: 18754 calls, 4080 bytes

no_small_stack.patch

test_python_call: 7482 calls (+307) before crash, stack: 1120 bytes/call (-48)
test_python_getitem: 6715 calls (+480) before crash, stack: 1248 bytes/call (-96)
test_python_iterator: 5693 calls (+349) before crash, stack: 1472 bytes/call (-96)

=> total: 19890 calls (+1136), 3840 bytes (-240)

The total gain is the removal of 5 small buffers of 48 bytes: 240 bytes.

----------
Added file: http://bugs.python.org/file46240/no_small_stack-2.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28870>
_______________________________________

From report at bugs.python.org  Tue Jan 10 07:08:38 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 10 Jan 2017 12:08:38 +0000
Subject: [issue28870] Reduce stack consumption of
 PyObject_CallFunctionObjArgs() and like
In-Reply-To: <1480891813.94.0.54795440575.issue28870@psf.upfronthosting.co.za>
Message-ID: <1484050118.58.0.109393972068.issue28870@psf.upfronthosting.co.za>


STINNER Victor added the comment:

> no_small_stack.patch:

Oops, you should read no_small_stack-2.patch in my previous message ;-)

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28870>
_______________________________________

From report at bugs.python.org  Tue Jan 10 07:15:10 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 10 Jan 2017 12:15:10 +0000
Subject: [issue28870] Reduce stack consumption of
 PyObject_CallFunctionObjArgs() and like
In-Reply-To: <1480891813.94.0.54795440575.issue28870@psf.upfronthosting.co.za>
Message-ID: <1484050510.21.0.465998796976.issue28870@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Python 3.5 (revision 8125d9a8152b), before all fastcall changes:

test_python_call: 8314 calls before crash, stack: 1008 bytes/call
test_python_getitem: 7483 calls before crash, stack: 1120 bytes/call
test_python_iterator: 6802 calls before crash, stack: 1232 bytes/call

=> total: 22599 calls, 3360 bytes

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28870>
_______________________________________

From report at bugs.python.org  Tue Jan 10 07:23:56 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Tue, 10 Jan 2017 12:23:56 +0000
Subject: [issue28870] Reduce stack consumption of
 PyObject_CallFunctionObjArgs() and like
In-Reply-To: <1480891813.94.0.54795440575.issue28870@psf.upfronthosting.co.za>
Message-ID: <1484051036.74.0.117930149916.issue28870@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

What are results with 3.4? There were several issues about stack overflow in 3.5 (issue25222, issue28179, issue28913).

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28870>
_______________________________________

From report at bugs.python.org  Tue Jan 10 08:04:20 2017
From: report at bugs.python.org (INADA Naoki)
Date: Tue, 10 Jan 2017 13:04:20 +0000
Subject: [issue15216] Support setting the encoding on a text stream after
 creation
In-Reply-To: <1340880558.16.0.0136840668667.issue15216@psf.upfronthosting.co.za>
Message-ID: <1484053460.96.0.353998039643.issue15216@psf.upfronthosting.co.za>


INADA Naoki added the comment:

I had not read the patch carefully when updating.
Now I'm reading pure Python part of the patch.

For writing side, this patch is very simple.
For reading side, this patch is very complex, because TextIOWrapper.tell()
and .seek() is complex already.

How about starting this API which works only when `self._decoder is None`?
Regardless acceptance of PEP 538 and PEP 540, this is important to change
stdio encoding.

We already failed to ship this important API in 3.6.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue15216>
_______________________________________

From report at bugs.python.org  Tue Jan 10 08:20:09 2017
From: report at bugs.python.org (INADA Naoki)
Date: Tue, 10 Jan 2017 13:20:09 +0000
Subject: [issue15216] Support setting the encoding on a text stream after
 creation
In-Reply-To: <1340880558.16.0.0136840668667.issue15216@psf.upfronthosting.co.za>
Message-ID: <1484054409.6.0.0112956238648.issue15216@psf.upfronthosting.co.za>


INADA Naoki added the comment:

This patch doesn't change current behavior of "decode by chunk".
It's difficult to support changing encoding via header, like
Python's encoding cookie.

>>> import io
>>> buff = io.BytesIO("hello\n?????".encode('utf-8'))
>>> f = io.TextIOWrapper(buff, encoding='ascii')
>>> f.readline()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/inada-n/work/python/cpython/Lib/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe3 in position 6: ordinal not in range(128)
>>>
>>> buff = io.BytesIO("hello\n?????".encode('utf-8'))
>>> f = io.TextIOWrapper(buff, encoding='ascii')
>>> f.read(6)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/inada-n/work/python/cpython/Lib/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe3 in position 6: ordinal not in range(128)

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue15216>
_______________________________________

From report at bugs.python.org  Tue Jan 10 09:09:34 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 10 Jan 2017 14:09:34 +0000
Subject: [issue28870] Reduce stack consumption of
 PyObject_CallFunctionObjArgs() and like
In-Reply-To: <1484051036.74.0.117930149916.issue28870@psf.upfronthosting.co.za>
Message-ID: <CAMpsgwYd40f8chmnGUCVSaziHVGSz7gjYoWyhn9CF+6T3KdJow@mail.gmail.com>


STINNER Victor added the comment:

Python 3.4 (rev 6340c9fcc111):

test_python_call: 9700 calls before crash, stack: 864 bytes/call
test_python_getitem: 8314 calls before crash, stack: 1008 bytes/call
test_python_iterator: 7818 calls before crash, stack: 1072 bytes/call

=> total: 25832 calls, 2944 bytes

Python 2.7 (rev 0d4e0a736688):

test_python_call: 6162 calls before crash, stack: 1360 bytes/call
test_python_getitem: 5952 calls before crash, stack: 1408 bytes/call
test_python_iterator: 5885 calls before crash, stack: 1424 bytes/call

=> total: 17999 calls, 4192 bytes

Nice. At least, Python 3.7 is better than Python 2.7 (4080 bytes <
4192 bytes) :-) Python 3.4 stack usage was very low, and lower than
Python 3.5.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28870>
_______________________________________

From report at bugs.python.org  Tue Jan 10 09:28:10 2017
From: report at bugs.python.org (George Fischhof)
Date: Tue, 10 Jan 2017 14:28:10 +0000
Subject: [issue29223] Settable defaulting to decimal instead of float
Message-ID: <1484058490.62.0.883575101853.issue29223@psf.upfronthosting.co.za>


New submission from George Fischhof:

Hi There,

Settable defaulting to decimal instead of float

It would be good to be able to use decimal automatically instead of float if there is a setting. For example an environment variable or a flag file.

Where and when accuracy is more important than speed, the user could set this flag, and calculate with decimal numbers as learned in the school.

I think several people would use this function

Best regards,
George

----------
components: Interpreter Core
messages: 285114
nosy: georgefischhof
priority: normal
severity: normal
status: open
title: Settable defaulting to decimal instead of float
type: enhancement

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29223>
_______________________________________

From report at bugs.python.org  Tue Jan 10 09:36:47 2017
From: report at bugs.python.org (George Fischhof)
Date: Tue, 10 Jan 2017 14:36:47 +0000
Subject: [issue29224] OS related file operations (copy, move, delete,
 rename...) should be placed into one module
Message-ID: <1484059007.13.0.312635543812.issue29224@psf.upfronthosting.co.za>


New submission from George Fischhof:

Hi There,

OS related file operations (copy, move, delete, rename...) should be placed into one module...
As it quite confusing that they are in two moduls (os and shutil). 

I have read that one is higher level than other, but actually to use them I have to think which function can be found in which module. 

It is confuse for beginners, and makes the usage more complex instead of make it more simple (as Zen of Python says ;-) )

An alias could good, not to cause incompatibility.

Best regards,
George

----------
components: Library (Lib)
messages: 285115
nosy: georgefischhof
priority: normal
severity: normal
status: open
title: OS related file operations (copy, move, delete, rename...) should be placed into one module
type: enhancement

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29224>
_______________________________________

From report at bugs.python.org  Tue Jan 10 09:40:13 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 10 Jan 2017 14:40:13 +0000
Subject: [issue29224] OS related file operations (copy, move, delete,
 rename...) should be placed into one module
In-Reply-To: <1484059007.13.0.312635543812.issue29224@psf.upfronthosting.co.za>
Message-ID: <1484059213.22.0.498842333696.issue29224@psf.upfronthosting.co.za>


STINNER Victor added the comment:

You can design your own "oshutil" module which combines both modules.

They are technical reasons to have two separated modules.

----------
nosy: +haypo

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29224>
_______________________________________

From report at bugs.python.org  Tue Jan 10 09:46:43 2017
From: report at bugs.python.org (Jim Fasarakis-Hilliard)
Date: Tue, 10 Jan 2017 14:46:43 +0000
Subject: [issue29223] Settable defaulting to decimal instead of float
In-Reply-To: <1484058490.62.0.883575101853.issue29223@psf.upfronthosting.co.za>
Message-ID: <1484059603.54.0.202040673434.issue29223@psf.upfronthosting.co.za>


Jim Fasarakis-Hilliard added the comment:

This issue (and your other issue29223) would probably be a better for the python-ideas mailing list rather than the bug tracker.

See the FAQ in the Developer Guide for Python https://docs.python.org/devguide/faq.html#suggesting-changes

Both issues should probably be closed.

----------
nosy: +Jim Fasarakis-Hilliard

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29223>
_______________________________________

From report at bugs.python.org  Tue Jan 10 09:49:12 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 10 Jan 2017 14:49:12 +0000
Subject: [issue29223] Settable defaulting to decimal instead of float
In-Reply-To: <1484058490.62.0.883575101853.issue29223@psf.upfronthosting.co.za>
Message-ID: <1484059752.65.0.468700125605.issue29223@psf.upfronthosting.co.za>


STINNER Victor added the comment:

You are not the first one to propose the idea.

2012: "make decimal the default non-integer instead of float?"
https://mail.python.org/pipermail/python-ideas/2012-September/016250.html

2014: "Python Numbers as Human Concept Decimal System"
https://mail.python.org/pipermail/python-ideas/2014-March/026436.html


Related discussions:

2008: "Decimal literal?"
https://mail.python.org/pipermail/python-ideas/2008-December/002379.html

2015: "Python Float Update"
https://mail.python.org/pipermail/python-ideas/2015-June/033787.html

PEP 240 "Adding a Rational Literal to Python".

----------
nosy: +haypo

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29223>
_______________________________________

From report at bugs.python.org  Tue Jan 10 09:50:07 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 10 Jan 2017 14:50:07 +0000
Subject: [issue29223] Settable defaulting to decimal instead of float
In-Reply-To: <1484058490.62.0.883575101853.issue29223@psf.upfronthosting.co.za>
Message-ID: <1484059807.93.0.253561804858.issue29223@psf.upfronthosting.co.za>


STINNER Victor added the comment:

> This issue (and your other issue29223) ...

I guess that you mean issue #29224 "OS related file operations (copy, move, delete, rename...) should be placed into one module".

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29223>
_______________________________________

From report at bugs.python.org  Tue Jan 10 09:50:54 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 10 Jan 2017 14:50:54 +0000
Subject: [issue29224] OS related file operations (copy, move, delete,
 rename...) should be placed into one module
In-Reply-To: <1484059007.13.0.312635543812.issue29224@psf.upfronthosting.co.za>
Message-ID: <1484059854.26.0.0618793856083.issue29224@psf.upfronthosting.co.za>


STINNER Victor added the comment:

As Jim Fasarakis-Hilliard wrote in msg285117, the *bug* tracker is not the best place to propose changes, so I close this issue.

----------
resolution:  -> not a bug
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29224>
_______________________________________

From report at bugs.python.org  Tue Jan 10 09:51:26 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 10 Jan 2017 14:51:26 +0000
Subject: [issue29223] Settable defaulting to decimal instead of float
In-Reply-To: <1484058490.62.0.883575101853.issue29223@psf.upfronthosting.co.za>
Message-ID: <1484059886.05.0.39269034344.issue29223@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Jim is right, I close the bug. The *bug* tracker is not the best place to propose ideas: please use the python-ideas mailing list ;-)

----------
resolution:  -> not a bug
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29223>
_______________________________________

From report at bugs.python.org  Tue Jan 10 10:03:59 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 10 Jan 2017 15:03:59 +0000
Subject: [issue28870] Reduce stack consumption of
 PyObject_CallFunctionObjArgs() and like
In-Reply-To: <1480891813.94.0.54795440575.issue28870@psf.upfronthosting.co.za>
Message-ID: <1484060639.71.0.672568264809.issue28870@psf.upfronthosting.co.za>


STINNER Victor added the comment:

no_small_stack-2.patch has a very bad impact on performances:

haypo at speed-python$ python3 -m perf compare_to 2017-01-04_12-02-default-ee1390c9b585.json no_small_stack-2_refee1390c9b585.json -G --min-speed=5

Slower (59):
- telco: 15.7 ms +- 0.5 ms -> 23.4 ms +- 0.3 ms: 1.49x slower (+49%)
- scimark_sor: 393 ms +- 6 ms -> 579 ms +- 10 ms: 1.47x slower (+47%)
- json_loads: 56.9 us +- 0.9 us -> 83.1 us +- 2.4 us: 1.46x slower (+46%)
- unpickle_pure_python: 698 us +- 10 us -> 984 us +- 10 us: 1.41x slower (+41%)
- scimark_lu: 424 ms +- 22 ms -> 585 ms +- 33 ms: 1.38x slower (+38%)
- chameleon: 22.4 ms +- 0.2 ms -> 30.8 ms +- 0.3 ms: 1.38x slower (+38%)
- xml_etree_generate: 212 ms +- 3 ms -> 291 ms +- 4 ms: 1.37x slower (+37%)
- xml_etree_process: 177 ms +- 3 ms -> 240 ms +- 3 ms: 1.35x slower (+35%)
- raytrace: 1.04 sec +- 0.01 sec -> 1.40 sec +- 0.02 sec: 1.35x slower (+35%)
- logging_simple: 27.9 us +- 0.4 us -> 37.4 us +- 0.5 us: 1.34x slower (+34%)
- pickle_pure_python: 1.02 ms +- 0.01 ms -> 1.37 ms +- 0.02 ms: 1.34x slower (+34%)
- logging_format: 33.3 us +- 0.4 us -> 44.5 us +- 0.7 us: 1.34x slower (+34%)
- xml_etree_iterparse: 195 ms +- 5 ms -> 259 ms +- 7 ms: 1.32x slower (+32%)
- chaos: 236 ms +- 3 ms -> 306 ms +- 3 ms: 1.30x slower (+30%)
- regex_compile: 380 ms +- 3 ms -> 494 ms +- 5 ms: 1.30x slower (+30%)
- pathlib: 42.3 ms +- 0.5 ms -> 55.0 ms +- 0.6 ms: 1.30x slower (+30%)
- django_template: 364 ms +- 5 ms -> 471 ms +- 4 ms: 1.29x slower (+29%)
- call_method: 11.2 ms +- 0.2 ms -> 14.4 ms +- 0.2 ms: 1.29x slower (+29%)
- hexiom: 18.4 ms +- 0.2 ms -> 23.7 ms +- 0.2 ms: 1.29x slower (+29%)
- call_method_slots: 11.0 ms +- 0.3 ms -> 14.1 ms +- 0.1 ms: 1.28x slower (+28%)
- richards: 147 ms +- 4 ms -> 188 ms +- 5 ms: 1.28x slower (+28%)
- html5lib: 207 ms +- 7 ms -> 262 ms +- 6 ms: 1.27x slower (+27%)
- genshi_text: 71.5 ms +- 1.3 ms -> 90.3 ms +- 1.1 ms: 1.26x slower (+26%)
- deltablue: 14.2 ms +- 0.2 ms -> 17.9 ms +- 0.4 ms: 1.26x slower (+26%)
- genshi_xml: 164 ms +- 2 ms -> 207 ms +- 3 ms: 1.26x slower (+26%)
- sympy_str: 429 ms +- 5 ms -> 539 ms +- 4 ms: 1.25x slower (+25%)
- go: 493 ms +- 5 ms -> 619 ms +- 7 ms: 1.25x slower (+25%)
- mako: 35.4 ms +- 1.5 ms -> 44.2 ms +- 1.2 ms: 1.25x slower (+25%)
- sympy_expand: 959 ms +- 10 ms -> 1.19 sec +- 0.01 sec: 1.24x slower (+24%)
- nqueens: 215 ms +- 2 ms -> 268 ms +- 1 ms: 1.24x slower (+24%)
(...)

Benchmark ran on speed-python with PGO+LTO, Linux configured for benchmarks using python3 -m perf system tune.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28870>
_______________________________________

From report at bugs.python.org  Tue Jan 10 10:05:25 2017
From: report at bugs.python.org (Eryk Sun)
Date: Tue, 10 Jan 2017 15:05:25 +0000
Subject: [issue29224] OS related file operations (copy, move, delete,
 rename...) should be placed into one module
In-Reply-To: <1484059007.13.0.312635543812.issue29224@psf.upfronthosting.co.za>
Message-ID: <1484060725.72.0.975055506617.issue29224@psf.upfronthosting.co.za>


Changes by Eryk Sun <eryksun at gmail.com>:


----------
stage:  -> resolved

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29224>
_______________________________________

From report at bugs.python.org  Tue Jan 10 10:06:19 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Tue, 10 Jan 2017 15:06:19 +0000
Subject: [issue28870] Reduce stack consumption of
 PyObject_CallFunctionObjArgs() and like
In-Reply-To: <1480891813.94.0.54795440575.issue28870@psf.upfronthosting.co.za>
Message-ID: <1484060779.96.0.570043482988.issue28870@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

Thus Python 3.6 stack usage is about 20% larger than Python 3.5 and about 40% larger than Python 3.4. This is significant. :-(

no_small_stack-2.patch decreases it only by 6% (with possible performance loss).

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28870>
_______________________________________

From report at bugs.python.org  Tue Jan 10 10:09:44 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 10 Jan 2017 15:09:44 +0000
Subject: [issue28870] Reduce stack consumption of
 PyObject_CallFunctionObjArgs() and like
In-Reply-To: <1480891813.94.0.54795440575.issue28870@psf.upfronthosting.co.za>
Message-ID: <1484060984.55.0.86574305826.issue28870@psf.upfronthosting.co.za>


STINNER Victor added the comment:

> no_small_stack-2.patch decreases it only by 6% (with possible performance loss).

Yeah, if we want to come back to Python 3.4 efficiency, we need to find the other functions which now uses more stack memory ;-) The discussed "small stack" buffers are only responsible of 96 bytes, not a big deal compared to the total of 4080 bytes.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28870>
_______________________________________

From report at bugs.python.org  Tue Jan 10 10:13:52 2017
From: report at bugs.python.org (Dustin Spicuzza)
Date: Tue, 10 Jan 2017 15:13:52 +0000
Subject: [issue29208] BlockingIOError during system startup
In-Reply-To: <1483894212.45.0.948118557952.issue29208@psf.upfronthosting.co.za>
Message-ID: <1484061232.46.0.630524945713.issue29208@psf.upfronthosting.co.za>


Dustin Spicuzza added the comment:

I'm able to confirm that the patch does indeed fix the problem. Thanks everyone!

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29208>
_______________________________________

From report at bugs.python.org  Tue Jan 10 10:31:20 2017
From: report at bugs.python.org (jan matejek)
Date: Tue, 10 Jan 2017 15:31:20 +0000
Subject: [issue1294959] Problems with /usr/lib64 builds.
Message-ID: <1484062280.18.0.0274329406369.issue1294959@psf.upfronthosting.co.za>


jan matejek added the comment:

at this again, when porting SUSE patches to 3.6.0 :) ( :( )

Last time there was a discussion, Barry suggested using sysconfig variables to find the proper libdir. Trouble is, to fill out the variables, sysconfig itself uses two sources:
a) compiled-in information from the binary, and
b) Makefile items
Makefile seems an obvious location for storing the libdir info, except, well, to FIND the makefile in the first place, we are using sysconfig variables.... you see where this is heading.

So, given that sysconfig is the correct Primary Source for libdir info, we have three options to get it there:
1. compile it into the binary. sys.implementation.platlibdir seems as good as any -- i'm not completely convinced that this is a property of the _implementation_, but, well.
2. modify sysconfig.py at build time
3. guess it from the environment

And options (2) and (3) seem wrong, so i'm going with (1) for now

----------
versions: +Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue1294959>
_______________________________________

From report at bugs.python.org  Tue Jan 10 10:37:15 2017
From: report at bugs.python.org (Elvis Stansvik)
Date: Tue, 10 Jan 2017 15:37:15 +0000
Subject: [issue29225] distutils.command.install_lib.get_outputs() wrong with
 extensions built inplace
Message-ID: <1484062635.84.0.0152072002193.issue29225@psf.upfronthosting.co.za>


New submission from Elvis Stansvik:

I noticed the following strange behavior:

estan at newton:~/testdist$ find
.
./testext.c
./setup.cfg
./setup.py
estan at newton:~/testdist$ cat setup.py 
from distutils.core import setup, Extension

setup(
    name='testdist',
    version='1.0.0',
    ext_modules = [Extension('testext', ['testext.c'])]
)
estan at newton:~/testdist$ cat setup.cfg
[build_ext]
inplace=1
estan at newton:~/testdist$ ~/cpython/python setup.py install --prefix=~/prefix --record record.txt 
running install
running build
running build_ext
building 'testext' extension
creating build
creating build/temp.linux-x86_64-3.7-pydebug
gcc -pthread -Wno-unused-result -Wsign-compare -g -Og -Wall -Wstrict-prototypes -fPIC -I/home/estan/cpython/Include -I/home/estan/cpython -c testext.c -o build/temp.linux-x86_64-3.7-pydebug/testext.o
gcc -pthread -shared build/temp.linux-x86_64-3.7-pydebug/testext.o -o /home/estan/testdist/testext.cpython-37dm-x86_64-linux-gnu.so
running install_lib
warning: install_lib: 'build/lib.linux-x86_64-3.7-pydebug' does not exist -- no Python modules to install

running install_egg_info
Creating /home/estan/prefix/lib/python3.7/site-packages/
Writing /home/estan/prefix/lib/python3.7/site-packages/testdist-1.0.0-py3.7.egg-info
writing list of installed files to 'record.txt'
estan at newton:~/testdist$ cat record.txt 
/home/estan/prefix/lib/python3.7/site-packages/n-37dm-x86_64-linux-gnu.so
/home/estan/prefix/lib/python3.7/site-packages/testdist-1.0.0-py3.7.egg-info
estan at newton:~/testdist$ find ~/prefix
/home/estan/prefix
/home/estan/prefix/lib
/home/estan/prefix/lib/python3.7
/home/estan/prefix/lib/python3.7/site-packages
/home/estan/prefix/lib/python3.7/site-packages/testdist-1.0.0-py3.7.egg-info
estan at newton:~/testdist$

Notice how in the written record.txt, the path to the extension is wrong ("testext.cpytho" is missing), and the extension is not installed.

I did a little digging and found that the reason is install_lib.get_outputs() returns the wrong result for extensions that are built inplace: It assumes they are built in the build directory.

The attached patch fixes the problem. The failure of the included test case without the change to install_lib is:

======================================================================
FAIL: test_get_outputs_inplace_ext (distutils.tests.test_install_lib.InstallLibTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/estan/cpython/Lib/distutils/tests/test_install_lib.py", line 91, in test_get_outputs_inplace_ext
    self.assertEqual(expected, actual)
AssertionError: Lists differ: ['/tmp/tmptt4nhzgi/foo.cpython-37dm-x86_64-linux-gnu.so'] != ['/tmp/tmptt4nhzgi/dm-x86_64-linux-gnu.so']

First differing element 0:
'/tmp/tmptt4nhzgi/foo.cpython-37dm-x86_64-linux-gnu.so'
'/tmp/tmptt4nhzgi/dm-x86_64-linux-gnu.so'

- ['/tmp/tmptt4nhzgi/foo.cpython-37dm-x86_64-linux-gnu.so']
?                    --------------

+ ['/tmp/tmptt4nhzgi/dm-x86_64-linux-gnu.so']

----------------------------------------------------------------------

Notice the missing "foo.cpython-37". With the change to install_lib, the test passes.

----------
components: Distutils
messages: 285127
nosy: Elvis Stansvik, dstufft, eric.araujo
priority: normal
severity: normal
status: open
title: distutils.command.install_lib.get_outputs() wrong with extensions built inplace
type: behavior

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29225>
_______________________________________

From report at bugs.python.org  Tue Jan 10 10:42:17 2017
From: report at bugs.python.org (Elvis Stansvik)
Date: Tue, 10 Jan 2017 15:42:17 +0000
Subject: [issue29225] distutils.command.install_lib.get_outputs() wrong with
 extensions built inplace
In-Reply-To: <1484062635.84.0.0152072002193.issue29225@psf.upfronthosting.co.za>
Message-ID: <1484062937.07.0.516459985395.issue29225@psf.upfronthosting.co.za>


Changes by Elvis Stansvik <elvstone at gmail.com>:


----------
keywords: +patch
Added file: http://bugs.python.org/file46241/fix-install-lib-with-inplace-ext.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29225>
_______________________________________

From report at bugs.python.org  Tue Jan 10 10:46:46 2017
From: report at bugs.python.org (Mark Lawrence)
Date: Tue, 10 Jan 2017 15:46:46 +0000
Subject: [issue1294959] Problems with /usr/lib64 builds.
Message-ID: <1484063206.01.0.145298064148.issue1294959@psf.upfronthosting.co.za>


Changes by Mark Lawrence <breamoreboy at gmail.com>:


----------
nosy:  -BreamoreBoy

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue1294959>
_______________________________________

From report at bugs.python.org  Tue Jan 10 10:48:36 2017
From: report at bugs.python.org (Elvis Stansvik)
Date: Tue, 10 Jan 2017 15:48:36 +0000
Subject: [issue29225] distutils.command.install_lib.get_outputs() wrong with
 extensions built inplace
In-Reply-To: <1484062635.84.0.0152072002193.issue29225@psf.upfronthosting.co.za>
Message-ID: <1484063316.71.0.739841247769.issue29225@psf.upfronthosting.co.za>


Changes by Elvis Stansvik <elvstone at gmail.com>:


----------
versions: +Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29225>
_______________________________________

From report at bugs.python.org  Tue Jan 10 11:02:07 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 10 Jan 2017 16:02:07 +0000
Subject: [issue28870] Reduce stack consumption of
 PyObject_CallFunctionObjArgs() and like
In-Reply-To: <1480891813.94.0.54795440575.issue28870@psf.upfronthosting.co.za>
Message-ID: <1484064127.42.0.262310218668.issue28870@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Stack used by each C function of test_python_call.

3.4:

(a) method_call: 64

(b) PyObject_Call: 48
(b) function_call: 160
(b) PyEval_EvalCodeEx: 176

(c) PyEval_EvalFrameEx: 256
(c) call_function: 0
(c) do_call: 0
(c) PyObject_Call: 48

(d) slot_tp_call: 64
(d) PyObject_Call: 48

=> total: 864


default:

(a) method_call: 80

(b) _PyObject_FastCallDict: 64
(b) _PyFunction_FastCallDict: 208
(b) _PyEval_EvalCodeWithName: 176

(c) _PyEval_EvalFrameDefault: 320
(c) call_function: 80
(c) _PyObject_FastCallKeywords: 80

(d) slot_tp_call: 64
(d) PyObject_Call: 48

=> total: 1120


Groups of functions, 3.4 => default:

(a) 64 => 80 (+16)
(b) 384 => 448 (+64)
(c) 304 => 480 (+176)
(d) 112 => 112 (=)


I used gdb:

(gdb) set $last=0
(gdb) define size
> print $last - (uintptr_t)$rsp
> set $last = (uintptr_t)$rsp
> down
(gdb) up
(gdb) up
(gdb) up
(... until a first method_call ...)
(gdb) size
(gdb) size
...

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28870>
_______________________________________

From report at bugs.python.org  Tue Jan 10 11:40:05 2017
From: report at bugs.python.org (Simon Grantham)
Date: Tue, 10 Jan 2017 16:40:05 +0000
Subject: [issue29226] Comment generates syntax error
Message-ID: <1484066405.87.0.420039297563.issue29226@psf.upfronthosting.co.za>


New submission from Simon Grantham:

Placing the word "coding:" in a hash tag comment in a file causes a syntax error.  Eg:

~ $ cat tst.py

# coding: Wow! How odd!

~ $ python tst.py
  File "tst.py", line 2
SyntaxError: encoding problem: Wow
~ $

----------
components: Interpreter Core
messages: 285129
nosy: sgrantham
priority: normal
severity: normal
status: open
title: Comment generates syntax error
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29226>
_______________________________________

From report at bugs.python.org  Tue Jan 10 11:40:42 2017
From: report at bugs.python.org (Simon Grantham)
Date: Tue, 10 Jan 2017 16:40:42 +0000
Subject: [issue29226] Comment generates syntax error
In-Reply-To: <1484066405.87.0.420039297563.issue29226@psf.upfronthosting.co.za>
Message-ID: <1484066442.59.0.0780014353693.issue29226@psf.upfronthosting.co.za>


Changes by Simon Grantham <sgrantham at walled.net>:


----------
type:  -> compile error

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29226>
_______________________________________

From report at bugs.python.org  Tue Jan 10 11:44:30 2017
From: report at bugs.python.org (Ammar Askar)
Date: Tue, 10 Jan 2017 16:44:30 +0000
Subject: [issue29226] Comment generates syntax error
In-Reply-To: <1484066405.87.0.420039297563.issue29226@psf.upfronthosting.co.za>
Message-ID: <1484066670.64.0.340725373057.issue29226@psf.upfronthosting.co.za>


Ammar Askar added the comment:

This is because the "coding:" in a comment is a special syntax used to define the encoding of the file. The only documentation I could find for it on the official docs is a brief mention here: https://docs.python.org/3/howto/unicode.html#the-string-type

More details can be found on the PEP for it:
https://www.python.org/dev/peps/pep-0263/

The SyntaxError itself is not a bug, though I think the documentation for this feature is certainly a little sparse.

----------
nosy: +ammar2
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29226>
_______________________________________

From report at bugs.python.org  Tue Jan 10 12:01:35 2017
From: report at bugs.python.org (Rocco Matano)
Date: Tue, 10 Jan 2017 17:01:35 +0000
Subject: [issue22107] tempfile module misinterprets access denied error on
 Windows
In-Reply-To: <1406724041.52.0.0365391579019.issue22107@psf.upfronthosting.co.za>
Message-ID: <1484067695.51.0.696976968271.issue22107@psf.upfronthosting.co.za>


Rocco Matano added the comment:

I just experienced the described problem using Python 3.6.0 (Python 3.6.0 (v3.6.0:41df79263a11, Dec 23 2016, 08:06:12) [MSC v.1900 64 bit (AMD64)] on win32), but i do not understand the current status of this issue: On the one hand it is marked as 'open', which seems to imply that is still not resolved. On the other hand the resolution was set to 'fixed' in May 2015. Should i open a new issue for Python 3.6?

----------
nosy: +rocco.matano
versions: +Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue22107>
_______________________________________

From report at bugs.python.org  Tue Jan 10 12:03:32 2017
From: report at bugs.python.org (Armin Rigo)
Date: Tue, 10 Jan 2017 17:03:32 +0000
Subject: [issue28885] Python 3.5.2 strange-behavior issues (from PyPy)
In-Reply-To: <1481025768.53.0.205646799997.issue28885@psf.upfronthosting.co.za>
Message-ID: <1484067812.7.0.867461558161.issue28885@psf.upfronthosting.co.za>


Armin Rigo added the comment:

> Armin, it would help if you report all cases as separate issues.

I asked on python-dev before creating these three issues, and got the opposite answer.  If you decide it was a bad idea after all, I will open separate issues in the future.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28885>
_______________________________________

From report at bugs.python.org  Tue Jan 10 12:10:45 2017
From: report at bugs.python.org (Jim Fasarakis-Hilliard)
Date: Tue, 10 Jan 2017 17:10:45 +0000
Subject: [issue29226] encoding comment generates a SyntaxError
In-Reply-To: <1484066405.87.0.420039297563.issue29226@psf.upfronthosting.co.za>
Message-ID: <1484068245.53.0.896703099559.issue29226@psf.upfronthosting.co.za>


Jim Fasarakis-Hilliard added the comment:

If you use an encoding declaration the encoding must be recognized by Python. 

This is clearly stated in the documentation: https://docs.python.org/3/reference/lexical_analysis.html#encoding-declarations

----------
nosy: +Jim Fasarakis-Hilliard
title: Comment generates syntax error -> encoding comment generates a SyntaxError

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29226>
_______________________________________

From report at bugs.python.org  Tue Jan 10 12:12:08 2017
From: report at bugs.python.org (Simon Grantham)
Date: Tue, 10 Jan 2017 17:12:08 +0000
Subject: [issue29226] Comment generates syntax error
In-Reply-To: <1484066670.64.0.340725373057.issue29226@psf.upfronthosting.co.za>
Message-ID: <4d9a8ccd-df7c-f7c8-edee-0c4a38fd1fe3@walled.net>


Simon Grantham added the comment:

Thanks Ammar.  Curiously, the comment I had put in my code was a note 
regarding usage and actually didn't contain the word "coding:" but the 
word "encoding:".

# curl -v --header "Transfer-Encoding: chunked" -d @somefile.txt 
"http://localhost:80/"

Perhaps this is a slightly different kind of bug in that the interpreter 
doesn't regex /[^a-zA-Z]coding:/ .  Or a documentation bug. ;)

Simon

On 1/10/2017 11:44 AM, Ammar Askar wrote:
> Ammar Askar added the comment:
>
> This is because the "coding:" in a comment is a special syntax used to define the encoding of the file. The only documentation I could find for it on the official docs is a brief mention here: https://docs.python.org/3/howto/unicode.html#the-string-type
>
> More details can be found on the PEP for it:
> https://www.python.org/dev/peps/pep-0263/
>
> The SyntaxError itself is not a bug, though I think the documentation for this feature is certainly a little sparse.
>
> ----------
> nosy: +ammar2
> resolution:  -> not a bug
> stage:  -> resolved
> status: open -> closed
>
> _______________________________________
> Python tracker <report at bugs.python.org>
> <http://bugs.python.org/issue29226>
> _______________________________________

----------
title: encoding comment generates a SyntaxError -> Comment generates syntax error

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29226>
_______________________________________

From report at bugs.python.org  Tue Jan 10 12:12:25 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Tue, 10 Jan 2017 17:12:25 +0000
Subject: [issue22107] tempfile module misinterprets access denied error on
 Windows
In-Reply-To: <1406724041.52.0.0365391579019.issue22107@psf.upfronthosting.co.za>
Message-ID: <1484068345.31.0.31361529268.issue22107@psf.upfronthosting.co.za>


Changes by Serhiy Storchaka <storchaka+cpython at gmail.com>:


----------
components: +Library (Lib)
resolution: fixed -> 
stage: resolved -> 
versions: +Python 3.7 -Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue22107>
_______________________________________

From report at bugs.python.org  Tue Jan 10 12:50:46 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 10 Jan 2017 17:50:46 +0000
Subject: [issue29227] Reduce C stack consumption in function calls
Message-ID: <1484070646.73.0.530578838486.issue29227@psf.upfronthosting.co.za>


New submission from STINNER Victor:

Attached patch reduce C stack consumption in function calls. It's the follow-up of the issue #28870.


Reference (rev a30cdf366c02):

test_python_call: 7175 calls before crash, stack: 1168 bytes/call
test_python_getitem: 6235 calls before crash, stack: 1344 bytes/call
test_python_iterator: 5344 calls before crash, stack: 1568 bytes/call

=> total: 18754 calls, 4080 bytes


With "Inline call_function() in ceval.c":

test_python_call: 7936 calls before crash, stack: 1056 bytes/call
test_python_getitem: 6387 calls before crash, stack: 1312 bytes/call
test_python_iterator: 5755 calls before crash, stack: 1456 bytes/call

=> total: 20078 calls, 3824 bytes


With inline and "_PY_FASTCALL_SMALL_STACK: 5 arg (40 B) => 3 arg (24 B)":

test_python_call: 8058 calls before crash, stack: 1040 bytes/call
test_python_getitem: 6630 calls before crash, stack: 1264 bytes/call
test_python_iterator: 5952 calls before crash, stack: 1408 bytes/call

=> total: 20640 calls, 3712 bytes


I applied testcapi_stack_pointer.patch and run stack_overflow_28870-sp.py of the issue #28870 to produce these statistics.

With the patch, Python 3.7 is still not as good as Python 3.5 (msg285109), but it's a first enhancement.

----------
components: Interpreter Core
files: less_stack.patch
keywords: patch
messages: 285135
nosy: haypo, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Reduce C stack consumption in function calls
type: performance
versions: Python 3.7
Added file: http://bugs.python.org/file46242/less_stack.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29227>
_______________________________________

From report at bugs.python.org  Tue Jan 10 12:51:18 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 10 Jan 2017 17:51:18 +0000
Subject: [issue28870] Reduce stack consumption of
 PyObject_CallFunctionObjArgs() and like
In-Reply-To: <1480891813.94.0.54795440575.issue28870@psf.upfronthosting.co.za>
Message-ID: <1484070678.86.0.749009435605.issue28870@psf.upfronthosting.co.za>


STINNER Victor added the comment:

I created the issue #29227 "Reduce C stack consumption in function calls" which contains a first simple patch with a significant effect on the C stack.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28870>
_______________________________________

From report at bugs.python.org  Tue Jan 10 12:57:16 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 10 Jan 2017 17:57:16 +0000
Subject: [issue28870] Reduce stack consumption of
 PyObject_CallFunctionObjArgs() and like
In-Reply-To: <1480891813.94.0.54795440575.issue28870@psf.upfronthosting.co.za>
Message-ID: <1484071036.23.0.618149145832.issue28870@psf.upfronthosting.co.za>


STINNER Victor added the comment:

It seems like subfunc.patch approach using the "no inline" attribute helps.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28870>
_______________________________________

From report at bugs.python.org  Tue Jan 10 12:58:17 2017
From: report at bugs.python.org (Ammar Askar)
Date: Tue, 10 Jan 2017 17:58:17 +0000
Subject: [issue29226] Comment generates syntax error
In-Reply-To: <1484066405.87.0.420039297563.issue29226@psf.upfronthosting.co.za>
Message-ID: <1484071097.33.0.954327019688.issue29226@psf.upfronthosting.co.za>


Ammar Askar added the comment:

Thanks for the link Jim, looks like my googling skills are a bit amiss.

I believe the reason the matching allows "encoding" is because a recommended format in the documentation is:

# vim:fileencoding=<encoding-name>

I think changing the behavior of the matching to be stricter and only match on "coding:" could result in breakages of such lines. Instead I think the error thrown could be more descriptive since I don't think it's too helpful.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29226>
_______________________________________

From report at bugs.python.org  Tue Jan 10 13:06:51 2017
From: report at bugs.python.org (Elvis Stansvik)
Date: Tue, 10 Jan 2017 18:06:51 +0000
Subject: [issue29225] distutils.command.install_lib.get_outputs() wrong with
 extensions built inplace
In-Reply-To: <1484062635.84.0.0152072002193.issue29225@psf.upfronthosting.co.za>
Message-ID: <1484071611.11.0.715056497855.issue29225@psf.upfronthosting.co.za>


Elvis Stansvik added the comment:

Not really related to this fix, but a colleague also prudently pointed out that perhaps an

   assert file[:prefix_len] == build_dir + os.sep

before the

   outputs.append(os.path.join(self.install_dir,
    file[prefix_len:]))

in the helper for pure modules (_pure_outputs after my patch is applied) would be in order, to assert that what is cut away is what was intended. 

That sounds like a good idea to me, but could it give false negatives on Windows (case-insensitive file systems? or file systems where both / and \ are allowed?).

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29225>
_______________________________________

From report at bugs.python.org  Tue Jan 10 13:08:34 2017
From: report at bugs.python.org (Simon Grantham)
Date: Tue, 10 Jan 2017 18:08:34 +0000
Subject: [issue29226] Comment generates syntax error
In-Reply-To: <1484071097.33.0.954327019688.issue29226@psf.upfronthosting.co.za>
Message-ID: <5f7220e5-6356-5ca8-e90a-77acb43776dc@walled.net>


Simon Grantham added the comment:

My googling skills are a bit amiss too.  I searched for some kind of 
encoding pragma before reporting.  Perhaps just a footnote in the basic 
documentation of a python comment is the easiest solution.

Simon

On 1/10/2017 12:58 PM, Ammar Askar wrote:
> Ammar Askar added the comment:
>
> Thanks for the link Jim, looks like my googling skills are a bit amiss.
>
> I believe the reason the matching allows "encoding" is because a recommended format in the documentation is:
>
> # vim:fileencoding=<encoding-name>
>
> I think changing the behavior of the matching to be stricter and only match on "coding:" could result in breakages of such lines. Instead I think the error thrown could be more descriptive since I don't think it's too helpful.
>
> ----------
>
> _______________________________________
> Python tracker <report at bugs.python.org>
> <http://bugs.python.org/issue29226>
> _______________________________________

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29226>
_______________________________________

From report at bugs.python.org  Tue Jan 10 13:19:26 2017
From: report at bugs.python.org (yadayada)
Date: Tue, 10 Jan 2017 18:19:26 +0000
Subject: [issue29228] sqlite3 OperationalError on changing into WAL
 transaction mode
Message-ID: <1484072366.39.0.529572648914.issue29228@psf.upfronthosting.co.za>


New submission from yadayada:

Running

sqlite3.connect('foo.db').execute("PRAGMA journal_mode = WAL")

will throw

sqlite3.OperationalError: cannot change into wal mode from within a transaction

on Python 3.6.0, but not with older versions.

----------
messages: 285141
nosy: yadayada
priority: normal
severity: normal
status: open
title: sqlite3 OperationalError on changing into WAL transaction mode
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29228>
_______________________________________

From report at bugs.python.org  Tue Jan 10 13:33:37 2017
From: report at bugs.python.org (Berker Peksag)
Date: Tue, 10 Jan 2017 18:33:37 +0000
Subject: [issue29228] sqlite3 OperationalError on changing into WAL
 transaction mode
In-Reply-To: <1484072366.39.0.529572648914.issue29228@psf.upfronthosting.co.za>
Message-ID: <1484073217.75.0.476443553716.issue29228@psf.upfronthosting.co.za>


Berker Peksag added the comment:

Thanks for the report. This is a duplicate of issue 28518.

----------
components: +Library (Lib)
nosy: +berker.peksag
resolution:  -> duplicate
stage:  -> resolved
status: open -> closed
superseder:  -> execute("begin immediate") throwing OperationalError
type:  -> behavior

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29228>
_______________________________________

From report at bugs.python.org  Tue Jan 10 13:47:42 2017
From: report at bugs.python.org (Brett Cannon)
Date: Tue, 10 Jan 2017 18:47:42 +0000
Subject: [issue28885] Python 3.5.2 strange-behavior issues (from PyPy)
In-Reply-To: <1481025768.53.0.205646799997.issue28885@psf.upfronthosting.co.za>
Message-ID: <1484074062.46.0.614532979567.issue28885@psf.upfronthosting.co.za>


Brett Cannon added the comment:

I think if you're up for doing individual issues, Armin, that's preferred. But if it's too much work we will take it all in this single issue instead of risking the loss of the information. And if you want to use this issue as a meta one to track everything you report that's obviously fine.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28885>
_______________________________________

From report at bugs.python.org  Tue Jan 10 14:01:12 2017
From: report at bugs.python.org (Brett Cannon)
Date: Tue, 10 Jan 2017 19:01:12 +0000
Subject: [issue29214] Standard open() does not allow to specify file
 permissions.
In-Reply-To: <1483971421.1.0.906896747438.issue29214@psf.upfronthosting.co.za>
Message-ID: <1484074872.13.0.30275711389.issue29214@psf.upfronthosting.co.za>


Brett Cannon added the comment:

The point Serhiy is trying to make is that not everyone needs or cares about setting specific file permissions. Python's built-in open() function has not supported this for 26 years and so there's obviously not that much of a need if this has not consistently come up as a shortcoming.

Two, you say to "just pass" something like "0o644" as the mode. OK, but what does that even mean? As a Linux user you may know thanks to chmod usage, but a Windows user or someone who doesn't use a Linux shell won't have any idea what that octal constant represents. So how do you propose to help people set the proper mode? If you say "use os.O_CREAT" then you just made open() have functionality depend on the os module which no other built-in directly does for their API.

In other words you might call this simple, but I call it an advanced feature that's supported by os.fdopen(os.open()) already. So when I say a "clear design" I mean addressing the fact that it's non-obvious for beginners on how to use and something not everyone needs.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29214>
_______________________________________

From report at bugs.python.org  Tue Jan 10 14:19:23 2017
From: report at bugs.python.org (Brett Cannon)
Date: Tue, 10 Jan 2017 19:19:23 +0000
Subject: [issue29113] modulefinder no longer finds all required modules for
 Python itself, due to use of __import__ in sysconfig
In-Reply-To: <1483064165.82.0.932079190988.issue29113@psf.upfronthosting.co.za>
Message-ID: <1484075963.74.0.233619298922.issue29113@psf.upfronthosting.co.za>


Brett Cannon added the comment:

The limitation is unavoidable as modulefinder inspects bytecode for its inferencing, so any code that calls __import__() or importlib.import_module() will simply not work. So unless sysconfig can be updated reasonably back to a statically defined import this is just how it will be (I'll let doko comment on whether updating is possible and thus close this issue).

----------
nosy: +brett.cannon

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29113>
_______________________________________

From report at bugs.python.org  Tue Jan 10 14:22:09 2017
From: report at bugs.python.org (Jason Curtis)
Date: Tue, 10 Jan 2017 19:22:09 +0000
Subject: [issue29229] incompatible: unittest.mock.sentinel and
 multiprocessing.Pool.map()
Message-ID: <1484076129.07.0.522749594718.issue29229@psf.upfronthosting.co.za>


New submission from Jason Curtis:

When a sentinel object from unittest.mock.sentinel is passed through a multiprocessing.Pool.map, I expect it to still be comparable. 

As a user, I'd like to be able to write a unit test using sentinels on my unparallelized code, and then see that the test still passes after I parallelize the code using multiprocessing, so that I can make use of sentinels in regression testing.


Example:
```
from unittest import mock
import multiprocessing


def identity(x):
    return x

with multiprocessing.Pool() as pool:
    multiprocessed = list(pool.map(identity, [mock.sentinel.foo]))

assert identity(mock.sentinel.foo) == mock.sentinel.foo  # Passes
assert multiprocessed[0] == mock.sentinel.foo  # Fails
```

----------
components: Library (Lib)
messages: 285146
nosy: Jason Curtis
priority: normal
severity: normal
status: open
title: incompatible: unittest.mock.sentinel and multiprocessing.Pool.map()
type: behavior
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29229>
_______________________________________

From report at bugs.python.org  Tue Jan 10 14:25:41 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Tue, 10 Jan 2017 19:25:41 +0000
Subject: [issue29227] Reduce C stack consumption in function calls
In-Reply-To: <1484070646.73.0.530578838486.issue29227@psf.upfronthosting.co.za>
Message-ID: <1484076341.82.0.829679269954.issue29227@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

$ ./python -m perf timeit -s "from bench_recursion import test_python_call as test" -- "test(1000)"
Python 2.7:  5.10 ms +- 0.37 ms
Python 3.4:  4.38 ms +- 0.28 ms
Python 3.5:  4.19 ms +- 0.26 ms
Python 3.6:  3.93 ms +- 0.32 ms
Python 3.7:  3.26 ms +- 0.27 ms

$ ./python -m perf timeit -s "from bench_recursion import test_python_getitem as test" -- "test(1000)"
Python 2.7:  4.09 ms +- 0.26 ms
Python 3.4:  4.60 ms +- 0.23 ms
Python 3.5:  4.35 ms +- 0.28 ms
Python 3.6:  4.05 ms +- 0.34 ms
Python 3.7:  3.23 ms +- 0.23 ms

$ ./python -m perf timeit -s "from bench_recursion import test_python_iterator as test" -- "test(1000)"
Python 2.7:  7.85 ms +- 0.66 ms
Python 3.4:  9.31 ms +- 0.55 ms
Python 3.5:  9.83 ms +- 0.71 ms
Python 3.6:  8.99 ms +- 0.66 ms
Python 3.7:  8.58 ms +- 0.73 ms

----------
Added file: http://bugs.python.org/file46243/bench_recursion.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29227>
_______________________________________

From report at bugs.python.org  Tue Jan 10 14:38:20 2017
From: report at bugs.python.org (Ilya Kulakov)
Date: Tue, 10 Jan 2017 19:38:20 +0000
Subject: [issue29219] TracebackException(capture_locals=True) may fail with
 RecursionError
In-Reply-To: <1484012610.3.0.9642464783.issue29219@psf.upfronthosting.co.za>
Message-ID: <1484077100.21.0.223510162787.issue29219@psf.upfronthosting.co.za>


Ilya Kulakov added the comment:

I was not able to reproduce it.

The origin "unhandeled" exception happens after ctypes.cdll.LoadLibrary fails to load a library:

Traceback (most recent call last):
  File "...", line 852, in ...
  File ":/ctypes/__init__.py", line 425, in LoadLibrary
  File ":/ctypes/__init__.py", line 347, in __init__
OSError: [WinError 126] ????????

Then we raise our own exception.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29219>
_______________________________________

From report at bugs.python.org  Tue Jan 10 14:41:17 2017
From: report at bugs.python.org (Davin Potts)
Date: Tue, 10 Jan 2017 19:41:17 +0000
Subject: [issue29229] incompatible: unittest.mock.sentinel and
 multiprocessing.Pool.map()
In-Reply-To: <1484076129.07.0.522749594718.issue29229@psf.upfronthosting.co.za>
Message-ID: <1484077277.87.0.457955002018.issue29229@psf.upfronthosting.co.za>


Davin Potts added the comment:

This arises from the behavior of pickle (which is used by default in multiprocessing to serialize objects sent to / received from other processes in data exchanges), as seen with Python 3.6:

>>> import pickle
>>> x = pickle.dumps(mock.sentinel.foo)
>>> x
b'\x80\x03cunittest.mock\n_SentinelObject\nq\x00)\x81q\x01}q\x02X\x04\x00\x00\x00nameq\x03X\x03\x00\x00\x00fooq\x04sb.'
>>> pickle.loads(x)
sentinel.foo
>>> pickle.loads(x) == mock.sentinel.foo
False

----------
nosy: +davin

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29229>
_______________________________________

From report at bugs.python.org  Tue Jan 10 14:45:12 2017
From: report at bugs.python.org (Daniel Fackrell)
Date: Tue, 10 Jan 2017 19:45:12 +0000
Subject: [issue29230] Segfault in sqlite3
Message-ID: <1484077511.04.0.514172163049.issue29230@psf.upfronthosting.co.za>


New submission from Daniel Fackrell:

This crash occurs while running some unittests in one of our Django apps.

It duplicates on all three versions I've tried (2.7.10, 2.7.12, and 2.7.13). I don't have a Python3-capable code to test against at present.

It fails on my Mac OS system, and we can't duplicate the issue on an Ubuntu system. There's one particular test that consistently triggers it, but only if most of the other tests for the same file run first.

I've attached the .crash file (with Python 2.7.13), and included pip freeze and uname -a output below. Please let me know if I can provide more info to help with debugging this.

$ uname -a
Darwin c02pwm27g8wn.corp.verisys.com 15.6.0 Darwin Kernel Version 15.6.0: Wed Nov  2 20:30:56 PDT 2016; root:xnu-3248.60.11.1.2~2/RELEASE_X86_64 x86_64

$ pip freeze
coverage==4.3.1
Django==1.8.17
django-nose==1.4.4
funcsigs==1.0.2
lxml==3.4.4
mock==2.0.0
model-mommy==1.3.1
MySQL-python==1.2.5
nose==1.3.7
pbr==1.10.0
pycrypto==2.6.1
pytz==2016.10
requests==2.12.4
six==1.10.0
suds==0.4

----------
files: python_2017-01-09-152821_C02PWM27G8WN.crash
messages: 285150
nosy: intchanter
priority: normal
severity: normal
status: open
title: Segfault in sqlite3
type: crash
versions: Python 2.7
Added file: http://bugs.python.org/file46244/python_2017-01-09-152821_C02PWM27G8WN.crash

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29230>
_______________________________________

From report at bugs.python.org  Tue Jan 10 14:49:59 2017
From: report at bugs.python.org (Davin Potts)
Date: Tue, 10 Jan 2017 19:49:59 +0000
Subject: [issue29229] incompatible: unittest.mock.sentinel and
 multiprocessing.Pool.map()
In-Reply-To: <1484076129.07.0.522749594718.issue29229@psf.upfronthosting.co.za>
Message-ID: <1484077799.93.0.860905332903.issue29229@psf.upfronthosting.co.za>


Davin Potts added the comment:

I think this should be regarded as a duplicate of issue20804 though discussion in issue14577 is also related/relevant.

----------
superseder:  -> Sentinels identity lost when pickled (unittest.mock)

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29229>
_______________________________________

From report at bugs.python.org  Tue Jan 10 15:20:06 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Tue, 10 Jan 2017 20:20:06 +0000
Subject: [issue20804] Sentinels identity lost when pickled (unittest.mock)
In-Reply-To: <1393579357.22.0.503856996983.issue20804@psf.upfronthosting.co.za>
Message-ID: <1484079606.48.0.341480901519.issue20804@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

Proposed patch makes mock sentinels supporting pickling and copying. This allows passing them in multiprocessing.

I'm not sure whether this is a bug fix or a new feature.

The patch works only with 3.5+. Older versions need more complex solution.

----------
keywords: +patch
nosy: +serhiy.storchaka
stage:  -> patch review
versions: +Python 3.5, Python 3.6, Python 3.7 -Python 3.3, Python 3.4
Added file: http://bugs.python.org/file46245/mock_sentinel_pickle.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue20804>
_______________________________________

From report at bugs.python.org  Tue Jan 10 15:25:59 2017
From: report at bugs.python.org (Ned Deily)
Date: Tue, 10 Jan 2017 20:25:59 +0000
Subject: [issue29230] Segfault in sqlite3
In-Reply-To: <1484077511.04.0.514172163049.issue29230@psf.upfronthosting.co.za>
Message-ID: <1484079959.41.0.402270066151.issue29230@psf.upfronthosting.co.za>


Ned Deily added the comment:

Thanks for the report.  From a quick look at the macOS crash report, I'm guessing you are running with the Apple-supplied system Python 2.7 but with a virtualenv.  In any case, the Apple-supplied libsqlite3 is in use and there are previously reported problems using it with multiple threads, possibly multiple processes, accessing the same database file; see for example Issue27126.  Suggest you try using a Python instance that is linked with its own version of a current libsqlite3, rather than with the system one: Pythons installed from current python.org macOS installers, from Macports, or (probably) Homebrew should all fit the bill and see if the problems go away.

----------
nosy: +ned.deily
resolution:  -> third party
status: open -> pending

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29230>
_______________________________________

From report at bugs.python.org  Tue Jan 10 15:44:16 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Tue, 10 Jan 2017 20:44:16 +0000
Subject: [issue29219] TracebackException(capture_locals=True) may fail with
 RecursionError
In-Reply-To: <1484012610.3.0.9642464783.issue29219@psf.upfronthosting.co.za>
Message-ID: <1484081056.9.0.396722669158.issue29219@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

Proposed patch fixes an infinite recursion in a repr of uninitialized CDLL instance.

----------
keywords: +patch
stage:  -> patch review
Added file: http://bugs.python.org/file46246/ctypes_cdll_uninitialized_repr.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29219>
_______________________________________

From report at bugs.python.org  Tue Jan 10 16:11:59 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 10 Jan 2017 21:11:59 +0000
Subject: [issue29214] Standard open() does not allow to specify file
 permissions.
In-Reply-To: <1484074872.13.0.30275711389.issue29214@psf.upfronthosting.co.za>
Message-ID: <CAMpsgwb7z0MQTPtVs=FbD3rHHqh5UYBj+jh95u=FPuQBous2xQ@mail.gmail.com>


STINNER Victor added the comment:

I don't understand the problem with umask(). It's standard and affect all
code even C extension calling directly or indirectly open(). It is more
secure to use umask() than setting mode on a few Python open() calls no?

For example, Python creates .pyc files. You cannot (easily) modify the
open() call to set mode, whereas it is affected by umask().

If you call umask() at startup, it affects all threads, there is no race
condition like open()+fchmod().

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29214>
_______________________________________

From report at bugs.python.org  Tue Jan 10 16:12:53 2017
From: report at bugs.python.org (Davin Potts)
Date: Tue, 10 Jan 2017 21:12:53 +0000
Subject: [issue20804] Sentinels identity lost when pickled (unittest.mock)
In-Reply-To: <1393579357.22.0.503856996983.issue20804@psf.upfronthosting.co.za>
Message-ID: <1484082773.42.0.625041123437.issue20804@psf.upfronthosting.co.za>


Davin Potts added the comment:

Serhiy: The above discussion seemed to converge on the perspective that object identity should not survive pickling and that the point of a sentinel is object identity.  While your proposed patch may mechanically work, I believe it is in conflict with the outcome of the thoughtful discussion above.

----------
nosy: +davin

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue20804>
_______________________________________

From report at bugs.python.org  Tue Jan 10 16:14:07 2017
From: report at bugs.python.org (KeyWeeUsr)
Date: Tue, 10 Jan 2017 21:14:07 +0000
Subject: [issue29231] Broken MSIs in Python 3.5+
Message-ID: <1484082847.93.0.115252889304.issue29231@psf.upfronthosting.co.za>


New submission from KeyWeeUsr:

In versions _lower_ than 3.5.0 there was a `.MSI` installer for Windows, which had a really nice hidden option. That option looked like this:

    msiexec.exe /a "file.msi" /qb /L*V "file.log" ALLUSERS=0 TARGETDIR="target" CURRENTDIRECTORY="%cd%" <additional options>

which basically allows me to ignore admin rights, because the MSI turns to some kind of installer for whole network, thus the permission workaround for older Python versions.

With Python 3.5.0 was introduced a new `.EXE` installer, which has `.MSI` files packed in itself, you can get them out with:

    python-3.5.0.exe /layout [optional target directory]

yet there's a really annoying thing going around with this solution. When I do this, the `.MSI` files have a `-d.msi` suffix and when I manually unpack it with the `msiexec` command above, **every file** has that suffix too, which makes it a _completely damaged_ installation. Renaming the files isn't really an option as each file has `-d.<file ext>`, not `-d.msi.<file ext>` suffix, so it's too hard to rename it in a simple way with tools such as Batch, unless I want to check for multiple cases (e.g. folders).

Is there any way how to extract the content of the installer to a separate folder, like it was possible before **without** any additional stuff put into `Programs and features`, such as `Python 3.5.0 (64bit)` or similar?

Or other question - is there any way how to forbid the installer to access `Programs and features`, shut it from asking admin privileges and registry at all?

It's quite useful if I want to have multiple python fresh installations not bound to anything at all with testing as its main purpose. Note, that I have no intention to use python launcher (that `py.exe` thing), virtualenv or any other alternative "solution" as every of them allows me to install only a one **global** Python installation of the same version and/or is too big for quick usage and at the same time I need to have an option for testing on a fresh interpreter not modified by anything and completely cut off from the OS stuff.

Any ideas how can I achieve the same behavior even on newer Python versions with the EXE installer?

----------
components: Installation, Windows
messages: 285157
nosy: KeyWeeUsr, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Broken MSIs in Python 3.5+
type: behavior
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29231>
_______________________________________

From report at bugs.python.org  Tue Jan 10 16:22:29 2017
From: report at bugs.python.org (Christian Heimes)
Date: Tue, 10 Jan 2017 21:22:29 +0000
Subject: [issue29214] Standard open() does not allow to specify file
 permissions.
In-Reply-To: <1483971421.1.0.906896747438.issue29214@psf.upfronthosting.co.za>
Message-ID: <1484083349.75.0.164130405752.issue29214@psf.upfronthosting.co.za>


Christian Heimes added the comment:

Victor, you are correct. That was exactly my point.

The most secure way is to tighten security and set umask to 0o077. It's basically a white list or locked down approach. With umask 0o077 all subsequent files, directories and other resources will be created without any permission for group and others. This is even true for operations that create a Unix socket.

You have to change the permission of files to a more permissive mode explicitly. Any mistake is easy to spot (access denied) and not catastrophic.

By the way fchmod() isn't necessarily the optimal way to change permission by file descriptor. The behavior of fchmod() isn't well defined for socket files. On Linux fchmod() of a Unix socket file does not alter the permission bits of the socket device file.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29214>
_______________________________________

From report at bugs.python.org  Tue Jan 10 16:29:12 2017
From: report at bugs.python.org (Earl Maier)
Date: Tue, 10 Jan 2017 21:29:12 +0000
Subject: [issue29232] Quiet Install
Message-ID: <1484083752.73.0.652797539276.issue29232@psf.upfronthosting.co.za>


New submission from Earl Maier:

On quiet and passive Installs of Python 3.6 and 3.5 in windows 7 and 10 (x64), Python successfully installs but does not appear in programs and features. It would also appear that the append path and python launcher switches do not work for quiet install.  If installed manual everything functions as it should.

Quiet installs work just fine with version 2.7 (msi vs exe).

The installers used are the Windows x86-64 executable installer for both python 3.6.0 and 3.5.2.

----------
components: Windows
files: Capture.PNG
messages: 285159
nosy: earl.maier, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Quiet Install
type: behavior
versions: Python 3.5, Python 3.6
Added file: http://bugs.python.org/file46247/Capture.PNG

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29232>
_______________________________________

From report at bugs.python.org  Tue Jan 10 16:31:18 2017
From: report at bugs.python.org (KeyWeeUsr)
Date: Tue, 10 Jan 2017 21:31:18 +0000
Subject: [issue29231] Broken MSIs in Python 3.5+
In-Reply-To: <1484082847.93.0.115252889304.issue29231@psf.upfronthosting.co.za>
Message-ID: <1484083878.69.0.501868330191.issue29231@psf.upfronthosting.co.za>


Changes by KeyWeeUsr <keyweeusr at gmail.com>:


----------
versions: +Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29231>
_______________________________________

From report at bugs.python.org  Tue Jan 10 16:34:55 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 10 Jan 2017 21:34:55 +0000
Subject: [issue29227] Reduce C stack consumption in function calls
In-Reply-To: <1484070646.73.0.530578838486.issue29227@psf.upfronthosting.co.za>
Message-ID: <1484084095.89.0.815485258193.issue29227@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Oh wow! I'm impressed that Python 3 is better at each release! On 2 tests, Python 3.7 is faster than Python 2.7, but on test_python_iterator Python 3.7 is still slower. It seems like this specific test became much slower (+19%) on Python 3.4 compared to 2.7.

I guess that your benchmark is on unpatched Python.

I don't think that less_stack.patch has an impact on performances, but I guess because I'm curisous. It seems like it's a little bit faster. At least, it's not slower ;-)

test_python_call: Median +- std dev: [ref] 509 us +- 11 us -> [patch] 453 us +- 49 us: 1.12x faster (-11%)
test_python_getitem: Median +- std dev: [ref] 485 us +- 13 us -> [patch] 470 us +- 23 us: 1.03x faster (-3%)
test_python_iterator: Median +- std dev: [ref] 1.15 ms +- 0.05 ms -> [patch] 1.12 ms +- 0.07 ms: 1.03x faster (-3%)

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29227>
_______________________________________

From report at bugs.python.org  Tue Jan 10 16:38:04 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Tue, 10 Jan 2017 21:38:04 +0000
Subject: [issue20804] Sentinels identity lost when pickled (unittest.mock)
In-Reply-To: <1393579357.22.0.503856996983.issue20804@psf.upfronthosting.co.za>
Message-ID: <1484084284.8.0.996071486739.issue20804@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

Sorry, I had wrote this patch for issue29229 and didn't read the above discussion.

Copying and pickling preserve identity of some objects: functions, classes, enums, builtin singletons (None, False, True, ...). It was not obvious to me that this shouldn't work with mock sentinels.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue20804>
_______________________________________

From report at bugs.python.org  Tue Jan 10 17:52:30 2017
From: report at bugs.python.org (Matthias Klose)
Date: Tue, 10 Jan 2017 22:52:30 +0000
Subject: [issue29113] modulefinder no longer finds all required modules for
 Python itself, due to use of __import__ in sysconfig
In-Reply-To: <1483064165.82.0.932079190988.issue29113@psf.upfronthosting.co.za>
Message-ID: <1484088750.01.0.309924201664.issue29113@psf.upfronthosting.co.za>


Matthias Klose added the comment:

the idea is that we load a different _sysconfigdata module when we are cross building packages.  So we don't know the name in advance.  An ugly alternative would be a big if statement with conditional imports for all known cross build targets. Not sure if this is the better solution.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29113>
_______________________________________

From report at bugs.python.org  Tue Jan 10 18:00:51 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Tue, 10 Jan 2017 23:00:51 +0000
Subject: [issue29227] Reduce C stack consumption in function calls
In-Reply-To: <1484070646.73.0.530578838486.issue29227@psf.upfronthosting.co.za>
Message-ID: <1484089251.08.0.26668900197.issue29227@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

I didn't provide results with less_stack.patch because they were almost the same, just 1-3% faster. That might be just a random noise or compiler artifact. But may be an effect of inlining call_function().

Could you run full Python benchmarks? Decreasing the size of small stack doesn't impact a performance in these cases, but may impact a performance of calls with larger number of arguments. AFAIK the size of some small stacks already was decreased from 8 to 5.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29227>
_______________________________________

From report at bugs.python.org  Tue Jan 10 18:03:47 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 10 Jan 2017 23:03:47 +0000
Subject: [issue29227] Reduce C stack consumption in function calls
In-Reply-To: <1484070646.73.0.530578838486.issue29227@psf.upfronthosting.co.za>
Message-ID: <1484089427.71.0.399896816343.issue29227@psf.upfronthosting.co.za>


STINNER Victor added the comment:

I plan to run a benchmark when all my patches to reduce the stack consumption will be ready. I'm still trying all the various options to reduce the stack consumption. I'm trying to avoid hacks and reduce the number of changes. I'm already better than Python 2.7 and 3.5 on my local branch.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29227>
_______________________________________

From report at bugs.python.org  Tue Jan 10 18:05:00 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 10 Jan 2017 23:05:00 +0000
Subject: [issue29233] call_method(): call _PyObject_FastCall() rather than
 _PyObject_VaCallFunctionObjArgs()
Message-ID: <1484089500.65.0.838677833824.issue29233@psf.upfronthosting.co.za>


New submission from STINNER Victor:

Recently, I modified call_method() and call_maybe() of Objects/typeobject.c to avoid the creation of a temporary tuple: replace Py_VaBuildValue()+PyObject_Call() with the new _PyObject_VaCallFunctionObjArgs().

But while working on the issue #28870, I noticed that _PyObject_VaCallFunctionObjArgs() is not efficient and can be avoided. In typeobject.c, the number of arguments is hardcoded, so we don't need complex functions to compute the number of arguments and decide to allocate an array on the stack or on the heap.

----------
messages: 285165
nosy: haypo
priority: normal
severity: normal
status: open
title: call_method(): call _PyObject_FastCall() rather than _PyObject_VaCallFunctionObjArgs()

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29233>
_______________________________________

From report at bugs.python.org  Tue Jan 10 18:14:31 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 10 Jan 2017 23:14:31 +0000
Subject: [issue29233] call_method(): call _PyObject_FastCall() rather than
 _PyObject_VaCallFunctionObjArgs()
In-Reply-To: <1484089500.65.0.838677833824.issue29233@psf.upfronthosting.co.za>
Message-ID: <1484090071.61.0.786320439483.issue29233@psf.upfronthosting.co.za>


Changes by STINNER Victor <victor.stinner at gmail.com>:


----------
nosy: +serhiy.storchaka

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29233>
_______________________________________

From report at bugs.python.org  Tue Jan 10 18:14:42 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 10 Jan 2017 23:14:42 +0000
Subject: [issue29233] call_method(): call _PyObject_FastCall() rather than
 _PyObject_VaCallFunctionObjArgs()
In-Reply-To: <1484089500.65.0.838677833824.issue29233@psf.upfronthosting.co.za>
Message-ID: <1484090082.04.0.172648736271.issue29233@psf.upfronthosting.co.za>


Changes by STINNER Victor <victor.stinner at gmail.com>:


----------
components: +Interpreter Core
type:  -> performance
versions: +Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29233>
_______________________________________

From report at bugs.python.org  Tue Jan 10 18:22:33 2017
From: report at bugs.python.org (Roundup Robot)
Date: Tue, 10 Jan 2017 23:22:33 +0000
Subject: [issue29233] call_method(): call _PyObject_FastCall() rather than
 _PyObject_VaCallFunctionObjArgs()
In-Reply-To: <1484089500.65.0.838677833824.issue29233@psf.upfronthosting.co.za>
Message-ID: <20170110232223.104305.74096.99031C80@psf.io>


Roundup Robot added the comment:

New changeset b9404639a18c by Victor Stinner in branch 'default':
call_method() now uses _PyObject_FastCall()
https://hg.python.org/cpython/rev/b9404639a18c

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29233>
_______________________________________

From report at bugs.python.org  Tue Jan 10 18:22:57 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 10 Jan 2017 23:22:57 +0000
Subject: [issue29233] call_method(): call _PyObject_FastCall() rather than
 _PyObject_VaCallFunctionObjArgs()
In-Reply-To: <1484089500.65.0.838677833824.issue29233@psf.upfronthosting.co.za>
Message-ID: <1484090577.91.0.180682367255.issue29233@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Full commit message:

changeset:   106081:b9404639a18c
tag:         tip
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Wed Jan 11 00:07:40 2017 +0100
files:       Include/abstract.h Objects/abstract.c Objects/typeobject.c
description:
call_method() now uses _PyObject_FastCall()

Issue #29233: Replace the inefficient _PyObject_VaCallFunctionObjArgs() with
_PyObject_FastCall() in call_method() and call_maybe().

Only a few functions call call_method() and call it with a fixed number of
arguments. Avoid the complex and expensive _PyObject_VaCallFunctionObjArgs()
function, replace it with an array allocated on the stack with the exact number
of argumlents.

It reduces the stack consumption, bytes per call, before => after:

test_python_call: 1168 => 1152 (-16 B)
test_python_getitem: 1344 => 1008 (-336 B)
test_python_iterator: 1568 => 1232 (-336 B)

Remove the _PyObject_VaCallFunctionObjArgs() function which became useless.
Rename it to object_vacall() and make it private.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29233>
_______________________________________

From report at bugs.python.org  Tue Jan 10 19:05:11 2017
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 11 Jan 2017 00:05:11 +0000
Subject: [issue29234] Disable inlining of _PyStack_AsTuple() to reduce the
 stack consumption
Message-ID: <1484093111.93.0.908854837348.issue29234@psf.upfronthosting.co.za>


New submission from STINNER Victor:

While working on the issue #28870, I noticed that _PyStack_AsTuple() is inlined. Compared to Python 3.5 which didn't have this function, it seems like _PyStack_AsTuple() is responsible for an increase of the stack consumption.

----------
components: Interpreter Core
messages: 285168
nosy: haypo, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Disable inlining of _PyStack_AsTuple() to reduce the stack consumption
type: enhancement
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29234>
_______________________________________

From report at bugs.python.org  Tue Jan 10 19:20:39 2017
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 11 Jan 2017 00:20:39 +0000
Subject: [issue28870] Reduce stack consumption of
 PyObject_CallFunctionObjArgs() and like
In-Reply-To: <1480891813.94.0.54795440575.issue28870@psf.upfronthosting.co.za>
Message-ID: <1484094039.97.0.430484628044.issue28870@psf.upfronthosting.co.za>


STINNER Victor added the comment:

I pushed 3 changes:

* rev b9404639a18c: Issue #29233: call_method() now uses _PyObject_FastCall()
* rev 8481c379e2da: Issue #29227: inline call_function()
* rev 6478e6d0476f: Issue #29234: disable _PyStack_AsTuple() inlining


Before (rev a30cdf366c02):

test_python_call: 7175 calls before crash, stack: 1168 bytes/call
test_python_getitem: 6235 calls before crash, stack: 1344 bytes/call
test_python_iterator: 5344 calls before crash, stack: 1568 bytes/call

=> total: 18754 calls, 4080 bytes


With these 3 changes (rev 6478e6d0476f):

test_python_call: 8587 calls before crash, stack: 976 bytes/call
test_python_getitem: 9189 calls before crash, stack: 912 bytes/call
test_python_iterator: 7936 calls before crash, stack: 1056 bytes/call

=> total: 25712 calls, 2944 bytes


The default branch is now as good as Python 3.4, in term of stack consumption, and Python 3.4 was the Python version which used the least stack memory according to my tests.

I didn't touch _PY_FASTCALL_SMALL_STACK value, it's still 5 arguments (40 bytes). So my changes should not impact performances.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28870>
_______________________________________

From report at bugs.python.org  Tue Jan 10 19:22:40 2017
From: report at bugs.python.org (Brett Cannon)
Date: Wed, 11 Jan 2017 00:22:40 +0000
Subject: [issue2771] Test issue
In-Reply-To: <1210005645.74.0.283923986194.issue2771@psf.upfronthosting.co.za>
Message-ID: <1484094160.67.0.289801815848.issue2771@psf.upfronthosting.co.za>


Changes by Brett Cannon <brett at python.org>:


----------
pull_requests: +11

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue2771>
_______________________________________

From report at bugs.python.org  Tue Jan 10 19:26:08 2017
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 11 Jan 2017 00:26:08 +0000
Subject: [issue29234] Disable inlining of _PyStack_AsTuple() to reduce the
 stack consumption
In-Reply-To: <1484093111.93.0.908854837348.issue29234@psf.upfronthosting.co.za>
Message-ID: <1484094368.77.0.335618759413.issue29234@psf.upfronthosting.co.za>


STINNER Victor added the comment:

noinline_msvs.patch: implement _Py_NO_INLINE for Microsoft Visual Studio

#elif defined(_MSC_VER)
#  define _Py_NO_INLINE __declspec(noinline)

I didn't push this change because I don't have access to a Windows VM yet.

@Steve: Does noinline_msvs.patch look good to you?

I decided to make the macro private because I don't know if _Py_NO_INLINE can be combined with PyAPI_FUNC() which also uses __declspec(): __declspec(dllexport).

Is it possible to use __declspec() multiple times per function declaration?

----------
keywords: +patch
nosy: +steve.dower
Added file: http://bugs.python.org/file46248/noinline_msvs.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29234>
_______________________________________

From report at bugs.python.org  Tue Jan 10 19:26:48 2017
From: report at bugs.python.org (Brett Cannon)
Date: Wed, 11 Jan 2017 00:26:48 +0000
Subject: [issue2771] Test issue
In-Reply-To: <1210005645.74.0.283923986194.issue2771@psf.upfronthosting.co.za>
Message-ID: <1484094408.96.0.64258380844.issue2771@psf.upfronthosting.co.za>


Changes by Brett Cannon <brett at python.org>:


----------
pull_requests: +12

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue2771>
_______________________________________

From report at bugs.python.org  Tue Jan 10 19:26:53 2017
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 11 Jan 2017 00:26:53 +0000
Subject: [issue29233] call_method(): call _PyObject_FastCall() rather than
 _PyObject_VaCallFunctionObjArgs()
In-Reply-To: <1484089500.65.0.838677833824.issue29233@psf.upfronthosting.co.za>
Message-ID: <1484094413.18.0.658430368521.issue29233@psf.upfronthosting.co.za>


Changes by STINNER Victor <victor.stinner at gmail.com>:


----------
resolution:  -> fixed
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29233>
_______________________________________

From report at bugs.python.org  Tue Jan 10 19:28:06 2017
From: report at bugs.python.org (Roundup Robot)
Date: Wed, 11 Jan 2017 00:28:06 +0000
Subject: [issue29234] Disable inlining of _PyStack_AsTuple() to reduce the
 stack consumption
In-Reply-To: <1484093111.93.0.908854837348.issue29234@psf.upfronthosting.co.za>
Message-ID: <20170111002803.24547.87130.520863CC@psf.io>


Roundup Robot added the comment:

New changeset 6478e6d0476f by Victor Stinner in branch 'default':
Disable _PyStack_AsTuple() inlining
https://hg.python.org/cpython/rev/6478e6d0476f

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29234>
_______________________________________

From report at bugs.python.org  Tue Jan 10 19:28:06 2017
From: report at bugs.python.org (Roundup Robot)
Date: Wed, 11 Jan 2017 00:28:06 +0000
Subject: [issue29227] Reduce C stack consumption in function calls
In-Reply-To: <1484070646.73.0.530578838486.issue29227@psf.upfronthosting.co.za>
Message-ID: <20170111002803.24547.70350.9F55C024@psf.io>


Roundup Robot added the comment:

New changeset 8481c379e2da by Victor Stinner in branch 'default':
Inline call_function()
https://hg.python.org/cpython/rev/8481c379e2da

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29227>
_______________________________________

From report at bugs.python.org  Tue Jan 10 19:30:02 2017
From: report at bugs.python.org (Brett Cannon)
Date: Wed, 11 Jan 2017 00:30:02 +0000
Subject: [issue2771] Test issue
In-Reply-To: <1210005645.74.0.283923986194.issue2771@psf.upfronthosting.co.za>
Message-ID: <1484094602.5.0.921381373512.issue2771@psf.upfronthosting.co.za>


Changes by Brett Cannon <brett at python.org>:


----------
pull_requests: +13

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue2771>
_______________________________________

From report at bugs.python.org  Tue Jan 10 19:33:39 2017
From: report at bugs.python.org (Brett Cannon)
Date: Wed, 11 Jan 2017 00:33:39 +0000
Subject: [issue2771] Test issue
In-Reply-To: <1210005645.74.0.283923986194.issue2771@psf.upfronthosting.co.za>
Message-ID: <1484094819.8.0.620532057493.issue2771@psf.upfronthosting.co.za>


Changes by Brett Cannon <brett at python.org>:


----------
pull_requests: +14

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue2771>
_______________________________________

From report at bugs.python.org  Tue Jan 10 19:50:29 2017
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 11 Jan 2017 00:50:29 +0000
Subject: [issue28870] Reduce stack consumption of
 PyObject_CallFunctionObjArgs() and like
In-Reply-To: <1480891813.94.0.54795440575.issue28870@psf.upfronthosting.co.za>
Message-ID: <1484095829.09.0.357406575617.issue28870@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Result of attached bench_recursion-2.py comparing before/after the 3 changes reducing the stack consumption:

test_python_call: Median +- std dev: [a30cdf366c02] 512 us +- 12 us -> [6478e6d0476f] 467 us +- 21 us: 1.10x faster (-9%)
test_python_getitem: Median +- std dev: [a30cdf366c02] 485 us +- 26 us -> [6478e6d0476f] 437 us +- 18 us: 1.11x faster (-10%)
test_python_iterator: Median +- std dev: [a30cdf366c02] 1.15 ms +- 0.04 ms -> [6478e6d0476f] 1.03 ms +- 0.06 ms: 1.12x faster (-10%)

At least, it doesn't seem to be slower. Maybe the speedup comes from call_function() inlining. This function was probably already inlined when using PGO build.

The script was written by Serhiy in the issue #29227, I modified it to use the Runner.timeit() API for convenience.

----------
Added file: http://bugs.python.org/file46249/bench_recursion-2.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28870>
_______________________________________

From report at bugs.python.org  Tue Jan 10 19:56:45 2017
From: report at bugs.python.org (Jason Curtis)
Date: Wed, 11 Jan 2017 00:56:45 +0000
Subject: [issue29229] incompatible: unittest.mock.sentinel and
 multiprocessing.Pool.map()
In-Reply-To: <1484076129.07.0.522749594718.issue29229@psf.upfronthosting.co.za>
Message-ID: <1484096205.91.0.579876094763.issue29229@psf.upfronthosting.co.za>


Jason Curtis added the comment:

sounds right; closing as a duplicate of issue20804

----------
resolution:  -> duplicate
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29229>
_______________________________________

From report at bugs.python.org  Tue Jan 10 20:13:40 2017
From: report at bugs.python.org (Thane Brimhall)
Date: Wed, 11 Jan 2017 01:13:40 +0000
Subject: [issue29235] Allow profile/cProfile to be used as context managers
Message-ID: <1484097220.43.0.363931456874.issue29235@psf.upfronthosting.co.za>


New submission from Thane Brimhall:

The `enable` and `disable` methods on the profilers fit the description of a context manager very well. The following code:

    pr = cProfile.Profile()
    pr.enable()
    # ... do something ...
    pr.disable()
    pr.print_stats()

Would turn into something like this:

    with cProfile.Profile() as pr:
        # ... do something ...
    pr.print_stats()

The patch for this code would be trivial and backwards-compatible: simply add something like the following lines to the _lsprof.Profiler base class:

    def __enter__(self):
        self.enable()
        return self

    def __exit__(self, exc_type, exc_val, exc_tb):
        self.disable()

----------
components: Library (Lib)
messages: 285175
nosy: Thane Brimhall
priority: normal
severity: normal
status: open
title: Allow profile/cProfile to be used as context managers
type: enhancement
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29235>
_______________________________________

From report at bugs.python.org  Tue Jan 10 20:17:40 2017
From: report at bugs.python.org (Maciej Piechorka)
Date: Wed, 11 Jan 2017 01:17:40 +0000
Subject: [issue29236] 'an ASCII string of one or more PEM-encoded
 certificates' needs to be unicode
Message-ID: <1484097460.25.0.906949156541.issue29236@psf.upfronthosting.co.za>


New submission from Maciej Piechorka:

In documentation it is specified that cadata parameter in load_verify_locations is 'an ASCII string of one or more PEM-encoded certificates'. However the code is actually determining it based on PyUnicode_Check function so the 'ASCII string' actually needs to be unicode object. In Python 3 it seems to be fixed by checking by PyObject_GetBuffer.

----------
assignee: docs at python
components: Documentation, SSL
messages: 285176
nosy: docs at python, uzytkownik
priority: normal
severity: normal
status: open
title: 'an ASCII string of one or more PEM-encoded certificates' needs to be unicode
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29236>
_______________________________________

From report at bugs.python.org  Tue Jan 10 20:22:44 2017
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 11 Jan 2017 01:22:44 +0000
Subject: [issue28839] _PyFunction_FastCallDict(): replace PyTuple_New() with
 PyMem_Malloc()
In-Reply-To: <1480505567.26.0.13388381207.issue28839@psf.upfronthosting.co.za>
Message-ID: <1484097764.55.0.322831839468.issue28839@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Ok, I give up on that one. I don't think that it's worth it.

----------
resolution:  -> rejected
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28839>
_______________________________________

From report at bugs.python.org  Tue Jan 10 20:31:34 2017
From: report at bugs.python.org (Thane Brimhall)
Date: Wed, 11 Jan 2017 01:31:34 +0000
Subject: [issue29237] Create enum for pstats sorting options
Message-ID: <1484098294.24.0.117553666266.issue29237@psf.upfronthosting.co.za>


New submission from Thane Brimhall:

When using the cProfile and pstats modules, I often forget which string keys are available for sorting. It would be nice to add an enum for these so a user could have their linter and IDE check that value pre-runtime.

By subclassing both `str` and `Enum` this proposal would be backwards-compatible with all existing code.

The patch for such a change would be trivial:


1. Add a new Sort class to the pstats module:

class Sort(str, enum.Enum):
    calls = 'calls'  # call count
    cumulative = 'cumulative'  # cumulative time
    cumtime = 'cumtime'  # cumulative time
    file = 'file'  # file name
    filename = 'filename'  # file name
    module = 'module'  # file name
    ncalls = 'ncalls'  # call count
    pcalls = 'pcalls'  # primitive call count
    line = 'line'  # line number
    name = 'name'  # function name
    nfl = 'nfl'  # name/file/line
    stdname = 'stdname'  # standard name
    time = 'time'  # internal time
    tottime = 'tottime'  # internal time


2. Change the print_stats method signature on the profiler base and subclasses to look like this:

def print_stats(self, sort: Sort=Sort.stdname):


Optionally, you could build the Sort enum like below to remove redundant options and increase explicitness:

class Sort(str, enum.Enum):
    call_count = 'calls'
    cumulative_time = 'cumulative'
    filename = 'filename'
    primitive_call_count = 'pcalls'
    line_number = 'line'
    function_name = 'name'
    name_file_line = 'nfl'
    standard_name = 'stdname'
    internal_time = 'time'

----------
components: Library (Lib)
messages: 285178
nosy: Thane Brimhall
priority: normal
severity: normal
status: open
title: Create enum for pstats sorting options
type: enhancement
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29237>
_______________________________________

From report at bugs.python.org  Tue Jan 10 20:45:23 2017
From: report at bugs.python.org (Ethan Furman)
Date: Wed, 11 Jan 2017 01:45:23 +0000
Subject: [issue29237] Create enum for pstats sorting options
In-Reply-To: <1484098294.24.0.117553666266.issue29237@psf.upfronthosting.co.za>
Message-ID: <1484099123.26.0.560396452879.issue29237@psf.upfronthosting.co.za>


Ethan Furman added the comment:

To keep backwards compatibility and lesson the burden on new code, simply make the value of the duplicate names be the same:

    cumulative = 'cumulative'
    cumtime = 'cumulative'

This way the standard name is 'cumulative' but 'cumtime' works as well.

----------
nosy: +ethan.furman
stage:  -> needs patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29237>
_______________________________________

From report at bugs.python.org  Tue Jan 10 21:06:58 2017
From: report at bugs.python.org (Thomas Nyberg)
Date: Wed, 11 Jan 2017 02:06:58 +0000
Subject: =?utf-8?q?=5Bissue24875=5D_pyvenv_doesn=C2=B4t_install_PIP_inside_a_new_v?=
 =?utf-8?q?env_with_--system-site-package?=
In-Reply-To: <1439699040.07.0.199956094974.issue24875@psf.upfronthosting.co.za>
Message-ID: <1484100418.07.0.963365970838.issue24875@psf.upfronthosting.co.za>


Thomas Nyberg added the comment:

I'm attaching a small patch (against the current hg master branch) that I believe solves the problem, however, I think it was quite hacky.

Here is a walk-through of the logic.

1) If '--system-site-packages' is requested, then it is overrided initially in the process when creating the config file. I.e. it is hard-coded to 'false' in the pyvenv.cfg file.
2) Next ensurepip is run. This is being run in the same state as it would be if no '--system-site-packages' flag were passed, so this time it installs some packages to the venv's site-packages.
3) If necessary, the config file is regenerated without over-riding the system-site-packages flag.
4) If necessary, all packages installed in site-packages are removed and a new empty site-packages folder is regenerated.

The logic here is pretty horrid, but it seems to be functioning correctly. There are a couple possible issues here:

1) I assume that on all platforms the site-packages folder will always end up empty when the no '--system-site-packages' flag is passed.
2) I also assume that on all platforms there are no differences in the venv that is produced with or without this option except for the packages installed in the system-site-packages folder.

I don't really have any way to verify either of those assumption experimentally and I'm not familiar enough with ensurepip/pip to know that way either. I've only tested this on my debian system so it could very well not be portable.

Regardless of my misgivings of the quality of this patch, I think a solution to this problem is totally essential. The '--system-site-packages' flag is a great way to let your distro's package manager install most packages you need while allowing you then to install whatever others your distro does not provide. I use this all the time to simplify/decrease maintenance complexity. Without this fix, the --system-site-packages flag is essentially equivalent to a read-only environment and in that case, there's no point in using it at all.

----------
nosy: +thomas.nyberg
versions: +Python 3.7 -Python 3.4, Python 3.5
Added file: http://bugs.python.org/file46250/venv_site_packages.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24875>
_______________________________________

From report at bugs.python.org  Tue Jan 10 21:16:40 2017
From: report at bugs.python.org (Thomas Nyberg)
Date: Wed, 11 Jan 2017 02:16:40 +0000
Subject: =?utf-8?q?=5Bissue24875=5D_pyvenv_doesn=C2=B4t_install_PIP_inside_a_new_v?=
 =?utf-8?q?env_with_--system-site-package?=
In-Reply-To: <1439699040.07.0.199956094974.issue24875@psf.upfronthosting.co.za>
Message-ID: <1484101000.17.0.351268141716.issue24875@psf.upfronthosting.co.za>


Thomas Nyberg added the comment:

Now that I look over this thread again, I realize that my patch ended up basically evolving to being extremely similar to Mark Haase's. The only thing that I don't understand though is that he says that his shebangs are somehow messed up. For me the shebangs are all fine with my patch. Possibly this is difference between his system (OSX) and mine (debian 8)? In any case, I'm afraid I may have re-engineered almost the same solution as him (and therefore that it doesn't work right), but I guess I needed to go through the process to familiarize myself with the problem.

Mark if you're still listening to this thread, maybe you could test this out and see if you're still having the same shebang problems? I can't reproduce your issue.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24875>
_______________________________________

From report at bugs.python.org  Tue Jan 10 21:27:09 2017
From: report at bugs.python.org (Steve Dower)
Date: Wed, 11 Jan 2017 02:27:09 +0000
Subject: [issue29234] Disable inlining of _PyStack_AsTuple() to reduce the
 stack consumption
In-Reply-To: <1484093111.93.0.908854837348.issue29234@psf.upfronthosting.co.za>
Message-ID: <1484101629.6.0.95971378623.issue29234@psf.upfronthosting.co.za>


Steve Dower added the comment:

I'm fairly sure dllexport implies noinline, and it certainly does for the exported function (but maybe PGO will inline it within the module? hard to know). In any case, I agree with keeping it private as it's purely about optimization.

Do you have a test that can run against the released build? That will be the easiest way to tell whether it matters at all.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29234>
_______________________________________

From report at bugs.python.org  Tue Jan 10 22:00:31 2017
From: report at bugs.python.org (Thane Brimhall)
Date: Wed, 11 Jan 2017 03:00:31 +0000
Subject: [issue29238] Add more kwargs to cProfile's print_stats
Message-ID: <1484103631.29.0.405631580429.issue29238@psf.upfronthosting.co.za>


New submission from Thane Brimhall:

Using the pstats class to print off profiler results is helpful when you want to filter, order, and limit the number of returned lines. I'd rather see the most commonly-used subset of pstats functionality included in the profiler's print_stats implementation directly.


    # The current way
    pstats.Stats(profiler).strip_dirs().sort_stats('cumulative').reverse_order().print_stats(10)

    # Proposed way
    profiler.print_stats(strip_dirs=False, sort='cumulative', reverse=True, limit=10)


Currently only the `sort` kwarg is available. To me this implies that some level of control was originally intended to be available in the print_stats method anyway. I also feel like the proposed API is more readable and explicit.

Note that for complex situations you'd still need to use the pstats class directly, eg. substituting a different stream implementation or filtering/sorting by multiple values.

This would be a backwards-compatible patch and would be implemented something like this:


    def print_stats(self, sort=-1, limit=None, strip_dirs=True, reverse=True):
        import pstats
        stats = pstats.Stats(self)
        if strip_dirs:
            stats = stats.strip_dirs()
        stats = stats.sort_stats(sort)
        if reverse:
            stats = stats.reverse_order()
        stats.print_stats(limit)

----------
components: Library (Lib)
messages: 285183
nosy: Thane Brimhall
priority: normal
severity: normal
status: open
title: Add more kwargs to cProfile's print_stats
type: enhancement
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29238>
_______________________________________

From report at bugs.python.org  Tue Jan 10 22:45:25 2017
From: report at bugs.python.org (Jeremy Freeman)
Date: Wed, 11 Jan 2017 03:45:25 +0000
Subject: [issue28231] zipfile does not support pathlib
In-Reply-To: <1474445503.11.0.219964117103.issue28231@psf.upfronthosting.co.za>
Message-ID: <1484106325.45.0.252448868836.issue28231@psf.upfronthosting.co.za>


Jeremy Freeman added the comment:

I have reviewed the code and docs for the public API that should take a pathlib.Path object:

- zipfile.is_zipfile(filename)
  - filename
- zipfile.ZipFile(file)
  - file
- ZipFile.extract(member, path=None)
  - path
- ZipFile.extractall(path=None)
  - path
- ZipFile.write(filename)
  - filename
- zipfile.PyZipFile(file)
  - file
- PyZipFile.writepy(pathname)
  - pathname
- ZipInfo.from_file(filename, arcname=None)
  - filename

Does this appear complete?

Working on tests that probe each of these API points with pathlib.Path objects.

I am not sure what "Shouldn't the ZipFile.filename attribute be converted to str?" means, can you elaborate?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28231>
_______________________________________

From report at bugs.python.org  Tue Jan 10 22:52:29 2017
From: report at bugs.python.org (Ethan Furman)
Date: Wed, 11 Jan 2017 03:52:29 +0000
Subject: [issue29235] Allow profile/cProfile to be used as context managers
In-Reply-To: <1484097220.43.0.363931456874.issue29235@psf.upfronthosting.co.za>
Message-ID: <1484106749.03.0.606749560311.issue29235@psf.upfronthosting.co.za>


Changes by Ethan Furman <ethan at stoneleaf.us>:


----------
nosy: +ethan.furman
stage:  -> needs patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29235>
_______________________________________

From report at bugs.python.org  Tue Jan 10 22:53:36 2017
From: report at bugs.python.org (Ethan Furman)
Date: Wed, 11 Jan 2017 03:53:36 +0000
Subject: [issue29238] Add more kwargs to cProfile's print_stats
In-Reply-To: <1484103631.29.0.405631580429.issue29238@psf.upfronthosting.co.za>
Message-ID: <1484106816.43.0.947342343416.issue29238@psf.upfronthosting.co.za>


Changes by Ethan Furman <ethan at stoneleaf.us>:


----------
nosy: +ethan.furman

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29238>
_______________________________________

From report at bugs.python.org  Tue Jan 10 23:51:31 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Wed, 11 Jan 2017 04:51:31 +0000
Subject: [issue29221] ABC Recursion Error on isinstance() with less than
 recursion limit class hierarchy depth
In-Reply-To: <1484021927.49.0.387886726852.issue29221@psf.upfronthosting.co.za>
Message-ID: <1484110291.92.0.687623629703.issue29221@psf.upfronthosting.co.za>


Raymond Hettinger added the comment:

Not sure if this is related, but I have a simple Cache class that gets a recursion error at a depth of about 166 calls (a sixth of the expected 1000).  I suspect that the reason is the many of the steps individually add to the recursion depth causing the recursion limit to be reached at some integer multiple faster than would be expected.

----------
nosy: +rhettinger

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29221>
_______________________________________

From report at bugs.python.org  Tue Jan 10 23:52:14 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Wed, 11 Jan 2017 04:52:14 +0000
Subject: [issue29221] ABC Recursion Error on isinstance() with less than
 recursion limit class hierarchy depth
In-Reply-To: <1484021927.49.0.387886726852.issue29221@psf.upfronthosting.co.za>
Message-ID: <1484110334.12.0.875250703517.issue29221@psf.upfronthosting.co.za>


Changes by Raymond Hettinger <raymond.hettinger at gmail.com>:


Added file: http://bugs.python.org/file46251/cache_example.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29221>
_______________________________________

From report at bugs.python.org  Wed Jan 11 00:09:13 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Wed, 11 Jan 2017 05:09:13 +0000
Subject: [issue29215] pyport.h uses non C90-style comment
In-Reply-To: <1483972129.08.0.0486776479108.issue29215@psf.upfronthosting.co.za>
Message-ID: <1484111353.85.0.0605311311492.issue29215@psf.upfronthosting.co.za>


Xiang Zhang added the comment:

About the comment style confliction, I opened an issue on Github and assigned to you Benjamin. https://github.com/python/peps/issues/176

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29215>
_______________________________________

From report at bugs.python.org  Wed Jan 11 00:35:36 2017
From: report at bugs.python.org (Larry Hastings)
Date: Wed, 11 Jan 2017 05:35:36 +0000
Subject: [issue29125] Shell injection via TIX_LIBRARY when using tkinter.tix
In-Reply-To: <1483210830.52.0.920210756798.issue29125@psf.upfronthosting.co.za>
Message-ID: <1484112936.85.0.576862232091.issue29125@psf.upfronthosting.co.za>


Larry Hastings added the comment:

Well, clearly I'm not qualified to review the patch.  Could someone please review it?  I want to cherry-pick the fix for this issue for 3.5.3 final, which I tag in about four days.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29125>
_______________________________________

From report at bugs.python.org  Wed Jan 11 00:37:43 2017
From: report at bugs.python.org (Larry Hastings)
Date: Wed, 11 Jan 2017 05:37:43 +0000
Subject: [issue29006] 2.7.13 _sqlite more prone to "database table is locked"
In-Reply-To: <1482095939.25.0.891516594682.issue29006@psf.upfronthosting.co.za>
Message-ID: <1484113063.44.0.303088583635.issue29006@psf.upfronthosting.co.za>


Larry Hastings added the comment:

Ping.  Hoping to resolve this in time for 3.5.3, which I tag in about four days.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29006>
_______________________________________

From report at bugs.python.org  Wed Jan 11 00:49:16 2017
From: report at bugs.python.org (Anthony Scopatz)
Date: Wed, 11 Jan 2017 05:49:16 +0000
Subject: [issue29221] ABC Recursion Error on isinstance() with less than
 recursion limit class hierarchy depth
In-Reply-To: <1484021927.49.0.387886726852.issue29221@psf.upfronthosting.co.za>
Message-ID: <1484113756.74.0.0553330031621.issue29221@psf.upfronthosting.co.za>


Anthony Scopatz added the comment:

It certainly seems related. Attached is an image that displays the scaling of the cache example.  The full notebook that generated this image is at [1]. The notebook shows that it does seem to converge towards a value of 1/6th. 

1. https://gist.github.com/scopatz/29b94326ec1f10056d27e9d9434b240a

----------
Added file: http://bugs.python.org/file46252/fibo.png

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29221>
_______________________________________

From report at bugs.python.org  Wed Jan 11 01:29:03 2017
From: report at bugs.python.org (=?utf-8?b?VmVkcmFuIMSMYcSNacSH?=)
Date: Wed, 11 Jan 2017 06:29:03 +0000
Subject: [issue27603] Migrate IDLE context menu items to shell extension
In-Reply-To: <1469332799.74.0.70339682021.issue27603@psf.upfronthosting.co.za>
Message-ID: <1484116143.72.0.465242109805.issue27603@psf.upfronthosting.co.za>


Vedran ?a?i? added the comment:

Even worse: I had both 3.5 and 3.6 installed, and then uninstalled 3.5 when 3.6 was final (I suppose I'm not the only one). Now I have the nested menu with only one item, Open with IDLE 3.6. :-\

----------
nosy: +veky

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue27603>
_______________________________________

From report at bugs.python.org  Wed Jan 11 01:48:46 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Wed, 11 Jan 2017 06:48:46 +0000
Subject: [issue29234] Disable inlining of _PyStack_AsTuple() to reduce the
 stack consumption
In-Reply-To: <1484093111.93.0.908854837348.issue29234@psf.upfronthosting.co.za>
Message-ID: <1484117326.14.0.846107232096.issue29234@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

What is a stack usage effect of disabling inlining _PyStack_AsTuple()? Does it impact performance? Should inlining _PyStack_AsDict() be disabled too? Is it worth to extract slow paths of _PyObject_FastCallDict() and _PyObject_FastCallKeywords() into separate non-inlined functions?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29234>
_______________________________________

From report at bugs.python.org  Wed Jan 11 01:51:48 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Wed, 11 Jan 2017 06:51:48 +0000
Subject: [issue28870] Reduce stack consumption of
 PyObject_CallFunctionObjArgs() and like
In-Reply-To: <1480891813.94.0.54795440575.issue28870@psf.upfronthosting.co.za>
Message-ID: <1484117508.63.0.924599718444.issue28870@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

Awesome! You are great Victor!

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28870>
_______________________________________

From report at bugs.python.org  Wed Jan 11 01:53:53 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Wed, 11 Jan 2017 06:53:53 +0000
Subject: [issue29125] Shell injection via TIX_LIBRARY when using tkinter.tix
In-Reply-To: <1483210830.52.0.920210756798.issue29125@psf.upfronthosting.co.za>
Message-ID: <1484117633.75.0.416115613982.issue29125@psf.upfronthosting.co.za>


Changes by Serhiy Storchaka <storchaka+cpython at gmail.com>:


----------
nosy: +terry.reedy, zach.ware

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29125>
_______________________________________

From report at bugs.python.org  Wed Jan 11 01:58:06 2017
From: report at bugs.python.org (Roundup Robot)
Date: Wed, 11 Jan 2017 06:58:06 +0000
Subject: [issue29220] Python 3.6 regression/change using
 logging.addLevelName() to clear a name
In-Reply-To: <1484016439.6.0.79629021761.issue29220@psf.upfronthosting.co.za>
Message-ID: <20170111065803.16773.48461.9191C02D@psf.io>


Roundup Robot added the comment:

New changeset 99ad6e871459 by Vinay Sajip in branch 'default':
Closes #29220: Fixed regression in logging.getLevelName().
https://hg.python.org/cpython/rev/99ad6e871459

----------
nosy: +python-dev
resolution:  -> fixed
stage:  -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29220>
_______________________________________

From report at bugs.python.org  Wed Jan 11 02:01:41 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Wed, 11 Jan 2017 07:01:41 +0000
Subject: [issue28231] zipfile does not support pathlib
In-Reply-To: <1474445503.11.0.219964117103.issue28231@psf.upfronthosting.co.za>
Message-ID: <1484118101.81.0.0199929426377.issue28231@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

> I am not sure what "Shouldn't the ZipFile.filename attribute be converted to str?" means, can you elaborate?

If you pass a Path object to ZipFile constructor, should the filename attribute be a Path object or a str?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28231>
_______________________________________

From report at bugs.python.org  Wed Jan 11 02:08:51 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Wed, 11 Jan 2017 07:08:51 +0000
Subject: [issue29220] Python 3.6 regression/change using
 logging.addLevelName() to clear a name
In-Reply-To: <1484016439.6.0.79629021761.issue29220@psf.upfronthosting.co.za>
Message-ID: <1484118531.7.0.152566848128.issue29220@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

Could you provide a working example of using your trick Mark? I think that even with empty name the output contains extra separators.

----------
nosy: +serhiy.storchaka

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29220>
_______________________________________

From report at bugs.python.org  Wed Jan 11 02:25:14 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Wed, 11 Jan 2017 07:25:14 +0000
Subject: [issue29220] Python 3.6 regression/change using
 logging.addLevelName() to clear a name
In-Reply-To: <1484016439.6.0.79629021761.issue29220@psf.upfronthosting.co.za>
Message-ID: <1484119514.7.0.812214771741.issue29220@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

As for the patch itself, I would add an indentation for the second "if":

    result = _levelToName.get(level)
    if result is None:
        result = _nameToLevel.get(level)
        if result is None:
            result = "Level %s" % level
    return result

or even use multiple returns for the sake of microoptimization:

    result = _levelToName.get(level)
    if result is not None:
        return result
    result = _nameToLevel.get(level)
    if result is not None:
        return result
    return "Level %s" % level

But I'm not sure that empty name is valid. It can cause problems when parse a configuration file or logs.

I don't understand the use of _nameToLevel. getLevelName('CRITICAL') returns 50, that even is not a string.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29220>
_______________________________________

From report at bugs.python.org  Wed Jan 11 02:37:30 2017
From: report at bugs.python.org (=?utf-8?b?0JzQsNGA0Log0JrQvtGA0LXQvdCx0LXRgNCz?=)
Date: Wed, 11 Jan 2017 07:37:30 +0000
Subject: [issue29214] Standard open() does not allow to specify file
 permissions.
In-Reply-To: <1483971421.1.0.906896747438.issue29214@psf.upfronthosting.co.za>
Message-ID: <1484120250.24.0.142291154922.issue29214@psf.upfronthosting.co.za>


???? ????????? added the comment:

@haypo, suppose, one thread wants file with permissions A, and other thread wants permissions B. If they will "set" them through umask(), race condition may occur.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29214>
_______________________________________

From report at bugs.python.org  Wed Jan 11 02:46:21 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Wed, 11 Jan 2017 07:46:21 +0000
Subject: [issue29214] Standard open() does not allow to specify file
 permissions.
In-Reply-To: <1483971421.1.0.906896747438.issue29214@psf.upfronthosting.co.za>
Message-ID: <1484120781.57.0.749021637275.issue29214@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

If you needs non-default permission for particular file, you can use one of available ways mentioned above.

I think this issue can be closed. The original issues are resolved:

1. os.open() (corresponding to syscall open()) allows this.
2. There are other ways to create Python file with specific permissions (pass fd or use opener).

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29214>
_______________________________________

From report at bugs.python.org  Wed Jan 11 02:55:12 2017
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 11 Jan 2017 07:55:12 +0000
Subject: [issue29214] Standard open() does not allow to specify file
 permissions.
In-Reply-To: <1483971421.1.0.906896747438.issue29214@psf.upfronthosting.co.za>
Message-ID: <1484121312.73.0.802668840059.issue29214@psf.upfronthosting.co.za>


STINNER Victor added the comment:

> @haypo, suppose, one thread wants file with permissions A, and other thread wants permissions B. If they will "set" them through umask(), race condition may occur.

Ok, let's say thta you want file "x" with permission A=0o777 and file "y" with permission B=0b700.

You start with os.umask(0o077), as suggested by Christian.

>>> import os
>>> os.umask(0o077)
2
>>> x=open("x", "w")
>>> y=open("y", "w")
>>> os.system("ls -l x y")
-rw-------. 1 haypo haypo 0 11 janv. 08:48 x
-rw-------. 1 haypo haypo 0 11 janv. 08:48 y
0
>>> os.fchmod(x.fileno(), 0o777)
>>> os.system("ls -l x y")
-rwxrwxrwx. 1 haypo haypo 0 11 janv. 08:48 x
-rw-------. 1 haypo haypo 0 11 janv. 08:48 y
0

This is a time window where the file "x" has the permission -rw------- instead of the expected -rwxrwxrwx. Ok, it's expected. But it's not an easy since initial permissions are are more strict that expected permisions.

The race condition only occurs if you work the other way, start with os.umask(0777) (default umask) and then change permissions of the file "x" to stricter. In this specific case, you *already* have two ways to do that in Python 3 (no change needed):

  fp = open("x", opener=partial(os.open, mode=0o700))
or:
  fd = os.open("x", os.O_WRONLY | os.O_CREAT, 0o700)
  fp = open(fd, "wb")

--

About documenting the default mode: the problem is that I'm not sure that *all* functions in Python, especially when using third party libraries, use the same default mode. Morever, the mode on Windows has a very different meaning. Permission per group is handled by ACLs, not with the integer "mode":
https://msdn.microsoft.com/en-us/library/z0kc8e3z.aspx

On Windows, pmode of _open() only contains read and write permissions of the file, for everyone.

For all these reasons, I close the issue.

----------
resolution:  -> fixed
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29214>
_______________________________________

From report at bugs.python.org  Wed Jan 11 03:04:18 2017
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 11 Jan 2017 08:04:18 +0000
Subject: [issue28870] Reduce stack consumption of
 PyObject_CallFunctionObjArgs() and like
In-Reply-To: <1480891813.94.0.54795440575.issue28870@psf.upfronthosting.co.za>
Message-ID: <1484121858.68.0.912190416951.issue28870@psf.upfronthosting.co.za>


STINNER Victor added the comment:

I also ran the reliable performance benchmark suite with LTO+PGO. There is no significant performance change on these benchmarks:
https://speed.python.org/changes/?rev=b9404639a18c&exe=5&env=speed-python

The largest change is on scimark_lu (-13%), but there was an hiccup on the previous change which is probably a small unstability in the benchmark. It's not a speedup of these changes.

The second largest change is on spectral_norm: +9%. But this benchmark is known to be unstable, there was already a small peak previously. Again, I don't think that it's related to the changes.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28870>
_______________________________________

From report at bugs.python.org  Wed Jan 11 03:04:34 2017
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 11 Jan 2017 08:04:34 +0000
Subject: [issue29234] Disable inlining of _PyStack_AsTuple() to reduce the
 stack consumption
In-Reply-To: <1484093111.93.0.908854837348.issue29234@psf.upfronthosting.co.za>
Message-ID: <1484121874.51.0.269707405178.issue29234@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Full commit message:
---
Disable _PyStack_AsTuple() inlining

Issue #29234: Inlining _PyStack_AsTuple() into callers increases their stack
consumption, Disable inlining to optimize the stack consumption.

Add _Py_NO_INLINE: use __attribute__((noinline)) of GCC and Clang.

It reduces the stack consumption, bytes per call, before => after:

test_python_call: 1040 => 976 (-64 B)
test_python_getitem: 976 => 912 (-64 B)
test_python_iterator: 1120 => 1056 (-64 B)

=> total: 3136 => 2944 (- 192 B)
---

Serhiy Storchaka: "What is a stack usage effect of disabling inlining _PyStack_AsTuple()?"

The total effect on the 3 tests is to reduce the stack consumption by 192 bytes/call, or 64 bytes/call (8 CPU words) for each test.


> Does it impact performance?

I ran a benchmark on 3 changes at once. The effect is a speedup, not a slowdown:
http://bugs.python.org/issue28870#msg285173

I don't expect any significant performance impact for the change 6478e6d0476f.


> Should inlining _PyStack_AsDict() be disabled too?

Good question. I didn't try to write a benchmark calling this function. It would help to have numbers to take a decision.

I tried to push the fewer changes which have the largest impact on the stack consumption. There is still room to reduce it even further.


> Is it worth to extract slow paths of _PyObject_FastCallDict() and _PyObject_FastCallKeywords() into separate non-inlined functions?

Do you mean for performance or stack consumption? I don't know. If you would like to know, you should run a benchmark to measure that.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29234>
_______________________________________

From report at bugs.python.org  Wed Jan 11 04:11:58 2017
From: report at bugs.python.org (=?utf-8?q?Vlastimil_Z=C3=ADma?=)
Date: Wed, 11 Jan 2017 09:11:58 +0000
Subject: [issue20804] Sentinels identity lost when pickled (unittest.mock)
In-Reply-To: <1393579357.22.0.503856996983.issue20804@psf.upfronthosting.co.za>
Message-ID: <1484125918.26.0.679376066011.issue20804@psf.upfronthosting.co.za>


Vlastimil Z?ma added the comment:

Since the issue was dug up, I've created a patch for documentation.

I still think, that it is worth stating the equality is not preserved for sentinels when copied as the message produced by tests is rather confusing:

    class SentinelTestCase(TestCase):
        def test_sentinel(self):
            self.assertEqual(sentinel.foo, copy(sentinel.foo))
            # AssertionError: sentinel.foo != sentinel.foo

----------
Added file: http://bugs.python.org/file46253/sentinel-doc.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue20804>
_______________________________________

From report at bugs.python.org  Wed Jan 11 04:15:42 2017
From: report at bugs.python.org (Michael Foord)
Date: Wed, 11 Jan 2017 09:15:42 +0000
Subject: [issue20804] Sentinels identity lost when pickled (unittest.mock)
In-Reply-To: <1393579357.22.0.503856996983.issue20804@psf.upfronthosting.co.za>
Message-ID: <1484126142.27.0.966521922221.issue20804@psf.upfronthosting.co.za>


Michael Foord added the comment:

So my thinking has evolved slightly on it. If it's *possible* for sentinels to be copied and pickled and preserve identity then I'm happy for it. I think the right semantics for copying a sentinel is that you get the original object back. If you pickle a sentinel and then unpickle back *into the same process* you should get the same object back. 

David worried that this confuses the semantics of pickling/copying because it isn't generally the case - so I'm open to further discussion on it, but if Serhiy has fixed the "problem" here I'm happy for it to go in.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue20804>
_______________________________________

From report at bugs.python.org  Wed Jan 11 04:21:51 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Wed, 11 Jan 2017 09:21:51 +0000
Subject: [issue29099] sqlite3 timestamp converter cannot handle timezone
In-Reply-To: <1482966269.67.0.620298847284.issue29099@psf.upfronthosting.co.za>
Message-ID: <1484126511.15.0.854652467664.issue29099@psf.upfronthosting.co.za>


Changes by Xiang Zhang <angwerzx at 126.com>:


Added file: http://bugs.python.org/file46254/sqlite3-3.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29099>
_______________________________________

From report at bugs.python.org  Wed Jan 11 04:40:23 2017
From: report at bugs.python.org (Vinay Sajip)
Date: Wed, 11 Jan 2017 09:40:23 +0000
Subject: [issue29220] Python 3.6 regression/change using
 logging.addLevelName() to clear a name
In-Reply-To: <1484016439.6.0.79629021761.issue29220@psf.upfronthosting.co.za>
Message-ID: <1484127623.82.0.302601086874.issue29220@psf.upfronthosting.co.za>


Vinay Sajip added the comment:

> I think that even with empty name the output contains extra separators

Not sure what you mean - please clarify with an example.

> I would add an indentation for the second "if" ... or even use multiple returns for the sake of microoptimization

I will do a further update soon.

> I don't understand the use of _nameToLevel. getLevelName('CRITICAL') returns 50, that even is not a string.

Historically, getLevelName() gets the name for a level *but also the level for a name* (I perhaps should have done a separate API for that, like getLevelForName(), but it's too late now due to backward compatibility requirements - it's been like this for a long time.)

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29220>
_______________________________________

From report at bugs.python.org  Wed Jan 11 05:05:57 2017
From: report at bugs.python.org (Martin Panter)
Date: Wed, 11 Jan 2017 10:05:57 +0000
Subject: [issue15216] Support setting the encoding on a text stream after
 creation
In-Reply-To: <1340880558.16.0.0136840668667.issue15216@psf.upfronthosting.co.za>
Message-ID: <1484129157.94.0.477811292309.issue15216@psf.upfronthosting.co.za>


Martin Panter added the comment:

Inada, I think you messed up the positioning of bits of the patch. E.g. there are now test methods declared inside a helper function (rather than a test class).

Since it seems other people are in favour of this API, I would like to expand it a bit to cover two uses cases (see set_encoding-newline.patch):

* change the error handler without affecting the main character encoding
* set the newline encoding (also suggested by Serhiy)

Regarding Serhiy?s other suggestion about buffering parameters, perhaps TextIOWrapper.line_buffering could become a writable attribute instead, and the class could grow a similar write_through attribute. I don?t think these affect encoding or decoding, so they could be treated independently.

The algorithm for rewinding unread data is complicated and can fail. What is the advantage of using it? What is the use case for reading from a stream and then changing the encoding, without a guarantee that it will work?

Even if it is enhanced to never ?fail?, it will still have strange behaviour, such as data loss when a decoder is fed a single byte and produces multiple characters (e.g. CR newline, backslashreplace, UTF-7).

One step in the right direction IMO would be to only support calling set_encoding() when no extra read data has been buffered (or to explicitly say that any buffered data is silently dropped). So there is no support for changing the encoding halfway through a disk file, but it may be appropriate if you can regulate the bytes being read, e.g. from a terminal (user input), pipe, socket, etc.

But I would be happy enough without set_encoding(), and with something like my rewrap() function at the bottom of <https://github.com/vadmium/data/blob/master/data.py#L526>. It returns a fresh TextIOWrapper, but when you exit the context manager you can continue to reuse the old stream with the old settings.

----------
Added file: http://bugs.python.org/file46255/set_encoding-newline.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue15216>
_______________________________________

From report at bugs.python.org  Wed Jan 11 05:07:06 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Wed, 11 Jan 2017 10:07:06 +0000
Subject: [issue29220] Python 3.6 regression/change using
 logging.addLevelName() to clear a name
In-Reply-To: <1484016439.6.0.79629021761.issue29220@psf.upfronthosting.co.za>
Message-ID: <1484129226.7.0.984886247085.issue29220@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

> Historically, getLevelName() gets the name for a level *but also the level for a name*

Ah, then getLevelName() was broken by issue27937.

>>> logging.getLevelName('NOTSET')
'Level NOTSET'

The fix should be applied to 3.5.

----------
resolution: fixed -> 
stage: resolved -> needs patch
status: closed -> open
versions: +Python 3.5, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29220>
_______________________________________

From report at bugs.python.org  Wed Jan 11 05:23:31 2017
From: report at bugs.python.org (Armin Rigo)
Date: Wed, 11 Jan 2017 10:23:31 +0000
Subject: [issue29006] 2.7.13 _sqlite more prone to "database table is locked"
In-Reply-To: <1482095939.25.0.891516594682.issue29006@psf.upfronthosting.co.za>
Message-ID: <1484130211.64.0.120912362991.issue29006@psf.upfronthosting.co.za>


Armin Rigo added the comment:

If I had a say, I would indeed revert 030e100f048a (2.7 branch) and 81f614dd8136 (3.5 branch) as well as forward-port the revert to 3.6 and trunk.  Then we wait for someone that really knows why the change was done in the first place.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29006>
_______________________________________

From report at bugs.python.org  Wed Jan 11 05:27:25 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Wed, 11 Jan 2017 10:27:25 +0000
Subject: [issue20804] Sentinels identity lost when pickled (unittest.mock)
In-Reply-To: <1393579357.22.0.503856996983.issue20804@psf.upfronthosting.co.za>
Message-ID: <1484130445.04.0.80046099466.issue20804@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

I just don't know well the purpose of sentinels, and don't . I see fourth possible options:

1. Raise an exception when copy or pickle a sentinel. This is an option when we want to be sure that copying and pickling are not involved and we get the same object as passed.

2. Create an unique sentinel with unique unambiguous name when copy or unpickle a sentinel. This is a behavior of commonly used sentinels (object(), ['sentinel'], etc), and current behavior of mock.sentinel (except that the name is left the same and this is confusing).

3. Preserve identity when copy or pickle/unpickle a sentinel. This is an option when test a function that uses multiprocessing or persistent cache and should restore an equivalent object. This is a behavior of enums, classes, functions or other global named objects.

4. Preserve equality but not identity when copy or pickle/unpickle a sentinel. This is similar to the third option.

All options look reasonable to me. I don't know what case is more common, what behavior is more expectable.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue20804>
_______________________________________

From report at bugs.python.org  Wed Jan 11 05:28:19 2017
From: report at bugs.python.org (Jim Fasarakis-Hilliard)
Date: Wed, 11 Jan 2017 10:28:19 +0000
Subject: [issue29239] Fix wrong issue number in what's new entry
Message-ID: <1484130499.71.0.0502378123939.issue29239@psf.upfronthosting.co.za>


Changes by Jim Fasarakis-Hilliard <d.f.hilliard at gmail.com>:


----------
assignee: docs at python
components: Documentation
files: fix_issue.patch
keywords: patch
nosy: Jim Fasarakis-Hilliard, docs at python
priority: normal
severity: normal
status: open
title: Fix wrong issue number in what's new entry
versions: Python 3.6
Added file: http://bugs.python.org/file46256/fix_issue.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29239>
_______________________________________

From report at bugs.python.org  Wed Jan 11 05:29:04 2017
From: report at bugs.python.org (Christian Heimes)
Date: Wed, 11 Jan 2017 10:29:04 +0000
Subject: [issue29214] Standard open() does not allow to specify file
 permissions.
In-Reply-To: <1483971421.1.0.906896747438.issue29214@psf.upfronthosting.co.za>
Message-ID: <1484130544.6.0.9085273276.issue29214@psf.upfronthosting.co.za>


Christian Heimes added the comment:

I like the partial opener trick. Can we have the examples in the public documentation of open(), too?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29214>
_______________________________________

From report at bugs.python.org  Wed Jan 11 05:35:16 2017
From: report at bugs.python.org (INADA Naoki)
Date: Wed, 11 Jan 2017 10:35:16 +0000
Subject: [issue15216] Support setting the encoding on a text stream after
 creation
In-Reply-To: <1340880558.16.0.0136840668667.issue15216@psf.upfronthosting.co.za>
Message-ID: <1484130916.32.0.662761708206.issue15216@psf.upfronthosting.co.za>


INADA Naoki added the comment:

> Inada, I think you messed up the positioning of bits of the patch. E.g. there are now test methods declared > inside a helper function (rather than a test class).

I'm sorry.  `patch -p1` merged previous patch into wrong place, and test passed accidently.

> Since it seems other people are in favour of this API, I would like to expand it a bit to cover two uses  cases (see set_encoding-newline.patch):
> 
> * change the error handler without affecting the main character encoding
> * set the newline encoding (also suggested by Serhiy)

+1.  Since stdio is configured before running Python program, TextIOWrapper should be configurable after creation, as possible.

> Regarding Serhiy?s other suggestion about buffering parameters, perhaps TextIOWrapper.line_buffering could become a writable attribute instead, and the class could grow a similar write_through attribute. I don?t think these affect encoding or decoding, so they could be treated independently.

Could them go another new issue?
This issue is too long to read already.

> The algorithm for rewinding unread data is complicated and can fail. What is the advantage of using it? What is the use case for reading from a stream and then changing the encoding, without a guarantee that it will work?
>
> Even if it is enhanced to never ?fail?, it will still have strange behaviour, such as data loss when a decoder is fed a single byte and produces multiple characters (e.g. CR newline, backslashreplace, UTF-7).

When I posted the set_encoding-7.patch, I hadn't read io module deeply.  I just solved conflict and ran test.
After that, I read the code and I feel same thing (see msg285111 and msg285112).
Let's drop support changing encoding while reading.
It's significant step that allowing changing stdin encoding only before reading anything from it.


> One step in the right direction IMO would be to only support calling set_encoding() when no extra read data has been buffered (or to explicitly say that any buffered data is silently dropped). So there is no support for changing the encoding halfway through a disk file, but it may be appropriate if you can regulate the bytes being read, e.g. from a terminal (user input), pipe, socket, etc.

Totally agree.


> But I would be happy enough without set_encoding(), and with something like my rewrap() function at the bottom of <https://github.com/vadmium/data/blob/master/data.py#L526>. It returns a fresh TextIOWrapper, but when you exit the context manager you can continue to reuse the old stream with the old settings.

I want one obvious way to control encoding and error handler from Python, (not from environment variable).
Rewrapping stream seems hacky way, rather than obvious way.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue15216>
_______________________________________

From report at bugs.python.org  Wed Jan 11 05:38:56 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Wed, 11 Jan 2017 10:38:56 +0000
Subject: [issue29099] sqlite3 timestamp converter cannot handle timezone
In-Reply-To: <1482966269.67.0.620298847284.issue29099@psf.upfronthosting.co.za>
Message-ID: <1484131136.25.0.729915682536.issue29099@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

LGTM except that arguments of assertEqual() should be swapped.

----------
stage: patch review -> commit review

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29099>
_______________________________________

From report at bugs.python.org  Wed Jan 11 05:52:06 2017
From: report at bugs.python.org (Michael Foord)
Date: Wed, 11 Jan 2017 10:52:06 +0000
Subject: [issue20804] Sentinels identity lost when pickled (unittest.mock)
In-Reply-To: <1393579357.22.0.503856996983.issue20804@psf.upfronthosting.co.za>
Message-ID: <1484131926.31.0.502786003251.issue20804@psf.upfronthosting.co.za>


Michael Foord added the comment:

I think option 3 is the correct semantic behaviour for sentinels, and if there are already examples of this in the standard library then it *doesn't* violate expected behaviour of pickling and copying (sentinel and singleton objects can be permitted to retain this property through copying and pickling if it is a document facet of their behaviour and a natural result of their use cases).

Sentinels exist *purely* to have unique, *identifiable* objects. So retaining identity is their "expected behaviour" as identity is their defining feature. 

So Serhiy, if you're happy that you've implemented this correctly - with tests and documentation updates (I see a separate doc patch) - go ahead and commit.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue20804>
_______________________________________

From report at bugs.python.org  Wed Jan 11 06:02:14 2017
From: report at bugs.python.org (Jim Fasarakis-Hilliard)
Date: Wed, 11 Jan 2017 11:02:14 +0000
Subject: [issue29116] Make str and bytes error messages on concatenation
 conform with other sequences
In-Reply-To: <1483124211.75.0.715869443236.issue29116@psf.upfronthosting.co.za>
Message-ID: <1484132534.85.0.743297397399.issue29116@psf.upfronthosting.co.za>


Jim Fasarakis-Hilliard added the comment:

As I currently see this: 

 - The error message for str can be changed to the one used for other sequences 'can only concatenate str (not "type") to str'

 - The error message for arrays can be changed to use concatenate instead of append, too.

For bytes I see a conundrum, on one hand it can be changed to mention the buffer protocol which might confuse new users more than it helps them or, it can try and mention the objects that currently conform to it which might be exhaustive and long.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29116>
_______________________________________

From report at bugs.python.org  Wed Jan 11 06:19:52 2017
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 11 Jan 2017 11:19:52 +0000
Subject: [issue29240] Implementation of the PEP 540: Add a new UTF-8 mode
Message-ID: <1484133592.17.0.263024020483.issue29240@psf.upfronthosting.co.za>


New submission from STINNER Victor:

This issue tracks the implementation of the PEP 540.

Attached pep540_cli.py script can be used to play with it.

----------
components: Interpreter Core, Library (Lib), Unicode
files: pep540_cli.py
messages: 285214
nosy: ezio.melotti, haypo
priority: normal
pull_requests: 15
severity: normal
status: open
title: Implementation of the PEP 540: Add a new UTF-8 mode
type: enhancement
versions: Python 3.7
Added file: http://bugs.python.org/file46257/pep540_cli.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29240>
_______________________________________

From report at bugs.python.org  Wed Jan 11 06:27:22 2017
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 11 Jan 2017 11:27:22 +0000
Subject: [issue29240] Implementation of the PEP 540: Add a new UTF-8 mode
In-Reply-To: <1484133592.17.0.263024020483.issue29240@psf.upfronthosting.co.za>
Message-ID: <1484134042.51.0.347511837763.issue29240@psf.upfronthosting.co.za>


STINNER Victor added the comment:

pep540.patch: first draft

Changes:

* Add sys.flags.utf8mode
* Add -X utf8 command line option
* Add PYTHONUTF8 environment variable
* sys.stdin, sys.stdout and sys.stderr encoding and errors are modified in UTF-8 mode
* open() default encoding and errors is modified in the UTF-8 mode
* Add Lib/test/test_utf8mode.py
* Skip a few tests relying on the locale encoding if the UTF-8 mode is enabled
* Document changes

Allowed options:

* Disable UTF-8 mode: -X utf8=0 or PYTHONUTF8=0
* Enable UTF-8 mode: -X utf8=1 or PYTHONUTF8=1
* Enable UTf-8 Strict mode: -X utf8=strict or PYTHONUTF8=strict
* Other -X utf8 and PYTHONUTF8 values cause a fatal error

Prioririties (highest to lowest):

* open() encoding and errors arguments
* PYTHONIOENCODING
* UTF-8 mode
* os.device_encoding()
* locale encoding

TODO:

* re-encode sys.argv from the local encoding to UTF-8 in Py_Main() when the UTF-8 mode is enabled
* support strict mode in Py_DecodeLocale() and Py_EncodeLocale()

----------
keywords: +patch
Added file: http://bugs.python.org/file46258/pep540.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29240>
_______________________________________

From report at bugs.python.org  Wed Jan 11 06:32:58 2017
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 11 Jan 2017 11:32:58 +0000
Subject: [issue29240] Implementation of the PEP 540: Add a new UTF-8 mode
In-Reply-To: <1484133592.17.0.263024020483.issue29240@psf.upfronthosting.co.za>
Message-ID: <1484134378.67.0.443227597254.issue29240@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Examples with pep540_cli.py.

Python 3.5:

$ python3 pep540_cli.py 
sys.argv: ['pep540_cli.py']
stdin: UTF-8/strict
stdout: UTF-8/strict
stderr: UTF-8/backslashreplace
open(): UTF-8/strict

$ LC_ALL=C python3 pep540_cli.py 
sys.argv: ['pep540_cli.py']
stdin: ANSI_X3.4-1968/surrogateescape
stdout: ANSI_X3.4-1968/surrogateescape
stderr: ANSI_X3.4-1968/backslashreplace
open(): ANSI_X3.4-1968/strict


Patched Python 3.7:


$ ./python pep540_cli.py 
UTF-8 mode: 0
sys.argv: ['pep540_cli.py']
stdin: UTF-8/strict
stdout: UTF-8/strict
stderr: UTF-8/backslashreplace
open(): UTF-8/strict

$ LC_ALL=C ./python pep540_cli.py 
UTF-8 mode: 1
sys.argv: ['pep540_cli.py']
stdin: utf-8/surrogateescape
stdout: utf-8/surrogateescape
stderr: utf-8/backslashreplace
open(): utf-8/surrogateescape

$ ./python -X utf8 pep540_cli.py 
UTF-8 mode: 1
sys.argv: ['pep540_cli.py']
stdin: utf-8/surrogateescape
stdout: utf-8/surrogateescape
stderr: utf-8/backslashreplace
open(): utf-8/surrogateescape

$ ./python -X utf8=strict pep540_cli.py 
UTF-8 mode: 2
sys.argv: ['pep540_cli.py']
stdin: utf-8/strict
stdout: utf-8/strict
stderr: utf-8/backslashreplace
open(): utf-8/strict

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29240>
_______________________________________

From report at bugs.python.org  Wed Jan 11 07:16:00 2017
From: report at bugs.python.org (Roundup Robot)
Date: Wed, 11 Jan 2017 12:16:00 +0000
Subject: [issue29239] Fix wrong issue number in what's new entry
Message-ID: <20170111121424.1300.47311.61AECB9E@psf.io>


New submission from Roundup Robot:

New changeset 114b03fa4c04 by Martin Panter in branch '3.6':
Issue #29239: Fix --enable-optimizations bug number
https://hg.python.org/cpython/rev/114b03fa4c04

New changeset 4e29c7f2b3e5 by Martin Panter in branch 'default':
Issue #29239: Merge bug number from 3.6
https://hg.python.org/cpython/rev/4e29c7f2b3e5

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29239>
_______________________________________

From report at bugs.python.org  Wed Jan 11 07:16:00 2017
From: report at bugs.python.org (Roundup Robot)
Date: Wed, 11 Jan 2017 12:16:00 +0000
Subject: [issue15657] Error in Python 3 docs for PyMethodDef
In-Reply-To: <1344970966.7.0.115664419243.issue15657@psf.upfronthosting.co.za>
Message-ID: <20170111121424.1300.61365.6050961A@psf.io>


Roundup Robot added the comment:

New changeset d7d2d24003f5 by Martin Panter in branch '3.5':
Issue #15657: METH_KEYWORDS cannot be used alone in Python 3
https://hg.python.org/cpython/rev/d7d2d24003f5

New changeset c140e72492a4 by Martin Panter in branch '3.6':
Issue #15657: Delete incorrect statement from PyMethodDef documentation
https://hg.python.org/cpython/rev/c140e72492a4

New changeset 021fd2ff7ca4 by Martin Panter in branch '3.6':
Issue #15657: Merge other doc fix from 3.5
https://hg.python.org/cpython/rev/021fd2ff7ca4

New changeset 1058e151049a by Martin Panter in branch 'default':
Issue #15657: Merge METH_KEYWORDS doc from 3.6
https://hg.python.org/cpython/rev/1058e151049a

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue15657>
_______________________________________

From report at bugs.python.org  Wed Jan 11 07:23:46 2017
From: report at bugs.python.org (Martin Panter)
Date: Wed, 11 Jan 2017 12:23:46 +0000
Subject: [issue29239] Fix wrong issue number in what's new entry
In-Reply-To: <20170111121424.1300.47311.61AECB9E@psf.io>
Message-ID: <1484137426.87.0.118289891904.issue29239@psf.upfronthosting.co.za>


Martin Panter added the comment:

Thanks Jim

----------
nosy: +martin.panter
resolution:  -> fixed
stage:  -> resolved
status: open -> closed
versions: +Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29239>
_______________________________________

From report at bugs.python.org  Wed Jan 11 07:28:01 2017
From: report at bugs.python.org (JGoutin)
Date: Wed, 11 Jan 2017 12:28:01 +0000
Subject: [issue29241] sys._enablelegacywindowsfsencoding() don't apply to
 os.fsencode and os.fsdecode
Message-ID: <1484137681.59.0.0301192140292.issue29241@psf.upfronthosting.co.za>


New submission from JGoutin:

The doc say that calling "sys._enablelegacywindowsfsencoding()" is equivalent to use "PYTHONLEGACYWINDOWSFSENCODING" environment variable.

In fact, this no apply to "os.fsencode" and "os.fsdecode".

Example with Python 3.6 64Bits on Windows 7 64 bits :


EXAMPLE CODE 1 (sys._enablelegacywindowsfsencoding()): 

import sys
import os

# Force the use of legacy encoding like versions of Python prior to 3.6.
sys._enablelegacywindowsfsencoding()

# Show actual file system encoding
encoding = sys.getfilesystemencoding()
print('File system encoding:', encoding)

# os.fsencode(filename) VS filename.encode(File system encoding)
print(os.fsencode('?'), '?'.encode(encoding))

>>> File system encoding: mbcs
>>> b'\xc3\xa9' b'\xe9'


First is encoded with "utf-8" and not "mbcs" (The actual File system encoding)


EXAMPLE CODE 2 (PYTHONLEGACYWINDOWSFSENCODING): 

import sys
import os

# Force the use of legacy encoding like versions of Python prior to 3.6.
# "PYTHONLEGACYWINDOWSFSENCODING" environment variable set before running Python.

# Show actual file system encoding
encoding = sys.getfilesystemencoding()
print('File system encoding:', encoding)

# os.fsencode(filename) VS filename.encode(File system encoding)
print(os.fsencode('?'), '?'.encode(encoding))

>>> File system encoding: mbcs
>>> b'\xe9' b'\xe9'


Everything encoded with "mbcs" (The actual File system encoding)


In "os.fsencode" and "os.fsdecode" encoding and errors are cached on start and never updated by "sys._enablelegacywindowsfsencoding()" after.

----------
components: Windows
messages: 285220
nosy: JGoutin, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: sys._enablelegacywindowsfsencoding() don't apply to os.fsencode and os.fsdecode
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29241>
_______________________________________

From report at bugs.python.org  Wed Jan 11 07:50:57 2017
From: report at bugs.python.org (INADA Naoki)
Date: Wed, 11 Jan 2017 12:50:57 +0000
Subject: [issue15216] Support setting the encoding on a text stream after
 creation
In-Reply-To: <1340880558.16.0.0136840668667.issue15216@psf.upfronthosting.co.za>
Message-ID: <1484139057.19.0.390937596211.issue15216@psf.upfronthosting.co.za>


INADA Naoki added the comment:

set_encoding-8.patch dropped support of changing encoding after read.
It is based on set_encoding-newline.patch

----------
Added file: http://bugs.python.org/file46259/set_encoding-8.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue15216>
_______________________________________

From report at bugs.python.org  Wed Jan 11 08:09:06 2017
From: report at bugs.python.org (Charalampos Stratakis)
Date: Wed, 11 Jan 2017 13:09:06 +0000
Subject: [issue1294959] Problems with /usr/lib64 builds.
Message-ID: <1484140146.85.0.633748658368.issue1294959@psf.upfronthosting.co.za>


Charalampos Stratakis added the comment:

The downstream patch we currently use in Fedora [0].

[0] http://pkgs.fedoraproject.org/cgit/rpms/python3.git/plain/00102-lib64.patch

----------
nosy: +cstratak

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue1294959>
_______________________________________

From report at bugs.python.org  Wed Jan 11 08:38:22 2017
From: report at bugs.python.org (Shane Smith)
Date: Wed, 11 Jan 2017 13:38:22 +0000
Subject: [issue27603] Migrate IDLE context menu items to shell extension
In-Reply-To: <1469332799.74.0.70339682021.issue27603@psf.upfronthosting.co.za>
Message-ID: <1484141902.86.0.370138623119.issue27603@psf.upfronthosting.co.za>


Shane Smith added the comment:

Hi Vedran, that seems to now be the default behavior, regardless of previous installs (my 3.6 was a clean install, and it's still nested).  Kind of a pain if you want to edit with IDLE frequently.

While we wait for a more integrated solution, the hackish way to restore desired behavior is to run a .bat file containing:

REG DELETE "HKEY_CLASSES_ROOT\Python.File\shell\editwithidle"

Then run a .reg file (also just a text file with altered extension so windows knows what to do with it) with:

Windows Registry Editor Version 5.00

[HKEY_CLASSES_ROOT\Python.File\shell\Edit with IDLE 3.6\command]
@="\"C:\\Program Files\\Python36\\pythonw.exe\" -m idlelib \"%L\" %*"

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue27603>
_______________________________________

From report at bugs.python.org  Wed Jan 11 09:16:19 2017
From: report at bugs.python.org (Vinay Sajip)
Date: Wed, 11 Jan 2017 14:16:19 +0000
Subject: [issue29220] Python 3.6 regression/change using
 logging.addLevelName() to clear a name
In-Reply-To: <1484016439.6.0.79629021761.issue29220@psf.upfronthosting.co.za>
Message-ID: <1484144179.57.0.382332382802.issue29220@psf.upfronthosting.co.za>


Vinay Sajip added the comment:

> The fix should be applied to 3.5.

Are you sure?

$ python3.5
Python 3.5.2 (default, Nov 17 2016, 17:05:23) 
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import logging
>>> logging.getLevelName(logging.NOTSET)
'NOTSET'
>>> logging.getLevelName('NOTSET')
0
>>> 

I think just 3.6 and default are affected - doesn't 3.5 seem OK?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29220>
_______________________________________

From report at bugs.python.org  Wed Jan 11 09:25:03 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Wed, 11 Jan 2017 14:25:03 +0000
Subject: [issue29220] Python 3.6 regression/change using
 logging.addLevelName() to clear a name
In-Reply-To: <1484016439.6.0.79629021761.issue29220@psf.upfronthosting.co.za>
Message-ID: <1484144703.54.0.584303956271.issue29220@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

I'm sure. e4b6faf22e8d is in 3.5 branch.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29220>
_______________________________________

From report at bugs.python.org  Wed Jan 11 09:29:38 2017
From: report at bugs.python.org (Vinay Sajip)
Date: Wed, 11 Jan 2017 14:29:38 +0000
Subject: [issue29220] Python 3.6 regression/change using
 logging.addLevelName() to clear a name
In-Reply-To: <1484016439.6.0.79629021761.issue29220@psf.upfronthosting.co.za>
Message-ID: <1484144978.77.0.411787122743.issue29220@psf.upfronthosting.co.za>


Vinay Sajip added the comment:

OK, will apply to 3.5 -> default.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29220>
_______________________________________

From report at bugs.python.org  Wed Jan 11 09:52:40 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Wed, 11 Jan 2017 14:52:40 +0000
Subject: [issue29099] sqlite3 timestamp converter cannot handle timezone
In-Reply-To: <1482966269.67.0.620298847284.issue29099@psf.upfronthosting.co.za>
Message-ID: <1484146360.58.0.611069410159.issue29099@psf.upfronthosting.co.za>


Xiang Zhang added the comment:

> LGTM except that arguments of assertEqual() should be swapped.

Thanks Serhiy! I'll take care of that. One problem remained is what to do with 2.7. There is no timezone object in 2.7. My preference is just pointing out the limitation that only naive datetime object is supported in the doc.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29099>
_______________________________________

From report at bugs.python.org  Wed Jan 11 10:03:16 2017
From: report at bugs.python.org (Dingyuan Wang)
Date: Wed, 11 Jan 2017 15:03:16 +0000
Subject: [issue29242] Crash on GC when compiling PyPy
Message-ID: <1484146996.34.0.185188210895.issue29242@psf.upfronthosting.co.za>


New submission from Dingyuan Wang:

When compiling the PyPy default branch [1] on a Debian testing machine with Python 2.7.13, cpython randomly crashes.

(gdb) bt
#0  update_refs () at ../Modules/gcmodule.c:332
#1  collect.lto_priv () at ../Modules/gcmodule.c:924
#2  0x000055555562a804 in collect_generations () at ../Modules/gcmodule.c:1050
#3  _PyObject_GC_Malloc () at ../Modules/gcmodule.c:1511
#4  0x0000555555640ef5 in PyType_GenericAlloc (nitems=0, type=0x555555a9dd20 <_PyExc_AttributeError>) at ../Objects/typeobject.c:781
#5  BaseException_new.lto_priv.68 () at ../Objects/exceptions.c:34
#6  0x0000555555642093 in type_call.lto_priv () at ../Objects/typeobject.c:749
#7  0x000055555563c5f3 in PyObject_Call () at ../Objects/abstract.c:2547
#8  0x000055555565a490 in PyEval_CallObjectWithKeywords () at ../Python/ceval.c:4221
#9  0x00005555556695d0 in PyErr_NormalizeException () at ../Python/errors.c:192
#10 0x0000555555656cad in PyEval_EvalFrameEx () at ../Python/ceval.c:3251
#11 0x000055555564e525 in PyEval_EvalCodeEx () at ../Python/ceval.c:3584
#12 0x000055555566ac2e in function_call.lto_priv () at ../Objects/funcobject.c:523
#13 0x000055555563c5f3 in PyObject_Call () at ../Objects/abstract.c:2547
#14 0x000055555568102e in instancemethod_call.lto_priv () at ../Objects/classobject.c:2602
#15 0x000055555563c5f3 in PyObject_Call () at ../Objects/abstract.c:2547
#16 0x00005555556f333a in call_method.lto_priv () at ../Objects/typeobject.c:1283
#17 0x000055555573eb8f in slot_tp_setattro.lto_priv () at ../Objects/typeobject.c:5654
#18 0x00005555556667e9 in PyObject_SetAttr () at ../Objects/object.c:1247
#19 0x0000555555650ec3 in PyEval_EvalFrameEx () at ../Python/ceval.c:2253
#20 0x000055555564e525 in PyEval_EvalCodeEx () at ../Python/ceval.c:3584
#21 0x000055555566ac2e in function_call.lto_priv () at ../Objects/funcobject.c:523
#22 0x000055555563c5f3 in PyObject_Call () at ../Objects/abstract.c:2547
#23 0x000055555568102e in instancemethod_call.lto_priv () at ../Objects/classobject.c:2602
#24 0x000055555563c5f3 in PyObject_Call () at ../Objects/abstract.c:2547
#25 0x000055555565a4dd in PyEval_CallObjectWithKeywords () at ../Python/ceval.c:4221
#26 0x00005555556f3e31 in slot_tp_hash.lto_priv () at ../Objects/typeobject.c:5506
#27 0x000055555568c908 in dict_subscript.lto_priv () at ../Objects/dictobject.c:1202
#28 0x000055555565618e in PyEval_EvalFrameEx () at ../Python/ceval.c:1539
#29 0x000055555564e525 in PyEval_EvalCodeEx () at ../Python/ceval.c:3584
#30 0x000055555566ac2e in function_call.lto_priv () at ../Objects/funcobject.c:523
#31 0x000055555563c5f3 in PyObject_Call () at ../Objects/abstract.c:2547
#32 0x000055555568102e in instancemethod_call.lto_priv () at ../Objects/classobject.c:2602
#33 0x000055555563c5f3 in PyObject_Call () at ../Objects/abstract.c:2547
#34 0x000055555565a490 in PyEval_CallObjectWithKeywords () at ../Python/ceval.c:4221
#35 0x000055555569ff3a in instance_subscript.lto_priv () at ../Objects/classobject.c:1105
#36 0x000055555565618e in PyEval_EvalFrameEx () at ../Python/ceval.c:1539
#37 0x000055555564e525 in PyEval_EvalCodeEx () at ../Python/ceval.c:3584
#38 0x000055555566ac2e in function_call.lto_priv () at ../Objects/funcobject.c:523
#39 0x000055555563c5f3 in PyObject_Call () at ../Objects/abstract.c:2547
#40 0x0000555555672dd9 in slot_tp_new.lto_priv () at ../Objects/typeobject.c:5849
#41 0x0000555555642093 in type_call.lto_priv () at ../Objects/typeobject.c:749
#42 0x000055555563c5f3 in PyObject_Call () at ../Objects/abstract.c:2547
#43 0x0000555555655e8f in do_call (nk=<optimized out>, na=1, pp_stack=0x7fffffffb950, func=<type at remote 0x555555d6d990>) at ../Python/ceval.c:4569
#44 call_function (oparg=<optimized out>, pp_stack=0x7fffffffb950) at ../Python/ceval.c:4374
#45 PyEval_EvalFrameEx () at ../Python/ceval.c:2989
#46 0x0000555555655c7f in fast_function (nk=<optimized out>, na=<optimized out>, n=<optimized out>, pp_stack=0x7fffffffbaa0, func=<function at remote 0x7fffc0098758>) at ../Python/ceval.c:4437
#47 call_function (oparg=<optimized out>, pp_stack=0x7fffffffbaa0) at ../Python/ceval.c:4372
#48 PyEval_EvalFrameEx () at ../Python/ceval.c:2989
#49 0x0000555555655c7f in fast_function (nk=<optimized out>, na=<optimized out>, n=<optimized out>, pp_stack=0x7fffffffbbf0, func=<function at remote 0x7fffc0098a28>) at ../Python/ceval.c:4437
#50 call_function (oparg=<optimized out>, pp_stack=0x7fffffffbbf0) at ../Python/ceval.c:4372
#51 PyEval_EvalFrameEx () at ../Python/ceval.c:2989
#52 0x0000555555655c7f in fast_function (nk=<optimized out>, na=<optimized out>, n=<optimized out>, pp_stack=0x7fffffffbd40, func=<function at remote 0x7fffc0106cf8>) at ../Python/ceval.c:4437
---Type <return> to continue, or q <return> to quit---
#53 call_function (oparg=<optimized out>, pp_stack=0x7fffffffbd40) at ../Python/ceval.c:4372
#54 PyEval_EvalFrameEx () at ../Python/ceval.c:2989
#55 0x0000555555655c7f in fast_function (nk=<optimized out>, na=<optimized out>, n=<optimized out>, pp_stack=0x7fffffffbe90, func=<function at remote 0x7fffc0094758>) at ../Python/ceval.c:4437
#56 call_function (oparg=<optimized out>, pp_stack=0x7fffffffbe90) at ../Python/ceval.c:4372
#57 PyEval_EvalFrameEx () at ../Python/ceval.c:2989
#58 0x0000555555655c7f in fast_function (nk=<optimized out>, na=<optimized out>, n=<optimized out>, pp_stack=0x7fffffffbfe0, func=<function at remote 0x7fffc002e230>) at ../Python/ceval.c:4437
#59 call_function (oparg=<optimized out>, pp_stack=0x7fffffffbfe0) at ../Python/ceval.c:4372
#60 PyEval_EvalFrameEx () at ../Python/ceval.c:2989
#61 0x0000555555655c7f in fast_function (nk=<optimized out>, na=<optimized out>, n=<optimized out>, pp_stack=0x7fffffffc130, func=<function at remote 0x7ffff596e9b0>) at ../Python/ceval.c:4437
#62 call_function (oparg=<optimized out>, pp_stack=0x7fffffffc130) at ../Python/ceval.c:4372
#63 PyEval_EvalFrameEx () at ../Python/ceval.c:2989
#64 0x0000555555655c7f in fast_function (nk=<optimized out>, na=<optimized out>, n=<optimized out>, pp_stack=0x7fffffffc280, func=<function at remote 0x7fffbfce3578>) at ../Python/ceval.c:4437
#65 call_function (oparg=<optimized out>, pp_stack=0x7fffffffc280) at ../Python/ceval.c:4372
#66 PyEval_EvalFrameEx () at ../Python/ceval.c:2989
#67 0x000055555564e525 in PyEval_EvalCodeEx () at ../Python/ceval.c:3584
#68 0x000055555565676e in fast_function (nk=0, na=<optimized out>, n=<optimized out>, pp_stack=0x7fffffffc490, func=<function at remote 0x7fffbfce39b0>) at ../Python/ceval.c:4447
#69 call_function (oparg=<optimized out>, pp_stack=0x7fffffffc490) at ../Python/ceval.c:4372
#70 PyEval_EvalFrameEx () at ../Python/ceval.c:2989
#71 0x0000555555655c7f in fast_function (nk=<optimized out>, na=<optimized out>, n=<optimized out>, pp_stack=0x7fffffffc5e0, func=<function at remote 0x7fffc002eaa0>) at ../Python/ceval.c:4437
#72 call_function (oparg=<optimized out>, pp_stack=0x7fffffffc5e0) at ../Python/ceval.c:4372
#73 PyEval_EvalFrameEx () at ../Python/ceval.c:2989
#74 0x0000555555655c7f in fast_function (nk=<optimized out>, na=<optimized out>, n=<optimized out>, pp_stack=0x7fffffffc730, func=<function at remote 0x7fffbffe9f50>) at ../Python/ceval.c:4437
#75 call_function (oparg=<optimized out>, pp_stack=0x7fffffffc730) at ../Python/ceval.c:4372
#76 PyEval_EvalFrameEx () at ../Python/ceval.c:2989
#77 0x0000555555655c7f in fast_function (nk=<optimized out>, na=<optimized out>, n=<optimized out>, pp_stack=0x7fffffffc880, func=<function at remote 0x7fffbfb54410>) at ../Python/ceval.c:4437
#78 call_function (oparg=<optimized out>, pp_stack=0x7fffffffc880) at ../Python/ceval.c:4372
#79 PyEval_EvalFrameEx () at ../Python/ceval.c:2989
#80 0x0000555555655c7f in fast_function (nk=<optimized out>, na=<optimized out>, n=<optimized out>, pp_stack=0x7fffffffc9d0, func=<function at remote 0x7fffbfce3e60>) at ../Python/ceval.c:4437
#81 call_function (oparg=<optimized out>, pp_stack=0x7fffffffc9d0) at ../Python/ceval.c:4372
#82 PyEval_EvalFrameEx () at ../Python/ceval.c:2989
#83 0x0000555555655c7f in fast_function (nk=<optimized out>, na=<optimized out>, n=<optimized out>, pp_stack=0x7fffffffcb20, func=<function at remote 0x7fffbfb668c0>) at ../Python/ceval.c:4437
#84 call_function (oparg=<optimized out>, pp_stack=0x7fffffffcb20) at ../Python/ceval.c:4372
#85 PyEval_EvalFrameEx () at ../Python/ceval.c:2989
#86 0x0000555555655c7f in fast_function (nk=<optimized out>, na=<optimized out>, n=<optimized out>, pp_stack=0x7fffffffcc70, func=<function at remote 0x7fffbfb66b90>) at ../Python/ceval.c:4437
#87 call_function (oparg=<optimized out>, pp_stack=0x7fffffffcc70) at ../Python/ceval.c:4372
#88 PyEval_EvalFrameEx () at ../Python/ceval.c:2989
#89 0x0000555555655c7f in fast_function (nk=<optimized out>, na=<optimized out>, n=<optimized out>, pp_stack=0x7fffffffcdc0, func=<function at remote 0x7fffbfb04b90>) at ../Python/ceval.c:4437
#90 call_function (oparg=<optimized out>, pp_stack=0x7fffffffcdc0) at ../Python/ceval.c:4372
#91 PyEval_EvalFrameEx () at ../Python/ceval.c:2989
#92 0x000055555564e525 in PyEval_EvalCodeEx () at ../Python/ceval.c:3584
#93 0x0000555555655f78 in fast_function (nk=1, na=<optimized out>, n=<optimized out>, pp_stack=0x7fffffffcfd0, func=<function at remote 0x7fffbfb041b8>) at ../Python/ceval.c:4447
#94 call_function (oparg=<optimized out>, pp_stack=0x7fffffffcfd0) at ../Python/ceval.c:4372
#95 PyEval_EvalFrameEx () at ../Python/ceval.c:2989
#96 0x0000555555655c7f in fast_function (nk=<optimized out>, na=<optimized out>, n=<optimized out>, pp_stack=0x7fffffffd120, func=<function at remote 0x7ffff4955d70>) at ../Python/ceval.c:4437
#97 call_function (oparg=<optimized out>, pp_stack=0x7fffffffd120) at ../Python/ceval.c:4372
#98 PyEval_EvalFrameEx () at ../Python/ceval.c:2989
#99 0x000055555564e525 in PyEval_EvalCodeEx () at ../Python/ceval.c:3584
#100 0x000055555566ade8 in function_call.lto_priv () at ../Objects/funcobject.c:523
#101 0x000055555563c5f3 in PyObject_Call () at ../Objects/abstract.c:2547
#102 0x00005555556531a5 in ext_do_call (nk=0, na=3, flags=<optimized out>, pp_stack=0x7fffffffd3d8, func=<function at remote 0x7ffff4955b90>) at ../Python/ceval.c:4666
#103 PyEval_EvalFrameEx () at ../Python/ceval.c:3028
#104 0x000055555564e525 in PyEval_EvalCodeEx () at ../Python/ceval.c:3584
#105 0x000055555565676e in fast_function (nk=1, na=<optimized out>, n=<optimized out>, pp_stack=0x7fffffffd5e0, func=<function at remote 0x7ffff4946488>) at ../Python/ceval.c:4447
---Type <return> to continue, or q <return> to quit---
#106 call_function (oparg=<optimized out>, pp_stack=0x7fffffffd5e0) at ../Python/ceval.c:4372
#107 PyEval_EvalFrameEx () at ../Python/ceval.c:2989
#108 0x0000555555655c7f in fast_function (nk=<optimized out>, na=<optimized out>, n=<optimized out>, pp_stack=0x7fffffffd730, func=<function at remote 0x7ffff4950230>) at ../Python/ceval.c:4437
#109 call_function (oparg=<optimized out>, pp_stack=0x7fffffffd730) at ../Python/ceval.c:4372
#110 PyEval_EvalFrameEx () at ../Python/ceval.c:2989
#111 0x000055555564e525 in PyEval_EvalCodeEx () at ../Python/ceval.c:3584
#112 0x000055555565676e in fast_function (nk=0, na=<optimized out>, n=<optimized out>, pp_stack=0x7fffffffd940, func=<function at remote 0x7ffff5ca47d0>) at ../Python/ceval.c:4447
#113 call_function (oparg=<optimized out>, pp_stack=0x7fffffffd940) at ../Python/ceval.c:4372
#114 PyEval_EvalFrameEx () at ../Python/ceval.c:2989
#115 0x000055555564e525 in PyEval_EvalCodeEx () at ../Python/ceval.c:3584
#116 0x000055555564e2c9 in PyEval_EvalCode (co=<optimized out>, globals=<optimized out>, locals=<optimized out>) at ../Python/ceval.c:669
#117 0x000055555567de7f in run_mod.lto_priv () at ../Python/pythonrun.c:1376
#118 0x0000555555678e52 in PyRun_FileExFlags () at ../Python/pythonrun.c:1362
#119 0x000055555567899e in PyRun_SimpleFileExFlags () at ../Python/pythonrun.c:948
#120 0x0000555555629b81 in Py_Main () at ../Modules/main.c:640
#121 0x00007ffff6f1a2b1 in __libc_start_main (main=0x5555556294b0 <main>, argc=4, argv=0x7fffffffdd88, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffffffdd78)
    at ../csu/libc-start.c:291
#122 0x00005555556293aa in _start ()

[1] https://bitbucket.org/pypy/pypy/commits/c501282f4438fe2675e6bfd9cc38374fa55d3da0

----------
components: Interpreter Core
messages: 285228
nosy: gumblex
priority: normal
severity: normal
status: open
title: Crash on GC when compiling PyPy
type: crash
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29242>
_______________________________________

From report at bugs.python.org  Wed Jan 11 10:29:35 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Wed, 11 Jan 2017 15:29:35 +0000
Subject: [issue29099] sqlite3 timestamp converter cannot handle timezone
In-Reply-To: <1482966269.67.0.620298847284.issue29099@psf.upfronthosting.co.za>
Message-ID: <1484148575.95.0.320261089186.issue29099@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

Agreed. And maybe raise more informative error for datetimes with a timezone.

I'm also not sure about 3.5 and 3.6. Datetimes with timezones were never supported. Currently all returned ditetime objects are naive. Getting an aware datetime can confuse applications.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29099>
_______________________________________

From report at bugs.python.org  Wed Jan 11 10:50:16 2017
From: report at bugs.python.org (Marc-Andre Lemburg)
Date: Wed, 11 Jan 2017 15:50:16 +0000
Subject: [issue29099] sqlite3 timestamp converter cannot handle timezone
In-Reply-To: <1482966269.67.0.620298847284.issue29099@psf.upfronthosting.co.za>
Message-ID: <1484149816.97.0.556850335715.issue29099@psf.upfronthosting.co.za>


Marc-Andre Lemburg added the comment:

I don't think this can be considered a bug fix, since it changes behavior for applications that read data from SQLite databases which were not created by Python.

Those application may now see datetime values with tz infos and will likely not be prepared to handle all the problems associated with this.

Is it possible to make the support optional for 3.5 and 3.6 and only enable it for 3.7 (with the possibility of disabling it again) ?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29099>
_______________________________________

From report at bugs.python.org  Wed Jan 11 11:04:29 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Wed, 11 Jan 2017 16:04:29 +0000
Subject: [issue29099] sqlite3 timestamp converter cannot handle timezone
In-Reply-To: <1482966269.67.0.620298847284.issue29099@psf.upfronthosting.co.za>
Message-ID: <1484150669.73.0.569277648635.issue29099@psf.upfronthosting.co.za>


Xiang Zhang added the comment:

I asked whether this should be treated as an enhancement or bug fix. Now with your concerns I think it fits as enhancement more.

> Is it possible to make the support optional for 3.5 and 3.6 and only enable it for 3.7 (with the possibility of disabling it again) ?

How about just make it just into 3.7 and just document only naive datetime objects are supported in 3.5, 3.6 and 2.7.
I am not sure it's worth to make it even optional in 3.7. Discarding tzinfo sounds weird, and worse, the behaviour is not deterministic, it could also fail with an exception.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29099>
_______________________________________

From report at bugs.python.org  Wed Jan 11 11:08:15 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Wed, 11 Jan 2017 16:08:15 +0000
Subject: [issue29099] sqlite3 timestamp converter cannot handle timezone
In-Reply-To: <1482966269.67.0.620298847284.issue29099@psf.upfronthosting.co.za>
Message-ID: <1484150895.12.0.194608855254.issue29099@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

The naive datetime converter is registered under the name "timestamp". The aware datetime converter or the universal datetime converter (if it is needed) can be registered under different names.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29099>
_______________________________________

From report at bugs.python.org  Wed Jan 11 11:15:14 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Wed, 11 Jan 2017 16:15:14 +0000
Subject: [issue29243] --enable-optimizations makes common build commands
 always need to compile from scratch
Message-ID: <1484151314.82.0.308427338149.issue29243@psf.upfronthosting.co.za>


New submission from Xiang Zhang:

In 3.6 we get --enable-optimizations. One thing I find annoyed with this flag is that it makes some build commands compile from scratch since it always clears the environment first. For example, with the commands listed at the top of Makefile:

./configure
make 
make test
make install

It compiles 3 times and considering the optimized build needs more time every time, it lasts long.

I am not sure this is a problem and feel free to close it.

----------
components: Build
messages: 285233
nosy: xiang.zhang
priority: normal
severity: normal
status: open
title: --enable-optimizations makes common build commands always need to compile from scratch
versions: Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29243>
_______________________________________

From report at bugs.python.org  Wed Jan 11 11:25:06 2017
From: report at bugs.python.org (Ethan Furman)
Date: Wed, 11 Jan 2017 16:25:06 +0000
Subject: [issue28231] zipfile does not support pathlib
In-Reply-To: <1474445503.11.0.219964117103.issue28231@psf.upfronthosting.co.za>
Message-ID: <1484151906.31.0.692633084556.issue28231@psf.upfronthosting.co.za>


Ethan Furman added the comment:

Any path/file attributes, etc, inside a ZipFile should be str.  ZipFile should also function properly if path/file requests are given as os.PathLike objects.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28231>
_______________________________________

From report at bugs.python.org  Wed Jan 11 11:25:18 2017
From: report at bugs.python.org (INADA Naoki)
Date: Wed, 11 Jan 2017 16:25:18 +0000
Subject: [issue29240] Implementation of the PEP 540: Add a new UTF-8 mode
In-Reply-To: <1484133592.17.0.263024020483.issue29240@psf.upfronthosting.co.za>
Message-ID: <1484151918.44.0.0954801757065.issue29240@psf.upfronthosting.co.za>


Changes by INADA Naoki <songofacandy at gmail.com>:


----------
nosy: +inada.naoki

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29240>
_______________________________________

From report at bugs.python.org  Wed Jan 11 11:34:51 2017
From: report at bugs.python.org (George King)
Date: Wed, 11 Jan 2017 16:34:51 +0000
Subject: [issue29057] Compiler failure on Mac OS X - sys/random.h
In-Reply-To: <1482514649.72.0.460227366509.issue29057@psf.upfronthosting.co.za>
Message-ID: <1484152491.91.0.814484071601.issue29057@psf.upfronthosting.co.za>


George King added the comment:

Still seeing this problem. Here was my exact process:

$ git clone git at github.com:python/cpython.git
$ cd cpython
$ git checkout 2.7
$ mkdir build
$ cd build
$ ../configure
$ make

In file included from ../Python/random.c:7:
/usr/include/sys/random.h:37:32: error: unknown type name 'u_int'

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29057>
_______________________________________

From report at bugs.python.org  Wed Jan 11 11:36:52 2017
From: report at bugs.python.org (George King)
Date: Wed, 11 Jan 2017 16:36:52 +0000
Subject: [issue29057] Compiler failure on Mac OS X - sys/random.h
In-Reply-To: <1482514649.72.0.460227366509.issue29057@psf.upfronthosting.co.za>
Message-ID: <1484152612.66.0.512189727443.issue29057@psf.upfronthosting.co.za>


George King added the comment:

This is using the latest apple toolchain on latest macOS 10.12.2:

$ gcc --version
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 8.0.0 (clang-800.0.42.1)
Target: x86_64-apple-darwin16.3.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

$ make --version
GNU Make 3.81
Copyright (C) 2006  Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.

This program built for i386-apple-darwin11.3.0

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29057>
_______________________________________

From report at bugs.python.org  Wed Jan 11 11:40:37 2017
From: report at bugs.python.org (Steve Dower)
Date: Wed, 11 Jan 2017 16:40:37 +0000
Subject: [issue27603] Migrate IDLE context menu items to shell extension
In-Reply-To: <1469332799.74.0.70339682021.issue27603@psf.upfronthosting.co.za>
Message-ID: <1484152837.14.0.356602158389.issue27603@psf.upfronthosting.co.za>


Steve Dower added the comment:

More user choice here is actually an incredibly huge burden. Managing file associations across a range of versions, many of which can no longer be updated, is hard enough without offering options. Not to mention the extra user interface required.

Personally, I'd rather the options be third party installers or tools. There's nothing stopping someone making an idle "installer" that sets up context menus and default actions however it likes. It would probably break uninstall/upgrade scenarios for the core Python install, but at least it's not a burden on our (very few) volunteers to diagnose, fix and prevent.

If or when the shell extension takes over the context menu (currently blocked on someone volunteering to do the implementation and testing), it may be possible to have registry keys to change the behavior. But I wouldn't count on it being highly configurable - that's not the aim of the main distro.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue27603>
_______________________________________

From report at bugs.python.org  Wed Jan 11 11:43:22 2017
From: report at bugs.python.org (Steve Dower)
Date: Wed, 11 Jan 2017 16:43:22 +0000
Subject: [issue29241] sys._enablelegacywindowsfsencoding() don't apply to
 os.fsencode and os.fsdecode
In-Reply-To: <1484137681.59.0.0301192140292.issue29241@psf.upfronthosting.co.za>
Message-ID: <1484153002.84.0.108360584588.issue29241@psf.upfronthosting.co.za>


Steve Dower added the comment:

If you import os first then that's acceptable and we should document it more clearly. Try calling enable before importing os.

I wouldn't be surprised if os is imported automatically, in which case we need to figure out some alternate caching mechanism that can be reset by the enable call (or demonstrate that the caching is of no benefit).

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29241>
_______________________________________

From report at bugs.python.org  Wed Jan 11 11:44:37 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Wed, 11 Jan 2017 16:44:37 +0000
Subject: [issue20804] Sentinels identity lost when pickled (unittest.mock)
In-Reply-To: <1393579357.22.0.503856996983.issue20804@psf.upfronthosting.co.za>
Message-ID: <1484153077.43.0.227641561165.issue20804@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

sentinel-doc.patch documents the contrary behaviour -- not preserving an identity and equality.

I haven't included doc changes because I don't know how to treat this change. As a bug fix, as a new feature, or as a new feature backported to maintained releases? If this is a new feature, it needs an entry in Whant's New and a versionchanged directive in the module documentation. If this is a bug fix, it likely doesn't need any special documenting.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue20804>
_______________________________________

From report at bugs.python.org  Wed Jan 11 11:45:43 2017
From: report at bugs.python.org (Steve Dower)
Date: Wed, 11 Jan 2017 16:45:43 +0000
Subject: [issue29231] Broken MSIs in Python 3.5+
In-Reply-To: <1484082847.93.0.115252889304.issue29231@psf.upfronthosting.co.za>
Message-ID: <1484153143.61.0.732017846575.issue29231@psf.upfronthosting.co.za>


Steve Dower added the comment:

I discussed installers on my blog at http://stevedower.id.au a while back. There are some other options linked from there, and also rationale for why things are how they are.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29231>
_______________________________________

From report at bugs.python.org  Wed Jan 11 11:46:48 2017
From: report at bugs.python.org (Michael Foord)
Date: Wed, 11 Jan 2017 16:46:48 +0000
Subject: [issue20804] Sentinels identity lost when pickled (unittest.mock)
In-Reply-To: <1393579357.22.0.503856996983.issue20804@psf.upfronthosting.co.za>
Message-ID: <1484153208.7.0.758554078845.issue20804@psf.upfronthosting.co.za>


Michael Foord added the comment:

It's a new feature.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue20804>
_______________________________________

From report at bugs.python.org  Wed Jan 11 11:53:09 2017
From: report at bugs.python.org (Shane Smith)
Date: Wed, 11 Jan 2017 16:53:09 +0000
Subject: [issue27603] Migrate IDLE context menu items to shell extension
In-Reply-To: <1469332799.74.0.70339682021.issue27603@psf.upfronthosting.co.za>
Message-ID: <1484153589.62.0.597282519379.issue27603@psf.upfronthosting.co.za>


Shane Smith added the comment:

I'm fine with a single implementation, so long as the implementation is what's best for the majority of users.  Not my intent to increase the burden of work.

So, let me ask the question alluded to in my first post: when is a nested menu actually desirable?  I would argue that the vast majority of users have one or two versions of Python installed at any given time.  And for that number, a flat menu is certainly preferred.

The X.Y version numbering in the context menu introduced as a result of issue23546 solves said issue.  I'm merely suggesting de-nesting the menu.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue27603>
_______________________________________

From report at bugs.python.org  Wed Jan 11 12:01:32 2017
From: report at bugs.python.org (Marc-Andre Lemburg)
Date: Wed, 11 Jan 2017 17:01:32 +0000
Subject: [issue29099] sqlite3 timestamp converter cannot handle timezone
In-Reply-To: <1484150895.12.0.194608855254.issue29099@psf.upfronthosting.co.za>
Message-ID: <1c8498e7-b852-740a-5195-a5c170054cf7@egenix.com>


Marc-Andre Lemburg added the comment:

On 11.01.2017 17:08, Serhiy Storchaka wrote:
> 
> The naive datetime converter is registered under the name "timestamp". The aware datetime converter or the universal datetime converter (if it is needed) can be registered under different names.

This sounds like a good idea.

Perhaps use "timestamptz" or something similar.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29099>
_______________________________________

From report at bugs.python.org  Wed Jan 11 12:03:52 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Wed, 11 Jan 2017 17:03:52 +0000
Subject: [issue20804] Sentinels identity lost when pickled (unittest.mock)
In-Reply-To: <1393579357.22.0.503856996983.issue20804@psf.upfronthosting.co.za>
Message-ID: <1484154232.48.0.233856083651.issue20804@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

Updated patch contains doc changes.

----------
type: behavior -> enhancement
versions:  -Python 3.5, Python 3.6
Added file: http://bugs.python.org/file46260/mock_sentinel_pickle2.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue20804>
_______________________________________

From report at bugs.python.org  Wed Jan 11 12:06:28 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Wed, 11 Jan 2017 17:06:28 +0000
Subject: [issue29099] sqlite3 timestamp converter cannot handle timezone
In-Reply-To: <1482966269.67.0.620298847284.issue29099@psf.upfronthosting.co.za>
Message-ID: <1484154388.15.0.78677448473.issue29099@psf.upfronthosting.co.za>


Changes by Serhiy Storchaka <storchaka+cpython at gmail.com>:


----------
stage: commit review -> needs patch
type: behavior -> enhancement
versions:  -Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29099>
_______________________________________

From mal at egenix.com  Wed Jan 11 12:16:18 2017
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed, 11 Jan 2017 18:16:18 +0100
Subject: [issue29099] sqlite3 timestamp converter cannot handle timezone
In-Reply-To: <1484150669.73.0.569277648635.issue29099@psf.upfronthosting.co.za>
References: <1484150669.73.0.569277648635.issue29099@psf.upfronthosting.co.za>
Message-ID: <480c5900-089f-847e-25a5-3dc52668b78c@egenix.com>

On 11.01.2017 17:04, Xiang Zhang wrote:
> I am not sure it's worth to make it even optional in 3.7. Discarding tzinfo sounds weird, and worse, the behaviour is not deterministic, it could also fail with an exception.

Best practice is to store all date/time values using UTC (or some
other fixed timezone) in databases and to manage the timezone/locale
information elsewhere.

The TZ info then becomes redundant and only results in more
database space being used as well as slower queries (due to the extra
TZ calculations being done; many databases internally store the
values as UTC to avoid having to do TZ calculations at query time).

Of course, there are use cases, where you'd still want to work
with TZ values in the database, so an extra converter for this
sounds like a good plan.

-- 
Marc-Andre Lemburg
eGenix.com


From report at bugs.python.org  Wed Jan 11 12:16:25 2017
From: report at bugs.python.org (Marc-Andre Lemburg)
Date: Wed, 11 Jan 2017 17:16:25 +0000
Subject: [issue29099] sqlite3 timestamp converter cannot handle timezone
In-Reply-To: <1484150669.73.0.569277648635.issue29099@psf.upfronthosting.co.za>
Message-ID: <480c5900-089f-847e-25a5-3dc52668b78c@egenix.com>


Marc-Andre Lemburg added the comment:

On 11.01.2017 17:04, Xiang Zhang wrote:
> I am not sure it's worth to make it even optional in 3.7. Discarding tzinfo sounds weird, and worse, the behaviour is not deterministic, it could also fail with an exception.

Best practice is to store all date/time values using UTC (or some
other fixed timezone) in databases and to manage the timezone/locale
information elsewhere.

The TZ info then becomes redundant and only results in more
database space being used as well as slower queries (due to the extra
TZ calculations being done; many databases internally store the
values as UTC to avoid having to do TZ calculations at query time).

Of course, there are use cases, where you'd still want to work
with TZ values in the database, so an extra converter for this
sounds like a good plan.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29099>
_______________________________________

From report at bugs.python.org  Wed Jan 11 12:20:57 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Wed, 11 Jan 2017 17:20:57 +0000
Subject: [issue28759] access to mkfifo,
 mknod and hard links is controled by SELinux MAC on Android
In-Reply-To: <1479715975.06.0.930254771903.issue28759@psf.upfronthosting.co.za>
Message-ID: <1484155257.32.0.267814022557.issue28759@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

I don't know what is a good name for such helper. supported_operation() looks too general. Maybe ask on Python-Dev?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28759>
_______________________________________

From report at bugs.python.org  Wed Jan 11 12:32:42 2017
From: report at bugs.python.org (KeyWeeUsr)
Date: Wed, 11 Jan 2017 17:32:42 +0000
Subject: [issue29231] Broken MSIs in Python 3.5+
In-Reply-To: <1484082847.93.0.115252889304.issue29231@psf.upfronthosting.co.za>
Message-ID: <1484155962.57.0.60908487413.issue29231@psf.upfronthosting.co.za>


KeyWeeUsr added the comment:

Re blog post "Why are there so many Python installers?"

> As a result, the old installer always requires administrative privileges just in case you choose to install for all users. This prevents installation of Python on machines where you do not have full control over the system.

So not true. Read the first command I used, that doesn't require any permission at all. I was able to install Python without any registry or P&F stuff at "Guest" account in an internet cafe where you couldn't even throw stuff to the Trash with the same MSI installer. Notice the "/a" switch, not "/i" (install).

Secondly, I'm glad for the change, but the installer is still in an imperfect shape - leaves traces. That makes you able to use only a single version (as mentioned before). Unless, of course, you don't want to copy&paste the whole folder, which although works is also stupid and still requires at least one successful installation with traces in the system.

> Always requires administrator privileges <-- no
Only allows installation for all users <-- no
Only allows configuration at the command line (via msiexec) <-- not sure what the "configuration" means in this context
Prevents the executable installer from installing for all users <-- I didn't really use that, but... what? Seems weird. Have you tried with "/a ALLUSERS=1"? It should purge the elevation.

Anyway, this has nothing to do with the issue, because what I'm talking about are MSIs that are _extracted_ from the current EXE installer via "/layout" switch. There comes the problem - you can't install from those MSIs (not directly, but probably works with EXE as a casual python installation - I don't want that), because the MSIs are ***corrupted*** in some weird way, probably some debug command is called together with the "/layout" judging from the "-d" string in the filenames - useful only for debugging the inside of the MSIs, otherwise completely unusable. Please, just try to do what I wrote to see what's actually going on.

If there is any way how to fix the "/layout" to unpack correctly, ++ for the new installer. Otherwise, although the EXE has a lot of features I use for system-wide interpreter, I hate the change.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29231>
_______________________________________

From report at bugs.python.org  Wed Jan 11 12:44:21 2017
From: report at bugs.python.org (Roundup Robot)
Date: Wed, 11 Jan 2017 17:44:21 +0000
Subject: [issue29220] Python 3.6 regression/change using
 logging.addLevelName() to clear a name
In-Reply-To: <1484016439.6.0.79629021761.issue29220@psf.upfronthosting.co.za>
Message-ID: <20170111174418.9646.6946.1544917F@psf.io>


Roundup Robot added the comment:

New changeset adf8312f377f by Vinay Sajip in branch '3.5':
Issue #29220: Improved fix and test.
https://hg.python.org/cpython/rev/adf8312f377f

New changeset a76eed0baa0f by Vinay Sajip in branch 'default':
Issue #29220: Merged fixes from 3.6.
https://hg.python.org/cpython/rev/a76eed0baa0f

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29220>
_______________________________________

From report at bugs.python.org  Wed Jan 11 12:45:29 2017
From: report at bugs.python.org (Vinay Sajip)
Date: Wed, 11 Jan 2017 17:45:29 +0000
Subject: [issue29220] Python 3.6 regression/change using
 logging.addLevelName() to clear a name
In-Reply-To: <1484016439.6.0.79629021761.issue29220@psf.upfronthosting.co.za>
Message-ID: <1484156729.63.0.43271120609.issue29220@psf.upfronthosting.co.za>


Changes by Vinay Sajip <vinay_sajip at yahoo.co.uk>:


----------
resolution:  -> fixed
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29220>
_______________________________________

From report at bugs.python.org  Wed Jan 11 13:05:21 2017
From: report at bugs.python.org (Michael Foord)
Date: Wed, 11 Jan 2017 18:05:21 +0000
Subject: [issue20804] Sentinels identity lost when pickled (unittest.mock)
In-Reply-To: <1393579357.22.0.503856996983.issue20804@psf.upfronthosting.co.za>
Message-ID: <1484157921.14.0.0123587222045.issue20804@psf.upfronthosting.co.za>


Michael Foord added the comment:

LGTM Serhiy - thank you for your work. Appreciated.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue20804>
_______________________________________

From report at bugs.python.org  Wed Jan 11 13:11:47 2017
From: report at bugs.python.org (rdb)
Date: Wed, 11 Jan 2017 18:11:47 +0000
Subject: [issue29244] Python 3.6 unnecessarily requires inttypes.h
Message-ID: <1484158307.25.0.0951457804811.issue29244@psf.upfronthosting.co.za>


New submission from rdb:

Python 3.6 now requires inttypes.h on all platforms.  However, this is not provided by MSVC 2010 and 2012, which is still used by some people who build extension modules for Python.

MSVC 2010 does provide stdint.h, and replacing the inttypes.h include with an include to stdint.h seems to work fine.

I would suggest a fix along the lines of this:

#if defined(_MSC_VER) && _MSC_VER < 1800
#include <stdint.h>
#else
#include <inttypes.h>
#endif

Alternatively, the HAVE_INTTYPES_H definition could be used to fall back to stdint.h, and it could be undefined for the MSVC build.

----------
components: Extension Modules
messages: 285250
nosy: rdb
priority: normal
severity: normal
status: open
title: Python 3.6 unnecessarily requires inttypes.h
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29244>
_______________________________________

From report at bugs.python.org  Wed Jan 11 13:13:57 2017
From: report at bugs.python.org (Roundup Robot)
Date: Wed, 11 Jan 2017 18:13:57 +0000
Subject: [issue20804] Sentinels identity lost when pickled (unittest.mock)
In-Reply-To: <1393579357.22.0.503856996983.issue20804@psf.upfronthosting.co.za>
Message-ID: <20170111181325.103639.40126.EC60C03D@psf.io>


Roundup Robot added the comment:

New changeset 98f061402fcf by Serhiy Storchaka in branch 'default':
Issue #20804: The unittest.mock.sentinel attributes now preserve their
https://hg.python.org/cpython/rev/98f061402fcf

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue20804>
_______________________________________

From report at bugs.python.org  Wed Jan 11 13:18:43 2017
From: report at bugs.python.org (Roundup Robot)
Date: Wed, 11 Jan 2017 18:18:43 +0000
Subject: [issue20804] Sentinels identity lost when pickled (unittest.mock)
In-Reply-To: <1393579357.22.0.503856996983.issue20804@psf.upfronthosting.co.za>
Message-ID: <20170111181827.1510.38487.8264740F@psf.io>


Roundup Robot added the comment:

New changeset 398b73dbb1a0 by Serhiy Storchaka in branch '3.5':
Issue #20804: Document the limitation of the unittest.mock.sentinel attributes.
https://hg.python.org/cpython/rev/398b73dbb1a0

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue20804>
_______________________________________

From report at bugs.python.org  Wed Jan 11 13:23:00 2017
From: report at bugs.python.org (Vinay Sajip)
Date: Wed, 11 Jan 2017 18:23:00 +0000
Subject: [issue29213] python -m venv activate.bat has weird mix of line endings
In-Reply-To: <1483971138.02.0.292455070123.issue29213@psf.upfronthosting.co.za>
Message-ID: <1484158980.43.0.345349532878.issue29213@psf.upfronthosting.co.za>


Vinay Sajip added the comment:

Although we don't currently have binaries under venv/scripts, that might change in the future. The logic could be changed to:

    with open(srcfile, 'rb') as f:
        data = f.read()
    if not srcfile.endswith('.exe'):
        try:
            data = data.decode('utf-8')
            data = self.replace_variables(data, context)
            data = data.encode('utf-8')
        except UnicodeError as e:
            data = None
            logger.warning('unable to copy script %r, '
                           'may be binary: %s', srcfile, e)
    if data is not None:
        with open(dstfile, 'wb') as f:
            f.write(data)
        shutil.copymode(srcfile, dstfile)

All the files in the nt subdirectory have CRLF endings, and the above should preserve them (whatever they are).

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29213>
_______________________________________

From report at bugs.python.org  Wed Jan 11 13:23:07 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Wed, 11 Jan 2017 18:23:07 +0000
Subject: [issue20804] Sentinels identity lost when pickled (unittest.mock)
In-Reply-To: <1393579357.22.0.503856996983.issue20804@psf.upfronthosting.co.za>
Message-ID: <1484158987.78.0.641214011907.issue20804@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

Thank you for your review Michael.

----------
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue20804>
_______________________________________

From report at bugs.python.org  Wed Jan 11 13:26:14 2017
From: report at bugs.python.org (Vinay Sajip)
Date: Wed, 11 Jan 2017 18:26:14 +0000
Subject: [issue22343] Install bash activate script on Windows when using venv
In-Reply-To: <1409948648.7.0.892236136888.issue22343@psf.upfronthosting.co.za>
Message-ID: <1484159174.45.0.511375481275.issue22343@psf.upfronthosting.co.za>


Vinay Sajip added the comment:

If we do this, the activate script will have LF (POSIX) line endings on Windows. Will this work for things like cygwin, Git-bash etc?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue22343>
_______________________________________

From report at bugs.python.org  Wed Jan 11 13:26:54 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Wed, 11 Jan 2017 18:26:54 +0000
Subject: [issue29244] Python 3.6 unnecessarily requires inttypes.h
In-Reply-To: <1484158307.25.0.0951457804811.issue29244@psf.upfronthosting.co.za>
Message-ID: <1484159214.6.0.367694332608.issue29244@psf.upfronthosting.co.za>


Changes by Serhiy Storchaka <storchaka+cpython at gmail.com>:


----------
components: +Build -Extension Modules
nosy: +benjamin.peterson
type:  -> compile error
versions: +Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29244>
_______________________________________

From report at bugs.python.org  Wed Jan 11 13:26:57 2017
From: report at bugs.python.org (Steve Dower)
Date: Wed, 11 Jan 2017 18:26:57 +0000
Subject: [issue29231] Broken MSIs in Python 3.5+
In-Reply-To: <1484082847.93.0.115252889304.issue29231@psf.upfronthosting.co.za>
Message-ID: <1484159217.05.0.833202883417.issue29231@psf.upfronthosting.co.za>


Steve Dower added the comment:

"msiexec /a" does not install anything - it converts the MSI into a source layout for lazy network installs. It is, and always has been, completely unsupported as an approach to installing Python, and we are under no obligation to maintain it.

Copy-pasting the install folder after installation is also unsupported. We obviously can't stop you doing it, and at times it is the best way to achieve certain things, but it is not supported and we do not have to make it work better.

The /layout command is intended for full offline installs: it downloads all optional packages without evaluating any conditions and puts them in a location where they will be used later without redownloading.

You've always been able to use only a single micro version of Python at a time when installing it. That has not changed (and will not, as it's by design). We have added new package formats that do not install the full development kit, and it sounds like the nuget packages (a.k.a. zip files by a different name) are perfect for you.

I have no idea what the "-d" suffix is, unless you mean the "_d.msi" modules which contain different files from the non-"_d" versions. The internals of the installer are just that - internals - but FYI there are typically 3 installers for each component. For example, "exe.msi" contains the files related to launching an interactive shell (including licenses, shortcuts, and icons); "exe_pdb.msi" contains the PDB files associated with files in exe.msi; "exe_d.msi" contains a debug build for those who are debugging native extensions and want consistent build settings between Python and their extension.

Your criticism of the *hypothetical* single MSI installer doesn't make any sense, since that installer does not exist. I included those points to show what restrictions would be necessary to avoid any such single MSI installer breaking users who would prefer to use the EXE installer.

There is no bug here, so I'm closing the issue.

----------
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29231>
_______________________________________

From report at bugs.python.org  Wed Jan 11 13:37:55 2017
From: report at bugs.python.org (Steve Dower)
Date: Wed, 11 Jan 2017 18:37:55 +0000
Subject: [issue27603] Migrate IDLE context menu items to shell extension
In-Reply-To: <1469332799.74.0.70339682021.issue27603@psf.upfronthosting.co.za>
Message-ID: <1484159874.99.0.731221800445.issue27603@psf.upfronthosting.co.za>


Steve Dower added the comment:

The last time I saw collected data on usage (based on Visual Studio users), there was no clear majority of "number of interpreter versions", but 3 was the most common (2 and 4 were roughly equal, 5 and 6 were more popular than 1).

The nesting also makes it considerably simpler for the installer to handle, since there are totally independent components contributing to the file association. This is very difficult to achieve in a reliable way (wrt upgrades/downgrades/uninstalls/repairs/order-of-install/etc.), and having the context menu here is the best balance of reliability, usability and ease of implementation.

Using the shell extension to generate the menu would also work well, as that is part of the launcher and independent of any particular Python installer. It can also automatically un-nest the menu when it's going to be short enough, or show the latest version at the top and nest all others. Ultimately though, implementing that requires somebody's time, and until somebody is motivated enough to do the work, it won't happen.

So your suggestion is valid, it just isn't motivating enough for me to make the change. That doesn't mean someone else can't do it, and from a personal level I will oppose a straight un-nesting (as I'm one of the outliers who has 20+ copies of Python installed on my machines :) ), but I'm totally in favour of a well implemented shell extension to make it more complete and easier to use.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue27603>
_______________________________________

From report at bugs.python.org  Wed Jan 11 13:40:40 2017
From: report at bugs.python.org (Zachary Ware)
Date: Wed, 11 Jan 2017 18:40:40 +0000
Subject: [issue29244] Python 3.6 unnecessarily requires inttypes.h
In-Reply-To: <1484158307.25.0.0951457804811.issue29244@psf.upfronthosting.co.za>
Message-ID: <1484160040.63.0.958278694343.issue29244@psf.upfronthosting.co.za>


Zachary Ware added the comment:

VS 2010 is not a supported compiler version for 3.6 (or 3.5) and VS 2012 is not a supported compiler for any version of Python. Extension modules built by those versions for use with Python 3.6 as supplied by python.org will only work in very particular circumstances which cannot be relied upon. Furthermore, VS 2015 community edition is free (as in beer) and perfectly suitable for building both Python and extension modules.

I'd recommend closing as won't fix, but if another committer feels strongly enough about it I'll defer to them.

----------
nosy: +zach.ware

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29244>
_______________________________________

From report at bugs.python.org  Wed Jan 11 13:42:00 2017
From: report at bugs.python.org (Brett Cannon)
Date: Wed, 11 Jan 2017 18:42:00 +0000
Subject: [issue29243] --enable-optimizations makes common build commands
 always need to compile from scratch
In-Reply-To: <1484151314.82.0.308427338149.issue29243@psf.upfronthosting.co.za>
Message-ID: <1484160120.89.0.85777139967.issue29243@psf.upfronthosting.co.za>


Brett Cannon added the comment:

Since optimizations uses PGO which should do a fresh run, that's why there's the clearing of the state.

I'm personally not bothered by it as you really should only being doing an optimized build once when you do the first install and then you're done. But if or anyone else can come up with a way to not clear out the results while not accidentally using stale PGO-collected stats or builds then I don't see why we can't speed it up.

----------
nosy: +brett.cannon
type:  -> enhancement

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29243>
_______________________________________

From report at bugs.python.org  Wed Jan 11 13:44:00 2017
From: report at bugs.python.org (Brett Cannon)
Date: Wed, 11 Jan 2017 18:44:00 +0000
Subject: [issue29244] Python 3.6 unnecessarily requires inttypes.h
In-Reply-To: <1484158307.25.0.0951457804811.issue29244@psf.upfronthosting.co.za>
Message-ID: <1484160240.5.0.13794014937.issue29244@psf.upfronthosting.co.za>


Brett Cannon added the comment:

Everything Zach said is accurate, so closing as "wont fix".

----------
nosy: +brett.cannon
resolution:  -> wont fix
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29244>
_______________________________________

From report at bugs.python.org  Wed Jan 11 13:46:52 2017
From: report at bugs.python.org (rdb)
Date: Wed, 11 Jan 2017 18:46:52 +0000
Subject: [issue29244] Python 3.6 unnecessarily requires inttypes.h
In-Reply-To: <1484158307.25.0.0951457804811.issue29244@psf.upfronthosting.co.za>
Message-ID: <1484160412.09.0.879941613745.issue29244@psf.upfronthosting.co.za>


rdb added the comment:

As far as I know, there should not be any ABI issues.  We've been building extension modules with non-matching MSVC versions for years without issues.

I find it hard to think of downsides to such a trivial fix.  It is somewhat frustrating that we will have to resort to shipping custom Python versions with patched headers.  Some people are constrained to older MSVC versions for reasons not in their control.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29244>
_______________________________________

From report at bugs.python.org  Wed Jan 11 13:59:13 2017
From: report at bugs.python.org (KeyWeeUsr)
Date: Wed, 11 Jan 2017 18:59:13 +0000
Subject: [issue29231] Broken MSIs in Python 3.5+
In-Reply-To: <1484082847.93.0.115252889304.issue29231@psf.upfronthosting.co.za>
Message-ID: <1484161153.3.0.58584916477.issue29231@psf.upfronthosting.co.za>


KeyWeeUsr added the comment:

python-3.5.2.exe /layout <folder>

11.01.2017  19:30    <DIR>          .
11.01.2017  19:30    <DIR>          ..
11.01.2017  19:30         3?035?136 core_d.msi
11.01.2017  19:30         2?240?512 core_pdb.msi
11.01.2017  19:30            98?304 dev_d.msi
11.01.2017  19:30           110?592 exe_d.msi
11.01.2017  19:30           102?400 exe_pdb.msi
11.01.2017  19:30         5?378?048 lib_d.msi
11.01.2017  19:30         4?820?992 lib_pdb.msi
29.10.2016  11:13        29?269?656 python-3.5.2.exe
11.01.2017  19:30         1?179?648 tcltk_d.msi
11.01.2017  19:30           139?264 tcltk_pdb.msi
11.01.2017  19:30           331?776 test_d.msi
11.01.2017  19:30           286?720 test_pdb.msi
              12 File(s)     46?993?048 bytes
               2 Dir(s)   1?245?736?960 bytes free
---
msiexec /a exe_d.msi targetdir=%cd%\<folder>

Msiexec window has title "Python 3.5.2 Executables (32-bit debug)" and the targetdir contains:

11.01.2017  19:36    <DIR>          .
11.01.2017  19:36    <DIR>          ..
25.06.2016  21:57            51?712 pythonw_d.exe
25.06.2016  21:57           364?544 pythonw_d.pdb
25.06.2016  21:57            51?712 python_d.exe
25.06.2016  21:57           364?544 python_d.pdb
               4 File(s)        832?512 bytes
               2 Dir(s)   1?264?963?584 bytes free

Notice, that everywhere is "_d" I talk about so much.

> ...and puts them in a location where they will be used later without redownloading...

Yes, in a state that only the EXE installer is able to use them.

> *hypothetical*

msiexec /a python-2.7.13.msi targetdir=%cd%\<folder>

11.01.2017  19:45    <DIR>          .
11.01.2017  19:45    <DIR>          ..
11.01.2017  19:45    <DIR>          DLLs
11.01.2017  19:45    <DIR>          Doc
11.01.2017  19:45    <DIR>          include
11.01.2017  19:45    <DIR>          Lib
11.01.2017  19:45    <DIR>          libs
17.12.2016  20:49            38?591 LICENSE.txt
17.12.2016  20:34           474?595 NEWS.txt
11.01.2017  19:45           835?584 python-2.7.13.msi
17.12.2016  20:44            27?136 python.exe
17.12.2016  20:43         2?639?872 python27.dll
17.12.2016  20:44            27?648 pythonw.exe
03.12.2016  21:01            56?938 README.txt
11.01.2017  19:45    <DIR>          tcl
11.01.2017  19:45    <DIR>          Tools
17.12.2016  20:44           111?104 w9xpopen.exe
               8 File(s)      4?211?468 bytes
               9 Dir(s)   1?162?199?040 bytes free

Fully usable, portable and working Python interpreter for Python installers <3.5.0 without a single beep about admin rights (kivy, numpy, scipy, cython, mingwpy, etc.). With some workaround probably even usable for ALLUSERS=1, but I'm not going to try that on my system.

How is Debug installer not an issue/bug in a Python Release? Am I missing something or is it ok to unpack Debug installers from an official installer and not the actually used ones, the ones that produce Release that works?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29231>
_______________________________________

From report at bugs.python.org  Wed Jan 11 14:26:59 2017
From: report at bugs.python.org (Steve Dower)
Date: Wed, 11 Jan 2017 19:26:59 +0000
Subject: [issue29231] Broken MSIs in Python 3.5+
In-Reply-To: <1484082847.93.0.115252889304.issue29231@psf.upfronthosting.co.za>
Message-ID: <1484162819.72.0.441122088843.issue29231@psf.upfronthosting.co.za>


Steve Dower added the comment:

Ah, you're using the full installer, which has the non-debug versions embedded and therefore does not ever need to download them.

If you start from the web installer, it will download the MSIs for the non-debug components as well. But as I said, you're in totally unsupported territory by not running the proper installer. See if the nuget packages are more helpful.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29231>
_______________________________________

From report at bugs.python.org  Wed Jan 11 14:27:10 2017
From: report at bugs.python.org (Paul Moore)
Date: Wed, 11 Jan 2017 19:27:10 +0000
Subject: [issue29231] Broken MSIs in Python 3.5+
In-Reply-To: <1484161153.3.0.58584916477.issue29231@psf.upfronthosting.co.za>
Message-ID: <CACac1F9sZi-tb6t=uGMSSRztDOdzYFJxoPHTwMn8ccwzwKvexg@mail.gmail.com>


Paul Moore added the comment:

> Fully usable, portable

*and unsupported* - you apparently ignored that point from Steve's comment

> and working Python interpreter

That's (essentially) pure luck. I don't think I've ever seen any
suggestion that the /a flag for *any* MSI installer (Python or
otherwise) will produce a working install - it's designed (and
documented by Microsoft) as being to allow system admins to unpack the
MSI onto a network share from where it can be installed onto user
machines (for details see
https://msdn.microsoft.com/en-us/library/windows/desktop/aa367541(v=vs.85).aspx).

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29231>
_______________________________________

From report at bugs.python.org  Wed Jan 11 14:31:29 2017
From: report at bugs.python.org (Ned Deily)
Date: Wed, 11 Jan 2017 19:31:29 +0000
Subject: [issue29057] Compiler failure on Mac OS X - sys/random.h
In-Reply-To: <1482514649.72.0.460227366509.issue29057@psf.upfronthosting.co.za>
Message-ID: <1484163089.76.0.0651656537505.issue29057@psf.upfronthosting.co.za>


Ned Deily added the comment:

Sorry, George, but I'm unable to reproduce the failure you observe following your recipe.  The only thing I can suggest off the top of my head is that you might have out-of-date system header files installed in /usr/include.  Those files are installed with:

sudo xcode-select --install

They don't get installed or updated by just installing Xcode itself.  See if that helps.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29057>
_______________________________________

From report at bugs.python.org  Wed Jan 11 14:44:31 2017
From: report at bugs.python.org (Kevin Bonham)
Date: Wed, 11 Jan 2017 19:44:31 +0000
Subject: [issue29245] Can't write to NamedTemporaryFile
Message-ID: <1484163871.38.0.591576246688.issue29245@psf.upfronthosting.co.za>


New submission from Kevin Bonham:

Python 3.6.0 (default, Dec 24 2016, 08:01:42) 
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.42.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from tempfile import NamedTemporaryFile
>>> tmp = NamedTemporaryFile()
>>> tmp.write("hello world!")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/Cellar/python3/3.6.0/Frameworks/Python.framework/Versions/3.6/lib/python3.6/tempfile.py", line 483, in func_wrapper
    return func(*args, **kwargs)
TypeError: a bytes-like object is required, not 'str'
>>> type(tmp)
<class 'tempfile._TemporaryFileWrapper'>

The more verbose error points to issue #18879 (http://bugs.python.org/issue18879), which seems very similar to my problem, but is marked as resolved.

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-23-b9a4238a79b7> in <module>()
----> 1 tmp_file.write("blah")

/usr/local/Cellar/python3/3.6.0/Frameworks/Python.framework/Versions/3.6/lib/python3.6/tempfile.py in func_wrapper(*args, **kwargs)
    481             @_functools.wraps(func)
    482             def func_wrapper(*args, **kwargs):
--> 483                 return func(*args, **kwargs)
    484             # Avoid closing the file as long as the wrapper is alive,
    485             # see issue #18879.

TypeError: a bytes-like object is required, not 'str'

This also seems like it might be related to http://bugs.python.org/issue28867, though I'm getting the same behavior with TemporaryFile:

>>> from tempfile import TemporaryFile
>>> tmp2 = TemporaryFile()
>>> tmp2.write("Hello World")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: a bytes-like object is required, not 'str'


I can do:

>>> with open(tmp.name, "w") as t:
...     t.write("Hello world!")


Is this intended behavior? The docs still say that these functions should return file-like objects.

----------
components: Library (Lib)
messages: 285266
nosy: Kevin Bonham
priority: normal
severity: normal
status: open
title: Can't write to NamedTemporaryFile
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29245>
_______________________________________

From report at bugs.python.org  Wed Jan 11 14:50:40 2017
From: report at bugs.python.org (KeyWeeUsr)
Date: Wed, 11 Jan 2017 19:50:40 +0000
Subject: [issue29231] Broken MSIs in Python 3.5+
In-Reply-To: <1484082847.93.0.115252889304.issue29231@psf.upfronthosting.co.za>
Message-ID: <1484164240.3.0.184212365666.issue29231@psf.upfronthosting.co.za>


KeyWeeUsr added the comment:

> Ah, you're using the full installer, which has the non-debug versions embedded and therefore does not ever need to download them.

Hm, makes sense now, however, isn't that a bug? I mean, even if it's embeded... I'm just curious ^^

> If you start from the web installer...

Aha, there's my problem! After using the web installer everything seems ok for me. Python now runs happily!

RE the unsupported: yeah, I kind of ignored that intentionally. Don't get me wrong, but the functionality you so badly want to push away is a wonderful thing. Ok, there's venv, I'm mostly satisfied with that or system-wide interpreter, but there are cases when you have to debug some annoying problems and you might broke the installation before thus a hypotetical issue arises. Maybe you deleted a file unintentionally, put a header for compilation here or there, renamed something...

What is the first thing someone asks you on for example IRC? "Have you tried a fresh Python installation?" Uhm, sure, because it's so easy if the installer throws things everywhere it can (registry, p&f, ...).

That's mostly not an issue, simple "Repair" is enough unless... you have a large build environment and reinstalling the environment from scratch could take a **long** while depending on the complexity of the environment. How do you test a fresh installation then? <-- Really, that's a serious question. I could come only with this before, though nuget coule be usable too ^^

Also, the network installer is almost the same thing as common MSI, but without elevation. It's like ~500kB or something and does basically this - adds keys to registry, to p&f, etc with a folder that you choose for installing via the network installer afaik. I just skip the part that makes Python unportable or burned into the system. What I do is basically what you do after compilation. Take binaries and put them somewhere, so that they run, nothing less, nothing more.

Thanks for your help, guys :)

----------
resolution: not a bug -> works for me

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29231>
_______________________________________

From report at bugs.python.org  Wed Jan 11 14:55:09 2017
From: report at bugs.python.org (R. David Murray)
Date: Wed, 11 Jan 2017 19:55:09 +0000
Subject: [issue29245] Can't write to NamedTemporaryFile
In-Reply-To: <1484163871.38.0.591576246688.issue29245@psf.upfronthosting.co.za>
Message-ID: <1484164509.31.0.758115769835.issue29245@psf.upfronthosting.co.za>


R. David Murray added the comment:

Yes, it is intended behavior, and it is documented.  The default mode for NamedTemporaryFile (and TemporaryFile) is shown in the docs as "w+b".

I suppose that used to be more convenient in python2 where there was almost no distinction between binary and text on unix, but I think we are unlikely to change the default at this point.

----------
nosy: +r.david.murray
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29245>
_______________________________________

From report at bugs.python.org  Wed Jan 11 15:31:36 2017
From: report at bugs.python.org (Pander)
Date: Wed, 11 Jan 2017 20:31:36 +0000
Subject: [issue6331] Add unicode script info to the unicode database
In-Reply-To: <1245790259.44.0.08771310344.issue6331@psf.upfronthosting.co.za>
Message-ID: <1484166696.55.0.488064833776.issue6331@psf.upfronthosting.co.za>


Pander added the comment:

Any updates or ideas on how to move this forward? See also https://bugs.python.org/issue16684 Thanks.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue6331>
_______________________________________

From report at bugs.python.org  Wed Jan 11 15:32:05 2017
From: report at bugs.python.org (Pander)
Date: Wed, 11 Jan 2017 20:32:05 +0000
Subject: [issue16684] Unicode property value abbreviated names and long names
In-Reply-To: <1355506392.97.0.314398568458.issue16684@psf.upfronthosting.co.za>
Message-ID: <1484166725.16.0.680920544158.issue16684@psf.upfronthosting.co.za>


Pander added the comment:

Any updates or ideas on how to move this forward? Meanwhile, should the issue relate to version 3.6? Thanks. Ah, see also https://bugs.python.org/issue6331 please

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue16684>
_______________________________________

From report at bugs.python.org  Wed Jan 11 15:37:51 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Wed, 11 Jan 2017 20:37:51 +0000
Subject: [issue6331] Add unicode script info to the unicode database
In-Reply-To: <1245790259.44.0.08771310344.issue6331@psf.upfronthosting.co.za>
Message-ID: <1484167071.16.0.221896856301.issue6331@psf.upfronthosting.co.za>


Changes by Serhiy Storchaka <storchaka+cpython at gmail.com>:


----------
versions: +Python 3.7 -Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue6331>
_______________________________________

From report at bugs.python.org  Wed Jan 11 15:38:16 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Wed, 11 Jan 2017 20:38:16 +0000
Subject: [issue16684] Unicode property value abbreviated names and long names
In-Reply-To: <1355506392.97.0.314398568458.issue16684@psf.upfronthosting.co.za>
Message-ID: <1484167096.01.0.00503450789259.issue16684@psf.upfronthosting.co.za>


Changes by Serhiy Storchaka <storchaka+cpython at gmail.com>:


----------
versions: +Python 3.7 -Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue16684>
_______________________________________

From report at bugs.python.org  Wed Jan 11 15:56:48 2017
From: report at bugs.python.org (Shane Smith)
Date: Wed, 11 Jan 2017 20:56:48 +0000
Subject: [issue27603] Migrate IDLE context menu items to shell extension
In-Reply-To: <1469332799.74.0.70339682021.issue27603@psf.upfronthosting.co.za>
Message-ID: <1484168208.13.0.581258124473.issue27603@psf.upfronthosting.co.za>


Shane Smith added the comment:

Since you're a developer, I'm sure you need a lot of versions installed so you can check for backwards combatibility (spelling intentional).  But until recently, only the most recent IDLE was in the context menu, so I'm guessing your workflow for that checking doesn't depend on the context menu.  (I liked the old way, BTW, but I can see how someone might not).

But so long as you're doing it the current way, it remains easily hacked back to the way I like it, so I don't have a whole lot to complain about.  Carry on, good sir.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue27603>
_______________________________________

From report at bugs.python.org  Wed Jan 11 16:17:04 2017
From: report at bugs.python.org (Greg Bengeult)
Date: Wed, 11 Jan 2017 21:17:04 +0000
Subject: [issue11681] -b option undocumented
In-Reply-To: <1301100825.87.0.69977993755.issue11681@psf.upfronthosting.co.za>
Message-ID: <1484169424.21.0.0184029686476.issue11681@psf.upfronthosting.co.za>


Greg Bengeult added the comment:

I have attached a patch for 2.7 that adds -b and -bb to the command line documentation in Modules/main.c and Doc/library/cmdline.rst, following the suggested text provided by Martin.

Interestingly, unicode(), bytearray(), and str() don't seem to be transitive in 2.7.

>>> u"3" == str(3)
True

>>> str(3) == bytearray("3")
True

>>> u"3" == bytearray("3")
False

----------
nosy: +gbengeult
Added file: http://bugs.python.org/file46261/b_option.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue11681>
_______________________________________

From report at bugs.python.org  Wed Jan 11 16:23:26 2017
From: report at bugs.python.org (Eric Fahlgren)
Date: Wed, 11 Jan 2017 21:23:26 +0000
Subject: [issue23407] os.walk always follows Windows junctions
In-Reply-To: <1423342763.81.0.179122512075.issue23407@psf.upfronthosting.co.za>
Message-ID: <1484169806.31.0.973153686338.issue23407@psf.upfronthosting.co.za>


Eric Fahlgren added the comment:

> # Junctions are not recognized as links.        self.assertFalse(os.path.islink(self.junction))

If the above comment is intended as a statement of fact, then it's inconsistent with the implementation of Py_DeleteFileW ( https://hg.python.org/cpython/file/v3.6.0/Modules/posixmodule.c#l4178 ).

----------
nosy: +eric.fahlgren

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23407>
_______________________________________

From report at bugs.python.org  Wed Jan 11 16:28:17 2017
From: report at bugs.python.org (Antoine Pitrou)
Date: Wed, 11 Jan 2017 21:28:17 +0000
Subject: [issue6331] Add unicode script info to the unicode database
In-Reply-To: <1245790259.44.0.08771310344.issue6331@psf.upfronthosting.co.za>
Message-ID: <1484170097.78.0.260969695496.issue6331@psf.upfronthosting.co.za>


Changes by Antoine Pitrou <pitrou at free.fr>:


----------
nosy:  -pitrou

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue6331>
_______________________________________

From report at bugs.python.org  Wed Jan 11 16:40:03 2017
From: report at bugs.python.org (George King)
Date: Wed, 11 Jan 2017 21:40:03 +0000
Subject: [issue29057] Compiler failure on Mac OS X - sys/random.h
In-Reply-To: <1482514649.72.0.460227366509.issue29057@psf.upfronthosting.co.za>
Message-ID: <1484170803.84.0.337227616205.issue29057@psf.upfronthosting.co.za>


George King added the comment:

I reinstalled the command line tools by downloading from developer.apple.com/download/more and the problem went away. No idea how they broke; I had previously installed the same version. In any case, sorry for the noise.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29057>
_______________________________________

From report at bugs.python.org  Wed Jan 11 17:04:22 2017
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 11 Jan 2017 22:04:22 +0000
Subject: [issue29240] Implementation of the PEP 540: Add a new UTF-8 mode
In-Reply-To: <1484133592.17.0.263024020483.issue29240@psf.upfronthosting.co.za>
Message-ID: <1484172262.4.0.149115202844.issue29240@psf.upfronthosting.co.za>


STINNER Victor added the comment:

pep540-2.patch: Patch version 2, updated to the latest version of the PEP 540. It has no more FIXME/TODO and has more unit tests. The main change is that the strict mode doesn't use strict anymore for OS data, but keeps surrogateescape. See the PEP for the rationale (especially the "Use the strict error handler for operating system data" alternative).

----------
Added file: http://bugs.python.org/file46262/pep540-2.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29240>
_______________________________________

From report at bugs.python.org  Wed Jan 11 17:13:06 2017
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 11 Jan 2017 22:13:06 +0000
Subject: [issue29240] Implementation of the PEP 540: Add a new UTF-8 mode
In-Reply-To: <1484133592.17.0.263024020483.issue29240@psf.upfronthosting.co.za>
Message-ID: <1484172786.5.0.94361760264.issue29240@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Oops, I introduced an obvious bug in my latest refactoring. It's now fixed in the patch version 3: pep540-3.patch.

----------
Added file: http://bugs.python.org/file46263/pep540-3.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29240>
_______________________________________

From report at bugs.python.org  Wed Jan 11 18:00:07 2017
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 11 Jan 2017 23:00:07 +0000
Subject: [issue29240] Implementation of the PEP 540: Add a new UTF-8 mode
In-Reply-To: <1484133592.17.0.263024020483.issue29240@psf.upfronthosting.co.za>
Message-ID: <1484175607.54.0.0577289957782.issue29240@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Hum, pep540-3.patch doesn't work if the locale encoding is different than ASCII and UTF-8. argv must be reencoded:

$ LC_ALL=fr_FR ./python -X utf8 -c 'import sys; print(ascii(sys.argv))' $(echo -ne "\xff")
['-c', '\xff']

The result should not depend on the locale, it should be the same than:

$ LC_ALL=fr_FR.utf8 ./python -X utf8 -c 'import sys; print(ascii(sys.argv))' $(echo -ne "\xff")
['-c', '\udcff']

$ LC_ALL=C ./python -X utf8 -c 'import sys; print(ascii(sys.argv))' $(echo -ne "\xff")
['-c', '\udcff']

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29240>
_______________________________________

From report at bugs.python.org  Wed Jan 11 18:01:39 2017
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 11 Jan 2017 23:01:39 +0000
Subject: [issue29240] Implementation of the PEP 540: Add a new UTF-8 mode
In-Reply-To: <1484133592.17.0.263024020483.issue29240@psf.upfronthosting.co.za>
Message-ID: <1484175699.32.0.211997909814.issue29240@psf.upfronthosting.co.za>


STINNER Victor added the comment:

I only tested the the PEP 540 implementation on Linux.

The PEP and its implementation should adjusted for Windows, especially Windows-only env vars like PYTHONLEGACYWINDOWSFSENCODING.

Changes are maybe also needed for Mac OS X and Android, which always use UTF-8. Currently, the locale encoding is still used on these platforms (ex: by open()). Is it possible to a locale encoding different than UTF-8 on Android for example?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29240>
_______________________________________

From report at bugs.python.org  Wed Jan 11 18:03:00 2017
From: report at bugs.python.org (Spiro Sideris)
Date: Wed, 11 Jan 2017 23:03:00 +0000
Subject: [issue29246] typing.Union raises RecursionError when comparing Union
 to other type
Message-ID: <1484175780.67.0.919413735976.issue29246@psf.upfronthosting.co.za>


New submission from Spiro Sideris:

The typing.Union module raises a RecursionError when comparing a Union with no additional tree_args to a type that is not a Union.

An example of the stack trace with the added test looks like the following:

0:00:00 [1/1] test_typing
test test_typing failed -- Traceback (most recent call last):
  File "/Users/spiro/Development/open_source/cpython/Lib/test/test_typing.py", line 221, in test_union_compare_other
    self.assertNotEqual(Union, object)
  File "/Users/spiro/Development/open_source/cpython/Lib/unittest/case.py", line 827, in assertNotEqual
    if not first != second:
  File "/Users/spiro/Development/open_source/cpython/Lib/typing.py", line 760, in __eq__
    return self._subs_tree() == other#return self._subs_tree() is not self and self._subs_tree() == other
  File "/Users/spiro/Development/open_source/cpython/Lib/typing.py", line 760, in __eq__
    return self._subs_tree() == other#return self._subs_tree() is not self and self._subs_tree() == other
  File "/Users/spiro/Development/open_source/cpython/Lib/typing.py", line 760, in __eq__
    return self._subs_tree() == other#return self._subs_tree() is not self and self._subs_tree() == other
  [Previous line repeated 233 more times]
  File "/Users/spiro/Development/open_source/cpython/Lib/typing.py", line 759, in __eq__
    if not isinstance(other, _Union):
RecursionError: maximum recursion depth exceeded in __instancecheck__

test_typing failed

1 test failed:
    test_typing

Total duration: 316 ms
Tests result: FAILURE

The test compares a Union with no tree_args to another type.

----------
components: Library (Lib)
files: spirowork.patch
keywords: patch
messages: 285279
nosy: spiside
priority: normal
severity: normal
status: open
title: typing.Union raises RecursionError when comparing Union to other type
type: crash
versions: Python 2.7, Python 3.3, Python 3.4, Python 3.5, Python 3.6, Python 3.7
Added file: http://bugs.python.org/file46264/spirowork.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29246>
_______________________________________

From report at bugs.python.org  Wed Jan 11 18:57:12 2017
From: report at bugs.python.org (INADA Naoki)
Date: Wed, 11 Jan 2017 23:57:12 +0000
Subject: [issue29240] Implementation of the PEP 540: Add a new UTF-8 mode
In-Reply-To: <1484133592.17.0.263024020483.issue29240@psf.upfronthosting.co.za>
Message-ID: <1484179032.5.0.309996769515.issue29240@psf.upfronthosting.co.za>


INADA Naoki added the comment:

> Hum, pep540-3.patch doesn't work if the locale encoding is different than ASCII and UTF-8. argv must be reencoded:

I want to skip reencoding.
On UTF-8 mode, arbitrary bytes in cmdline (e.g. broken filename passed by xarg) should be able to roundtrip by UTF-8/surrogateescape.

I don't trust wcstombs/mbstowcs.  It may not guarantee round tripping of arbitrary bytes.

Can -X utf8 option be processed before Py_Main()?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29240>
_______________________________________

From report at bugs.python.org  Wed Jan 11 21:56:23 2017
From: report at bugs.python.org (Greg Ward)
Date: Thu, 12 Jan 2017 02:56:23 +0000
Subject: [issue29218] distutils: Remove unused install_misc class
In-Reply-To: <1484002873.56.0.81761190557.issue29218@psf.upfronthosting.co.za>
Message-ID: <1484189783.65.0.0261827834927.issue29218@psf.upfronthosting.co.za>


Greg Ward added the comment:

LGTM, after a cursory glance at the code history. Been a loooong time since I understood this stuff deeply, though!

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29218>
_______________________________________

From report at bugs.python.org  Wed Jan 11 22:00:43 2017
From: report at bugs.python.org (Thomas Nyberg)
Date: Thu, 12 Jan 2017 03:00:43 +0000
Subject: =?utf-8?q?=5Bissue24875=5D_pyvenv_doesn=C2=B4t_install_PIP_inside_a_new_v?=
 =?utf-8?q?env_with_--system-site-package?=
In-Reply-To: <1439699040.07.0.199956094974.issue24875@psf.upfronthosting.co.za>
Message-ID: <1484190043.65.0.952031725654.issue24875@psf.upfronthosting.co.za>


Thomas Nyberg added the comment:

Hi Mark Haase,

I've gone through both of your patches and they both work for me as they should. I'm not sure why the first one isn't working for you, since it works for me. That one seems like it's solving the problem the "right" way. In any case, the second patch works for me as well so if that works for you, then at least we can confirm that it works for both your version of OSX and debian 8 which I'm running.

To any other higher powers: is Mark's second patch acceptable?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24875>
_______________________________________

From report at bugs.python.org  Wed Jan 11 22:06:53 2017
From: report at bugs.python.org (Nick Coghlan)
Date: Thu, 12 Jan 2017 03:06:53 +0000
Subject: [issue11681] -b option undocumented
In-Reply-To: <1301100825.87.0.69977993755.issue11681@psf.upfronthosting.co.za>
Message-ID: <1484190413.78.0.255991674941.issue11681@psf.upfronthosting.co.za>


Nick Coghlan added the comment:

Right, the lack of transitivity comes from the fact that:

    >>> u"3" == str(3)
    True

Is really shorthand for:

    >>> u"3" == str(3).decode("ascii")
    True

However, the implicit decoding only triggers for *exactly* bytes instances, so in the bytearray case it needs to be explicit:

    >>> u"3" == bytearray(b"3").decode("ascii")
    True

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue11681>
_______________________________________

From report at bugs.python.org  Wed Jan 11 22:26:37 2017
From: report at bugs.python.org (Nick Coghlan)
Date: Thu, 12 Jan 2017 03:26:37 +0000
Subject: [issue11681] -b option undocumented
In-Reply-To: <1301100825.87.0.69977993755.issue11681@psf.upfronthosting.co.za>
Message-ID: <1484191597.92.0.889246117789.issue11681@psf.upfronthosting.co.za>


Nick Coghlan added the comment:

Added some review comments on the patch. The only critical one was changing a :class:`bytes` reference to :class:`unicode`, but I also suggested it may be worth mentioning that it *doesn't* enable warnings about all binary/text comparisons the way the Python 3 one does.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue11681>
_______________________________________

From report at bugs.python.org  Wed Jan 11 23:00:54 2017
From: report at bugs.python.org (Jim Fasarakis-Hilliard)
Date: Thu, 12 Jan 2017 04:00:54 +0000
Subject: [issue29246] typing.Union raises RecursionError when comparing Union
 to other type
In-Reply-To: <1484175780.67.0.919413735976.issue29246@psf.upfronthosting.co.za>
Message-ID: <1484193654.65.0.295152991038.issue29246@psf.upfronthosting.co.za>


Changes by Jim Fasarakis-Hilliard <d.f.hilliard at gmail.com>:


----------
type: crash -> behavior
versions:  -Python 2.7, Python 3.3, Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29246>
_______________________________________

From report at bugs.python.org  Wed Jan 11 23:23:54 2017
From: report at bugs.python.org (Nathaniel Smith)
Date: Thu, 12 Jan 2017 04:23:54 +0000
Subject: [issue29247] Document return value of epoll.poll
Message-ID: <1484195034.62.0.718564451527.issue29247@psf.upfronthosting.co.za>


New submission from Nathaniel Smith:

The documentation for select.epoll.poll doesn't document the return value at all, which is somewhat important information :-)

I think it's a list of (fd, eventmask) tuples?

https://docs.python.org/3.7/library/select.html#select.epoll.poll

----------
assignee: docs at python
components: Documentation
messages: 285285
nosy: docs at python, njs
priority: normal
severity: normal
status: open
title: Document return value of epoll.poll
versions: Python 2.7, Python 3.5, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29247>
_______________________________________

From report at bugs.python.org  Wed Jan 11 23:32:19 2017
From: report at bugs.python.org (Kevin Christopher Henry)
Date: Thu, 12 Jan 2017 04:32:19 +0000
Subject: [issue22343] Install bash activate script on Windows when using venv
In-Reply-To: <1409948648.7.0.892236136888.issue22343@psf.upfronthosting.co.za>
Message-ID: <1484195539.0.0.394776846241.issue22343@psf.upfronthosting.co.za>


Kevin Christopher Henry added the comment:

I can confirm that LF endings work fine on Cygwin, and it's hard to imagine any bash implementation not handling that properly.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue22343>
_______________________________________

From report at bugs.python.org  Thu Jan 12 00:45:39 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Thu, 12 Jan 2017 05:45:39 +0000
Subject: [issue29099] sqlite3 timestamp converter cannot handle timezone
In-Reply-To: <1482966269.67.0.620298847284.issue29099@psf.upfronthosting.co.za>
Message-ID: <1484199939.01.0.141144635481.issue29099@psf.upfronthosting.co.za>


Xiang Zhang added the comment:

timestamptz.patch implements a new converter that's able to convert aware datetime objects.

----------
stage: needs patch -> patch review
Added file: http://bugs.python.org/file46265/timestamptz.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29099>
_______________________________________

From report at bugs.python.org  Thu Jan 12 01:06:22 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Thu, 12 Jan 2017 06:06:22 +0000
Subject: [issue29099] sqlite3 timestamp converter cannot handle timezone
In-Reply-To: <1482966269.67.0.620298847284.issue29099@psf.upfronthosting.co.za>
Message-ID: <1484201182.7.0.786093794687.issue29099@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

I think the timestamptz converter should either interpret strings without timezone as UTC (and perhaps understand the "Z" suffix as sqlite3 date() function) or raises an error. It should never return naive datetime.

The timestamp converter needs better error reporting when get an input with a timezone.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29099>
_______________________________________

From report at bugs.python.org  Thu Jan 12 01:24:39 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Thu, 12 Jan 2017 06:24:39 +0000
Subject: [issue29099] sqlite3 timestamp converter cannot handle timezone
In-Reply-To: <1482966269.67.0.620298847284.issue29099@psf.upfronthosting.co.za>
Message-ID: <1484202279.09.0.469584625318.issue29099@psf.upfronthosting.co.za>


Xiang Zhang added the comment:

> I think the timestamptz converter should either interpret strings without timezone as UTC (and perhaps understand the "Z" suffix as sqlite3 date() function) or raises an error. It should never return naive datetime.

Currently timestamptz just convert back what the user passed in, no matter naive or aware objects. What to do with them is left to the app. If we raise an error, could users use naive and aware objects together? And interpreting strings without timezone as UTC seems will mistranslate the object. For example, pass in datetime.datetime.now() and translate it back as UTC may not be right.

> The timestamp converter needs better error reporting when get an input with a timezone.

I thought about it but our intention to introduce a new converter is not to break old code. Doesn't add error reporting violate the intention? Users' code may not catch the error now.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29099>
_______________________________________

From report at bugs.python.org  Thu Jan 12 02:07:47 2017
From: report at bugs.python.org (JGoutin)
Date: Thu, 12 Jan 2017 07:07:47 +0000
Subject: [issue29241] sys._enablelegacywindowsfsencoding() don't apply to
 os.fsencode and os.fsdecode
In-Reply-To: <1484137681.59.0.0301192140292.issue29241@psf.upfronthosting.co.za>
Message-ID: <1484204867.11.0.47967616488.issue29241@psf.upfronthosting.co.za>


JGoutin added the comment:

import sys

# Force the use of legacy encoding like versions of Python prior to 3.6.
sys._enablelegacywindowsfsencoding()

# Show actual file system encoding
encoding = sys.getfilesystemencoding()
print('File system encoding:', encoding)

# os.fsencode(filename) VS filename.encode(File system encoding)
import os
print(os.fsencode('?'), '?'.encode(encoding))

>>> File system encoding: mbcs
>>> b'\xc3\xa9' b'\xe9'


The result is the same.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29241>
_______________________________________

From report at bugs.python.org  Thu Jan 12 02:28:40 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Thu, 12 Jan 2017 07:28:40 +0000
Subject: [issue29099] sqlite3 timestamp converter cannot handle timezone
In-Reply-To: <1482966269.67.0.620298847284.issue29099@psf.upfronthosting.co.za>
Message-ID: <1484206120.56.0.38487171515.issue29099@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

Naive and aware objects should not be mixed. Their actually are different types. The converter doesn't return str or bytes when can't parse an input as a datetime, and it shouldn't return a naive datetime if can't find a timezone.

SQLite date and time functions imply UTC if timezone is omitted. This is a reason for returning aware UTC datetime objects. On other side, Python is more powerful programming language, it distinguish naive and aware datetime objects, and it is unlikely that these two types are mixed in one database column. It is better to raise an error that silently return possible wrong result. It exceptional case user can write special converter or just call SQLite datetime() for unifying data format.

I think the old code unlikely will be broken if preserve an exception type and don't change conditions for raising an error. New error message can contain full input string and suggest to use the timestamptz converter if it looks as a datetime with timezone.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29099>
_______________________________________

From report at bugs.python.org  Thu Jan 12 02:40:47 2017
From: report at bugs.python.org (Roundup Robot)
Date: Thu, 12 Jan 2017 07:40:47 +0000
Subject: [issue29006] 2.7.13 _sqlite more prone to "database table is locked"
In-Reply-To: <1482095939.25.0.891516594682.issue29006@psf.upfronthosting.co.za>
Message-ID: <20170112074044.50795.69603.1DA7959B@psf.io>


Roundup Robot added the comment:

New changeset dd13098a5dc2 by Benjamin Peterson in branch '2.7':
revert 030e100f048a (#29006, #10513)
https://hg.python.org/cpython/rev/dd13098a5dc2

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29006>
_______________________________________

From report at bugs.python.org  Thu Jan 12 02:40:47 2017
From: report at bugs.python.org (Roundup Robot)
Date: Thu, 12 Jan 2017 07:40:47 +0000
Subject: [issue10513] sqlite3.InterfaceError after commit
In-Reply-To: <1290515734.43.0.643837241692.issue10513@psf.upfronthosting.co.za>
Message-ID: <20170112074044.50795.36684.CA6A7D14@psf.io>


Roundup Robot added the comment:

New changeset dd13098a5dc2 by Benjamin Peterson in branch '2.7':
revert 030e100f048a (#29006, #10513)
https://hg.python.org/cpython/rev/dd13098a5dc2

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue10513>
_______________________________________

From report at bugs.python.org  Thu Jan 12 02:41:11 2017
From: report at bugs.python.org (Benjamin Peterson)
Date: Thu, 12 Jan 2017 07:41:11 +0000
Subject: [issue10513] sqlite3.InterfaceError after commit
In-Reply-To: <1290515734.43.0.643837241692.issue10513@psf.upfronthosting.co.za>
Message-ID: <1484206871.36.0.587057724729.issue10513@psf.upfronthosting.co.za>


Changes by Benjamin Peterson <benjamin at python.org>:


----------
resolution: fixed -> 
status: closed -> open

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue10513>
_______________________________________

From report at bugs.python.org  Thu Jan 12 03:09:21 2017
From: report at bugs.python.org (Eryk Sun)
Date: Thu, 12 Jan 2017 08:09:21 +0000
Subject: [issue29248] os.readlink fails on Windows
Message-ID: <1484208560.98.0.375529555783.issue29248@psf.upfronthosting.co.za>


New submission from Eryk Sun:

win_readlink in Modules/posixmodule.c mistakenly treats the PrintNameOffset field of the reparse data buffer as a number of characters instead of bytes. Thus, if the offset is non-zero, the value returned is incorrect stack garbage. For example, the following should return "C:\\ProgramData":

    >>> os.readlink(r'C:\Users\All Users')
    '\u6c20\u3012\u041f\x01\u2768\u60b2\u031b\x02\x05\x00\u031e\x06\u8c01\u4012'

Craig Holmquist found this bug, as detailed in message 277385. He included a fix in the patch for issue 23407, but this should be addressed in the next maintenance release of 3.5 and 3.6.

----------
components: Library (Lib), Windows
messages: 285294
nosy: eryksun, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
stage: needs patch
status: open
title: os.readlink fails on Windows
type: behavior
versions: Python 3.5, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29248>
_______________________________________

From report at bugs.python.org  Thu Jan 12 03:36:14 2017
From: report at bugs.python.org (Gian-Carlo Pascutto)
Date: Thu, 12 Jan 2017 08:36:14 +0000
Subject: [issue29006] 2.7.13 _sqlite more prone to "database table is locked"
In-Reply-To: <1482095939.25.0.891516594682.issue29006@psf.upfronthosting.co.za>
Message-ID: <1484210174.16.0.500428336389.issue29006@psf.upfronthosting.co.za>


Gian-Carlo Pascutto added the comment:

>Then we wait for someone that really knows why the change was done in the first place.

Python 2.7 had a regression compared to 2.6 where a SELECT after a COMMIT would silently return the wrong data:

http://bugs.python.org/issue23129
http://bugs.python.org/issue10513

Neither exiting with a locked error nor producing the wrong data are particularly appealing results. Did 2.6 pass those sets that caused the attempted fix to get backed out?

----------
nosy: +Gian-Carlo Pascutto

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29006>
_______________________________________

From report at bugs.python.org  Thu Jan 12 04:18:36 2017
From: report at bugs.python.org (STINNER Victor)
Date: Thu, 12 Jan 2017 09:18:36 +0000
Subject: [issue29240] Implementation of the PEP 540: Add a new UTF-8 mode
In-Reply-To: <1484133592.17.0.263024020483.issue29240@psf.upfronthosting.co.za>
Message-ID: <1484212716.64.0.708448732138.issue29240@psf.upfronthosting.co.za>


STINNER Victor added the comment:

> Can -X utf8 option be processed before Py_Main()?

I'm trying to implement that, but it's hard to factorize the code. I will probably have to duplicate the code handling -E, -X utf8, PYTHONMALLOC and PYTHONUTF8 for wchar_t* (UCS4 or UTF-16) and char* (bytes).

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29240>
_______________________________________

From report at bugs.python.org  Thu Jan 12 05:19:41 2017
From: report at bugs.python.org (Chi Hsuan Yen)
Date: Thu, 12 Jan 2017 10:19:41 +0000
Subject: [issue29240] Implementation of the PEP 540: Add a new UTF-8 mode
In-Reply-To: <1484133592.17.0.263024020483.issue29240@psf.upfronthosting.co.za>
Message-ID: <1484216381.88.0.338848510963.issue29240@psf.upfronthosting.co.za>


Changes by Chi Hsuan Yen <yan12125 at gmail.com>:


----------
nosy: +Chi Hsuan Yen

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29240>
_______________________________________

From report at bugs.python.org  Thu Jan 12 05:19:54 2017
From: report at bugs.python.org (Jon Walsh)
Date: Thu, 12 Jan 2017 10:19:54 +0000
Subject: [issue29249] Pathlib glob ** bug
Message-ID: <1484216394.04.0.57434547579.issue29249@psf.upfronthosting.co.za>


New submission from Jon Walsh:

>>> from pathlib import Path
>>> Path("a/b/c/d/e.txt").match('a/*/**/*')
False

----------
components: Library (Lib)
messages: 285297
nosy: Jon Walsh
priority: normal
severity: normal
status: open
title: Pathlib glob ** bug
type: behavior
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29249>
_______________________________________

From report at bugs.python.org  Thu Jan 12 05:32:24 2017
From: report at bugs.python.org (STINNER Victor)
Date: Thu, 12 Jan 2017 10:32:24 +0000
Subject: [issue29240] Implementation of the PEP 540: Add a new UTF-8 mode
In-Reply-To: <1484216381.91.0.0400781106212.issue29240@psf.upfronthosting.co.za>
Message-ID: <CAMpsgwZDpMBG0QaC5qoc6ChohR6u7qnrsebAnaH-=M8sKw-OmQ@mail.gmail.com>


STINNER Victor added the comment:

Hum, test_utf8mode lacks an unit test on the -E command line option:
PYTHONUTF8 should be ignored if -E is used.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29240>
_______________________________________

From report at bugs.python.org  Thu Jan 12 05:42:45 2017
From: report at bugs.python.org (Eryk Sun)
Date: Thu, 12 Jan 2017 10:42:45 +0000
Subject: [issue29250] islink and stat follow_symlinks are inconsistent on
 Windows
Message-ID: <1484217765.9.0.444376233166.issue29250@psf.upfronthosting.co.za>


New submission from Eryk Sun:

win32_xstat_impl in Modules/posixmodule.c doesn't check the value of reparse_tag to limit traversal to only tags that are considered links. So we have cases in which os.path.islink returns False (e.g. junctions, pending the resolution of issue 23407), but os.stat and os.lstat disagree. For example:

    >>> os.path.islink(r'C:\Documents and Settings')
    False
    >>> s = os.stat(r'C:\Documents and Settings')
    >>> ls = os.lstat(r'C:\Documents and Settings')
    >>> s.st_ino == ls.st_ino
    False

At its strictest, traversal would be limited to just symbolic links. It could be relaxed to also include junctions, and even further to include all tags with the name surrogate bit set [1]. This includes the following Microsoft tags

    IO_REPARSE_TAG_MOUNT_POINT (junctions)
    IO_REPARSE_TAG_SYMLINK
    IO_REPARSE_TAG_IIS_CACHE

and non-Microsoft tags: 

    IO_REPARSE_TAG_SOLUTIONSOFT
    IO_REPARSE_TAG_OSR_SAMPLE
    IO_REPARSE_TAG_QI_TECH_HSM
    IO_REPARSE_TAG_MAXISCALE_HSM

I'm creating this issue as an enhancement for 3.7. The problem has existed for a long time, but it hasn't been a thorn in anyone's side as far as I know.

[1]: https://msdn.microsoft.com/en-us/library/aa365511

----------
components: Library (Lib), Windows
messages: 285299
nosy: eryksun, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
stage: needs patch
status: open
title: islink and stat follow_symlinks are inconsistent on Windows
type: enhancement
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29250>
_______________________________________

From report at bugs.python.org  Thu Jan 12 05:48:45 2017
From: report at bugs.python.org (Martin Panter)
Date: Thu, 12 Jan 2017 10:48:45 +0000
Subject: [issue17824] pty.spawn handles errors improperly
In-Reply-To: <1366772370.17.0.184801312009.issue17824@psf.upfronthosting.co.za>
Message-ID: <1484218125.46.0.00277213760782.issue17824@psf.upfronthosting.co.za>


Martin Panter added the comment:

The patch for Issue 26228 proposes an improvement to the situation, although it is not perfect and does not include a test. I wonder if it is possible to replace fork() and execlp() with a subprocess.Popen invokation, at least in a new Python release?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue17824>
_______________________________________

From report at bugs.python.org  Thu Jan 12 05:52:28 2017
From: report at bugs.python.org (Eryk Sun)
Date: Thu, 12 Jan 2017 10:52:28 +0000
Subject: [issue23407] os.walk always follows Windows junctions
In-Reply-To: <1423342763.81.0.179122512075.issue23407@psf.upfronthosting.co.za>
Message-ID: <1484218348.77.0.417232344925.issue23407@psf.upfronthosting.co.za>


Eryk Sun added the comment:

I opened issue 29248 for the os.readlink bug and issue 29250 for the inconsistency between os.path.islink and os.stat.

Handling junctions as links is new behavior, so I've changed this issue to be an enhancement for 3.7.

If the notion of a link is generalized to junctions, then maybe it should be further generalized to include all name-surrogate reparse tags [1]. Currently for Microsoft tags this includes 

    IO_REPARSE_TAG_MOUNT_POINT (junctions)
    IO_REPARSE_TAG_SYMLINK
    IO_REPARSE_TAG_IIS_CACHE

For non-Microsoft tags it includes 

    IO_REPARSE_TAG_SOLUTIONSOFT
    IO_REPARSE_TAG_OSR_SAMPLE
    IO_REPARSE_TAG_QI_TECH_HSM
    IO_REPARSE_TAG_MAXISCALE_HSM

The last two are outliers. HSM isn't the kind of immediate, fast access that one would expect from a symbolic link. All other HSM tags aren't categorized as name surrogates.

[1]: https://msdn.microsoft.com/en-us/library/aa365511

----------
type: behavior -> enhancement
versions: +Python 3.7 -Python 3.4, Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23407>
_______________________________________

From report at bugs.python.org  Thu Jan 12 05:53:22 2017
From: report at bugs.python.org (Eryk Sun)
Date: Thu, 12 Jan 2017 10:53:22 +0000
Subject: [issue29250] islink and stat follow_symlinks are inconsistent on
 Windows
In-Reply-To: <1484217765.9.0.444376233166.issue29250@psf.upfronthosting.co.za>
Message-ID: <1484218402.36.0.0317678273902.issue29250@psf.upfronthosting.co.za>


Changes by Eryk Sun <eryksun at gmail.com>:


----------
dependencies: +os.walk always follows Windows junctions

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29250>
_______________________________________

From report at bugs.python.org  Thu Jan 12 05:54:19 2017
From: report at bugs.python.org (Roundup Robot)
Date: Thu, 12 Jan 2017 10:54:19 +0000
Subject: [issue25591] refactor imaplib tests
In-Reply-To: <1447107407.81.0.130095933409.issue25591@psf.upfronthosting.co.za>
Message-ID: <20170112105415.26421.14433.EF08C36D@psf.io>


Roundup Robot added the comment:

New changeset a10d6ab65b23 by Victor Stinner in branch '3.5':
Issue #25591: Fix test_imaplib if ssl miss
https://hg.python.org/cpython/rev/a10d6ab65b23

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue25591>
_______________________________________

From report at bugs.python.org  Thu Jan 12 05:58:38 2017
From: report at bugs.python.org (Xavier de Gaye)
Date: Thu, 12 Jan 2017 10:58:38 +0000
Subject: [issue28759] access to mkfifo,
 mknod and hard links is controled by SELinux MAC on Android
In-Reply-To: <1479715975.06.0.930254771903.issue28759@psf.upfronthosting.co.za>
Message-ID: <1484218718.32.0.496685518481.issue28759@psf.upfronthosting.co.za>


Xavier de Gaye added the comment:

'supported_operation':
* The intent in using 'supported' was that the context manager could possibly be extended later.
* 'operation' is the terminology used in the GNU libc documentation [1].
I will replace 'supported_operation' with 'permitted_operation' unless a better name is proposed.

[1] https://www.gnu.org/software/libc/manual/html_node/Error-Codes.html

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28759>
_______________________________________

From report at bugs.python.org  Thu Jan 12 06:08:55 2017
From: report at bugs.python.org (Evan)
Date: Thu, 12 Jan 2017 11:08:55 +0000
Subject: [issue22343] Install bash activate script on Windows when using venv
In-Reply-To: <1409948648.7.0.892236136888.issue22343@psf.upfronthosting.co.za>
Message-ID: <1484219335.78.0.0603157237284.issue22343@psf.upfronthosting.co.za>


Evan added the comment:

Works for me in Git Bash.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue22343>
_______________________________________

From report at bugs.python.org  Thu Jan 12 06:11:22 2017
From: report at bugs.python.org (Andrew Dunai)
Date: Thu, 12 Jan 2017 11:11:22 +0000
Subject: [issue29249] Pathlib glob ** bug
In-Reply-To: <1484216394.04.0.57434547579.issue29249@psf.upfronthosting.co.za>
Message-ID: <1484219482.46.0.665635154599.issue29249@psf.upfronthosting.co.za>


Andrew Dunai added the comment:

Isn't this intended? According to https://docs.python.org/2/library/glob.html and wiki, typical UNIX glob pattern does not have the reqursive matching operator (`**`).

----------
nosy: +Andrew Dunai

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29249>
_______________________________________

From report at bugs.python.org  Thu Jan 12 06:18:51 2017
From: report at bugs.python.org (Michael Felt)
Date: Thu, 12 Jan 2017 11:18:51 +0000
Subject: [issue18235] _sysconfigdata.py wrong on AIX installations
In-Reply-To: <1371429201.94.0.372851990361.issue18235@psf.upfronthosting.co.za>
Message-ID: <1484219931.71.0.00929807541471.issue18235@psf.upfronthosting.co.za>


Michael Felt added the comment:

OK - just looked at what happens with Python3-3.6

new file name: _sysconfigdata_m_aix5_.py

values:
root at x064:[/data/prj/python/python3-3.6.0/X64/opt/lib/python3.6]grep SHARED _sysconf*.py | grep LD
 'BLDSHARED': '/opt/lib/python3.6/config-3.6m/ld_so_aix xlc_r '
 'LDCXXSHARED': '/opt/lib/python3.6/config-3.6m/ld_so_aix xlc_r '
 'LDSHARED': '/opt/lib/python3.6/config-3.6m/ld_so_aix xlc_r '


The 'build' worked, so I am guessing that BLDSHARED had a different value that what it shows here. 

The full pathname for ld_so_aix is correct.

As I mentioned in https://bugs.python.org/issue28311#msg277692 - still broken for Python2-2.7(.13) - shall try your patch above (once I figure out how to grab it)

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue18235>
_______________________________________

From report at bugs.python.org  Thu Jan 12 06:19:28 2017
From: report at bugs.python.org (Michael Felt)
Date: Thu, 12 Jan 2017 11:19:28 +0000
Subject: [issue27632] build on AIX fails when builddir != srcdir,
 more than bad path to ld_so_aix
In-Reply-To: <1469608112.86.0.803958819971.issue27632@psf.upfronthosting.co.za>
Message-ID: <1484219968.9.0.522460550294.issue27632@psf.upfronthosting.co.za>


Michael Felt added the comment:

> Regarding reopening Issue 10656, whatever you think is more appropriate. 

Not "scary", just irritating. I'll open a new one when I have more time to document it.

> Issue 16189 and Issue 25825 were about updating to match recent changes to directory names, and I thought we decided the changes were not applicable to 2.7.

My bad. I was looking at the subject titles (cannot load ld_so_aix, cannot build external modules). I'll focus on the 'patch' you mention below. As it stands, I have no idea how to determine if they have any relevance to anything...

> Regarding LDSHARED vs BLDSHARED, isn?t this the same as Issue 28311, which lead to Issue 18235? 

> If you can try the patch I mentioned at <https://bugs.python.org/issue18235#msg277745>, I suspect it may help.

Will do.
> Let me know if you need help adapting the patch for 2.7.

----------
nosy: +aixtools at gmail.com

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue27632>
_______________________________________

From report at bugs.python.org  Thu Jan 12 06:20:42 2017
From: report at bugs.python.org (Christian Heimes)
Date: Thu, 12 Jan 2017 11:20:42 +0000
Subject: [issue29249] Pathlib glob ** bug
In-Reply-To: <1484216394.04.0.57434547579.issue29249@psf.upfronthosting.co.za>
Message-ID: <1484220042.28.0.105669209223.issue29249@psf.upfronthosting.co.za>


Christian Heimes added the comment:

The ticket is not about glob but about pathlib.

Pathlib supports ** directory globbing, but it's only documented as prefix globbing, https://docs.python.org/3/library/pathlib.html#pathlib.Path.glob

----------
nosy: +christian.heimes

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29249>
_______________________________________

From report at bugs.python.org  Thu Jan 12 06:20:52 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Thu, 12 Jan 2017 11:20:52 +0000
Subject: [issue28759] access to mkfifo,
 mknod and hard links is controled by SELinux MAC on Android
In-Reply-To: <1479715975.06.0.930254771903.issue28759@psf.upfronthosting.co.za>
Message-ID: <1484220052.17.0.765969069735.issue28759@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

The purpose of this helper is skipping the test rather than failing if some OS operation is not permitted by current configuration. May be skip_if_not_permitted()?

Similar helpers that convert runtime error in skipping a test: import_module(), get_attribute(), bind_unix_socket(), system_must_validate_cert(). They are more concrete, guard some specific operation.

Context manager is not the only possible syntax. The helper can call a function:

    helper(os.mkfifo, filename)

or be a wrapper:

    helper(os.mkfifo)(filename)

If you prefer such syntax.

There are many possibilities, and I think it is worth to discuss this on Python-Dev.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28759>
_______________________________________

From report at bugs.python.org  Thu Jan 12 06:24:35 2017
From: report at bugs.python.org (Xavier de Gaye)
Date: Thu, 12 Jan 2017 11:24:35 +0000
Subject: [issue29176] /tmp does not exist on Android and is used by
 curses.window.putwin()
In-Reply-To: <1483699620.77.0.827616599041.issue29176@psf.upfronthosting.co.za>
Message-ID: <1484220275.41.0.541187995389.issue29176@psf.upfronthosting.co.za>


Changes by Xavier de Gaye <xdegaye at gmail.com>:


Added file: http://bugs.python.org/file46266/ncurses.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29176>
_______________________________________

From report at bugs.python.org  Thu Jan 12 06:29:12 2017
From: report at bugs.python.org (Christian Heimes)
Date: Thu, 12 Jan 2017 11:29:12 +0000
Subject: [issue29249] Pathlib glob ** bug
In-Reply-To: <1484216394.04.0.57434547579.issue29249@psf.upfronthosting.co.za>
Message-ID: <1484220552.04.0.152768410012.issue29249@psf.upfronthosting.co.za>


Changes by Christian Heimes <lists at cheimes.de>:


----------
nosy: +pitrou

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29249>
_______________________________________

From report at bugs.python.org  Thu Jan 12 06:32:21 2017
From: report at bugs.python.org (Xavier de Gaye)
Date: Thu, 12 Jan 2017 11:32:21 +0000
Subject: [issue29176] /tmp does not exist on Android and is used by
 curses.window.putwin()
In-Reply-To: <1483699620.77.0.827616599041.issue29176@psf.upfronthosting.co.za>
Message-ID: <1484220741.69.0.311371929809.issue29176@psf.upfronthosting.co.za>


Xavier de Gaye added the comment:

The test suite runner in the test.libregrtest.main module calls tempfile.gettempdir() and mkstemp_curses.patch uses the same logic to fix the problem on Android where the tests that I run use TMPDIR=/data/local/tmp on Android API 24.

As a side note, when the curses library is ncurses-6.0, the library must be patched in ncurses/base/lib_screen.c with the attached ncurses.patch, see:
    http://lists.gnu.org/archive/html/bug-ncurses/2016-05/msg00000.html
    https://bugs.python.org/issue27323

----------
Added file: http://bugs.python.org/file46267/mkstemp_curses.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29176>
_______________________________________

From report at bugs.python.org  Thu Jan 12 06:32:45 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Thu, 12 Jan 2017 11:32:45 +0000
Subject: [issue29249] Pathlib glob ** bug
In-Reply-To: <1484216394.04.0.57434547579.issue29249@psf.upfronthosting.co.za>
Message-ID: <1484220765.76.0.501247866149.issue29249@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

** is supported not just as a prefix. Path('./Lib').glob('**/*.py') emits the same paths as Path('.').glob('Lib/**/*.py'). But ** is supported only in glob(), not in match(). The support of ** in match() is not documented. Would be worth to document explicitly that it is not supported.

----------
assignee:  -> docs at python
components: +Documentation -Library (Lib)
nosy: +docs at python, serhiy.storchaka
stage:  -> needs patch
type: behavior -> enhancement
versions: +Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29249>
_______________________________________

From report at bugs.python.org  Thu Jan 12 06:44:18 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Thu, 12 Jan 2017 11:44:18 +0000
Subject: [issue29176] /tmp does not exist on Android and is used by
 curses.window.putwin()
In-Reply-To: <1483699620.77.0.827616599041.issue29176@psf.upfronthosting.co.za>
Message-ID: <1484221458.8.0.148710860471.issue29176@psf.upfronthosting.co.za>


Changes by Serhiy Storchaka <storchaka+cpython at gmail.com>:


----------
nosy: +haypo, serhiy.storchaka
stage: needs patch -> patch review

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29176>
_______________________________________

From report at bugs.python.org  Thu Jan 12 06:47:39 2017
From: report at bugs.python.org (Xavier de Gaye)
Date: Thu, 12 Jan 2017 11:47:39 +0000
Subject: [issue28759] access to mkfifo,
 mknod and hard links is controled by SELinux MAC on Android
In-Reply-To: <1479715975.06.0.930254771903.issue28759@psf.upfronthosting.co.za>
Message-ID: <1484221658.98.0.445088100478.issue28759@psf.upfronthosting.co.za>


Xavier de Gaye added the comment:

Please go ahead and start the discussion on Python-Dev.

----------
assignee: xdegaye -> 

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28759>
_______________________________________

From report at bugs.python.org  Thu Jan 12 07:04:20 2017
From: report at bugs.python.org (Martin Panter)
Date: Thu, 12 Jan 2017 12:04:20 +0000
Subject: [issue29251] Class __dict__ is only a mapping proxy
Message-ID: <1484222660.7.0.65822313324.issue29251@psf.upfronthosting.co.za>


New submission from Martin Panter:

The __dict__ attribute of class objects is documented as being a (standard) dictionary, but implemented with a proxy object. I propose to clarify the documentation in ?Custom classes? under <https://docs.python.org/3.5/reference/datamodel.html#the-standard-type-hierarchy>, and in <https://docs.python.org/3.5/library/functions.html#type>.

I believe my changes are also applicable to Python 2, as long as I point out the proxy is specific to ?new-style? classes.

----------
assignee: docs at python
components: Documentation
files: class-dict.patch
keywords: patch
messages: 285313
nosy: docs at python, martin.panter
priority: normal
severity: normal
stage: patch review
status: open
title: Class __dict__ is only a mapping proxy
type: behavior
versions: Python 2.7, Python 3.5, Python 3.6, Python 3.7
Added file: http://bugs.python.org/file46268/class-dict.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29251>
_______________________________________

From report at bugs.python.org  Thu Jan 12 07:04:49 2017
From: report at bugs.python.org (Michael Felt)
Date: Thu, 12 Jan 2017 12:04:49 +0000
Subject: [issue18235] _sysconfigdata.py wrong on AIX installations
In-Reply-To: <1371429201.94.0.372851990361.issue18235@psf.upfronthosting.co.za>
Message-ID: <1484222689.43.0.146699795019.issue18235@psf.upfronthosting.co.za>


Michael Felt added the comment:

The "key" bit of the patch is to move the code AND make this assignment:

    if _PYTHON_BUILD:
        vars['LDSHARED'] = vars['BLDSHARED']

Besides, imho, skipping around the problem that LDSHARED is not correctly assigned - it cannot work because BLDSHARED is not set to the 'installed' value.

And, frankly I am amazed (in Python3-3.6.0) both are equal AND both pointing at the 'installed' location.

OOPS: I just looked and the patch above is NOT in the Python3-3.6 branch, and as both variables are identical now in _sysconfigdata_m_aix5.py - neither assignment has any affect.

Again: comparing python2 and python3 it seems the latest Python3 now have the fullpath - and equal definitions - for both variables WHILE Python2 use 'relative' pathnames - BUT both are equal.

As far as the patch goes - it is irrevalant and can be removed in Python3 while a quick hack for Python2 would be to use (in the new location!)

    if _PYTHON_BUILD:
        vars['LDSHARED'] = vars['LDCXXSHARED']

Details:
root at x064:[/data/prj/python]grep SHARED python*/X64/opt/lib/python*/_sysconfig*.py | grep LD
python-2.7.12.0/X64/opt/lib/python2.7/_sysconfigdata.py: 'BLDSHARED': './Modules/ld_so_aix xlc_r -bI:./Modules/python.exp',
python-2.7.12.0/X64/opt/lib/python2.7/_sysconfigdata.py: 'LDCXXSHARED': '/opt/lib/python2.7/config/ld_so_aix xlc_r -bI:/opt/lib/python2.7/config/python.exp',
python-2.7.12.0/X64/opt/lib/python2.7/_sysconfigdata.py: 'LDSHARED': './Modules/ld_so_aix xlc_r -bI:./Modules/python.exp',
python-2.7.13/X64/opt/lib/python2.7/_sysconfigdata.py: 'BLDSHARED': 'Modules/ld_so_aix xlc_r -bI:Modules/python.exp -L/opt/lib',
python-2.7.13/X64/opt/lib/python2.7/_sysconfigdata.py: 'LDCXXSHARED': '/opt/lib/python2.7/config/ld_so_aix xlc_r -bI:/opt/lib/python2.7/config/python.exp',
python-2.7.13/X64/opt/lib/python2.7/_sysconfigdata.py: 'LDSHARED': 'Modules/ld_so_aix xlc_r -bI:Modules/python.exp -L/opt/lib',
python3-3.5.2/X64/opt/lib/python3.5/_sysconfigdata.py: 'BLDSHARED': '/opt/lib/python3.5/config/ld_so_aix xlc_r '
python3-3.5.2/X64/opt/lib/python3.5/_sysconfigdata.py: 'LDCXXSHARED': '/opt/lib/python3.5/config/ld_so_aix xlc_r '
python3-3.5.2/X64/opt/lib/python3.5/_sysconfigdata.py: 'LDSHARED': '/opt/lib/python3.5/config/ld_so_aix xlc_r '
python3-3.6.0/X64/opt/lib/python3.6/_sysconfigdata_m_aix5_.py: 'BLDSHARED': '/opt/lib/python3.6/config-3.6m/ld_so_aix xlc_r '
python3-3.6.0/X64/opt/lib/python3.6/_sysconfigdata_m_aix5_.py: 'LDCXXSHARED': '/opt/lib/python3.6/config-3.6m/ld_so_aix xlc_r '
python3-3.6.0/X64/opt/lib/python3.6/_sysconfigdata_m_aix5_.py: 'LDSHARED': '/opt/lib/python3.6/config-3.6m/ld_so_aix xlc_r '

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue18235>
_______________________________________

From report at bugs.python.org  Thu Jan 12 07:26:12 2017
From: report at bugs.python.org (Xavier de Gaye)
Date: Thu, 12 Jan 2017 12:26:12 +0000
Subject: [issue29185] test_distutils fails on Android API level 24
In-Reply-To: <1483737507.74.0.0285105710282.issue29185@psf.upfronthosting.co.za>
Message-ID: <1484223972.01.0.145564458726.issue29185@psf.upfronthosting.co.za>


Xavier de Gaye added the comment:

This new patch skips the second test when hard links fail on the platform under test.

----------
Added file: http://bugs.python.org/file46269/test_distutils_2.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29185>
_______________________________________

From report at bugs.python.org  Thu Jan 12 07:27:27 2017
From: report at bugs.python.org (Decorater)
Date: Thu, 12 Jan 2017 12:27:27 +0000
Subject: [issue29252] self in classes missinterpreted as a string.
Message-ID: <1484224047.15.0.809127294576.issue29252@psf.upfronthosting.co.za>


New submission from Decorater:

So, I have a class (it subclasses the zipimporter for my import hook) that allows me to not worry about issues when the hook is appended to the beginning of the path hooks.

The code to the class as follows:

class OriginalZipImport(zipimport.zipimporter):
    """
    Original zipimporter class. Modified for requirements of this import hook.
    """
    # Sadly I have to manually specify the file extensions the original
    # zipimporter supports to bypass an AttributeError.
    # according to https://docs.python.org/3.6/library/zipimport.html
    # zipimport only allows importing ``.py`` and ``.pyc`` files from zips.
    extensions = ['.py', '.pyc']

    def __init__(self, archivepath):
        super(OriginalZipImport, self.inst).__init__(self, archivepath)

    def find_loader(self, fullname, path=None):
        """
        Original find_loader.
        """
        self.inst.find_loader(fullname, path=path)

    def find_module(self, fullname, path=None):
        """
        Original find_module.
        """
        self.inst.find_module(self, fullname, path=path)

    def load_module(self, fullname):
        """
        Original load_module.
        """
        self.inst.load_module(self, fullname)

However instead of working like the way I would expect it gives this traceback:

Traceback (most recent call last):
  File "E:\Users\Elsword\Desktop\DecoraterBot\Async\\zipmaker\make_zip.py", line 11, in <module>
    from pathlib import Path
  File "<frozen importlib._bootstrap>", line 961, in _find_and_load
  File "<frozen importlib._bootstrap>", line 946, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 885, in _find_spec
  File "<frozen importlib._bootstrap_external>", line 1157, in find_spec
  File "<frozen importlib._bootstrap_external>", line 1129, in _get_spec
  File "<frozen importlib._bootstrap_external>", line 1273, in find_spec
  File "<frozen importlib._bootstrap_external>", line 1229, in _get_spec
  File "E:\Users\Elsword\Desktop\DecoraterBot\Async\zipmaker\py2pycx\_pycx_backend.py", line 180, in find_loader
    self.inst.find_loader(fullname, path=path)
AttributeError: 'str' object has no attribute 'inst'

Note: it does use a slightly modified make_zip.py that is different than the one that is on the python source trees. It is specifically for testing the hook to see if it works or not.

The hook is supposed to work however it is confusing why it is doing this as I did not set self to be an string so it should allow me to do all of this just fine.

----------
components: Interpreter Core
messages: 285316
nosy: Decorater
priority: normal
severity: normal
status: open
title: self in classes missinterpreted as a string.
versions: Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29252>
_______________________________________

From report at bugs.python.org  Thu Jan 12 07:28:02 2017
From: report at bugs.python.org (Wolfgang Maier)
Date: Thu, 12 Jan 2017 12:28:02 +0000
Subject: [issue18378] locale.getdefaultlocale() fails on Mac OS X with default
 language set to English
In-Reply-To: <1373113143.69.0.513997485887.issue18378@psf.upfronthosting.co.za>
Message-ID: <1484224082.78.0.299990358287.issue18378@psf.upfronthosting.co.za>


Wolfgang Maier added the comment:

To me this issue seems quite related to PEP 538. Maybe the LC_CTYPE coercion proposed in the PEP could be extended to cover the case of LC_CTYPE=UTF-8?

----------
nosy: +ncoghlan
versions: +Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue18378>
_______________________________________

From report at bugs.python.org  Thu Jan 12 07:59:26 2017
From: report at bugs.python.org (Nick Coghlan)
Date: Thu, 12 Jan 2017 12:59:26 +0000
Subject: [issue18378] locale.getdefaultlocale() fails on Mac OS X with default
 language set to English
In-Reply-To: <1373113143.69.0.513997485887.issue18378@psf.upfronthosting.co.za>
Message-ID: <1484225966.43.0.0898306542676.issue18378@psf.upfronthosting.co.za>


Nick Coghlan added the comment:

PEP 538 wouldn't help here, as there's nothing wrong with CPython's assumptions about the text encoding to use for operating system interfaces - it's assuming UTF-8 (because it's Mac OS X) and that assumption is correct (because it's Mac OS X).

The problem appears to be that locale.py was written primarily for Linux, and hence makes assumptions that aren't valid on BSD and Mac OS X.

Dmitry's suggested solution of taking the BSD/Mac OS X specific locale of "UTF-8" and universally accepting it as meaning (None, "UTF-8") sems like a sensible step forward, even if it doesn't resolve all the discrepancies.

Where PEP 538 and PEP 540 would come into play is when this setting gets forwarded over SSH to Linux servers (as then CPython *will* get the nominal system text encoding wrong), but that's independent of getting the locale module to handle it more gracefully.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue18378>
_______________________________________

From report at bugs.python.org  Thu Jan 12 08:12:07 2017
From: report at bugs.python.org (Wolfgang Maier)
Date: Thu, 12 Jan 2017 13:12:07 +0000
Subject: [issue18378] locale.getdefaultlocale() fails on Mac OS X with default
 language set to English
In-Reply-To: <1373113143.69.0.513997485887.issue18378@psf.upfronthosting.co.za>
Message-ID: <1484226727.33.0.0721827351302.issue18378@psf.upfronthosting.co.za>


Wolfgang Maier added the comment:

I think PEP 538 extended to the UTF-8 locale *would* help here. Specifically, it would coerce only LC_CTYPE to en_US.UTF-8 (unless OS X has C.UTF-8), which I guess is good enough for the purpose here.

I do agree that it is not the kind of problem that PEP 538 tries to solve right now, but it could be extended to cover other types of problematic locales like this one. Just wanted to make you aware of this possibility.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue18378>
_______________________________________

From report at bugs.python.org  Thu Jan 12 08:16:23 2017
From: report at bugs.python.org (STINNER Victor)
Date: Thu, 12 Jan 2017 13:16:23 +0000
Subject: [issue29176] /tmp does not exist on Android and is used by
 curses.window.putwin()
In-Reply-To: <1483699620.77.0.827616599041.issue29176@psf.upfronthosting.co.za>
Message-ID: <1484226983.22.0.935847428584.issue29176@psf.upfronthosting.co.za>


STINNER Victor added the comment:

The putwin() function uses mkstemp() with the path template /tmp/py.curses.getwin.XXXXXX.

I would prefer to use the Python function tempfile.mkstemp(). This function has a more portable behaviour. For example, Python is able to atomatically make the file descriptor non-inherirable if the OS supports this feature.

I suggest to expect a file descriptor in the C putwin() and write a Python putwin() which expects a file and pass file.fileno() if available, or use tempfile.TemporaryFile().

tempfile.TemporaryFile is even more secure because the file is not accessible from the regular file system on most platforms. This function is able to use the secure Linux O_TMPFILE flag and the O_TEMPORARY flag on Windows.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29176>
_______________________________________

From report at bugs.python.org  Thu Jan 12 08:16:46 2017
From: report at bugs.python.org (STINNER Victor)
Date: Thu, 12 Jan 2017 13:16:46 +0000
Subject: [issue29176] /tmp does not exist on Android and is used by
 curses.window.putwin()
In-Reply-To: <1483699620.77.0.827616599041.issue29176@psf.upfronthosting.co.za>
Message-ID: <1484227006.69.0.89702809011.issue29176@psf.upfronthosting.co.za>


STINNER Victor added the comment:

I change the issue type to security.

----------
type: behavior -> security

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29176>
_______________________________________

From report at bugs.python.org  Thu Jan 12 08:17:38 2017
From: report at bugs.python.org (STINNER Victor)
Date: Thu, 12 Jan 2017 13:17:38 +0000
Subject: [issue29176] /tmp does not exist on Android and is used by
 curses.window.putwin()
In-Reply-To: <1483699620.77.0.827616599041.issue29176@psf.upfronthosting.co.za>
Message-ID: <1484227058.93.0.896201606321.issue29176@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Oops, I missed Chi Hsuan Yen's comment:
> I guess replace mkstemp (C function) with tempfile.mkstemp (Python function) can solve the problem.

So yes, I agree :-)

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29176>
_______________________________________

From report at bugs.python.org  Thu Jan 12 08:18:12 2017
From: report at bugs.python.org (Berker Peksag)
Date: Thu, 12 Jan 2017 13:18:12 +0000
Subject: [issue29246] typing.Union raises RecursionError when comparing Union
 to other type
In-Reply-To: <1484175780.67.0.919413735976.issue29246@psf.upfronthosting.co.za>
Message-ID: <1484227092.02.0.413323635664.issue29246@psf.upfronthosting.co.za>


Changes by Berker Peksag <berker.peksag at gmail.com>:


----------
nosy: +gvanrossum
stage:  -> patch review

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29246>
_______________________________________

From report at bugs.python.org  Thu Jan 12 08:23:31 2017
From: report at bugs.python.org (Jon Walsh)
Date: Thu, 12 Jan 2017 13:23:31 +0000
Subject: [issue29249] Pathlib glob ** bug
In-Reply-To: <1484216394.04.0.57434547579.issue29249@psf.upfronthosting.co.za>
Message-ID: <1484227411.74.0.773110730581.issue29249@psf.upfronthosting.co.za>


Jon Walsh added the comment:

Seems a bit strange to not have glob() and match() working the same though. Is there any reason for that?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29249>
_______________________________________

From report at bugs.python.org  Thu Jan 12 08:31:39 2017
From: report at bugs.python.org (Xavier de Gaye)
Date: Thu, 12 Jan 2017 13:31:39 +0000
Subject: [issue29185] test_distutils fails on Android API level 24
In-Reply-To: <1483737507.74.0.0285105710282.issue29185@psf.upfronthosting.co.za>
Message-ID: <1484227899.58.0.377665059458.issue29185@psf.upfronthosting.co.za>


Xavier de Gaye added the comment:

can_symlink() is needed by test_posixpath and other tests, new patch.

----------
Added file: http://bugs.python.org/file46271/test_distutils_3.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29185>
_______________________________________

From report at bugs.python.org  Thu Jan 12 08:31:42 2017
From: report at bugs.python.org (STINNER Victor)
Date: Thu, 12 Jan 2017 13:31:42 +0000
Subject: [issue29240] Implementation of the PEP 540: Add a new UTF-8 mode
In-Reply-To: <1484133592.17.0.263024020483.issue29240@psf.upfronthosting.co.za>
Message-ID: <1484227902.22.0.915604327534.issue29240@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Patch version 4:

* Handle PYTHONLEGACYWINDOWSFSENCODING: this env var now disables the UTF-8 mode and has the priority over -X utf8 and PYTHONUTF8
* Add an unit test on PYTHONUTF8 env var and -E cmdline option
* Add an unit test on the POSIX locale
* Fix initstdio() to handle correctly empty PYTHONIOENCODING: this bug affects Python 3.6 as well and is not directly related to the PEP 540
* Fix to handle correctly PYTHONUTF8 set to an empty string (ignore it)
* Skip an unit test in test_utf8mode which failed with the POSIX locale

Note: This patch still has the sys.argv encoding bug with locale encodings different than ASCII and UTF-8.

----------
Added file: http://bugs.python.org/file46270/pep540-4.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29240>
_______________________________________

From report at bugs.python.org  Thu Jan 12 08:51:51 2017
From: report at bugs.python.org (Erik Bray)
Date: Thu, 12 Jan 2017 13:51:51 +0000
Subject: [issue29253] Fix test_asyncore tests on Cygwin
Message-ID: <1484229111.27.0.864294562114.issue29253@psf.upfronthosting.co.za>


New submission from Erik Bray:

This patch works around a couple different problems with running the test_asyncore tests in Cygwin, both of which are due to bugs in Cygwin itself and not in Python.  As such, I don't think Python should try to work around these issues in general, but I do think they're worth at least working around in the tests so that they can pass and/or skip.

The first issue is more severe, as it actually causes the test run to hang indefinitely due to a bug in Cygwin that can cause connect() on a socket to block indefinitely.  This occurs only in a situation where a "client" connects to a "server" that is not accept()-ing connections--a situation that occurs specifically in some of the asyncore tests where both the "client" and the "server" are in the same test process.  This applies a known workaround to the tests only, allowing them to pass.

The other bug simply causes a test failure--the bug is that getsockopt(SO_REUSEADDR) returns the wrong result after a setsockopt(SO_REUSEADDR) (the latter has the correct behavior; the value of the option just isn't reported back correctly after being set).  In this case the relevant test is simply skipped on Cygwin.

The patch is currently light on in-line documentation of the situation, but I can add that if it's otherwise approved.

----------
components: Tests
files: 0001-Fix-test_asyncore-tests-on-Cygwin-by-working-around-.patch
keywords: patch
messages: 285326
nosy: erik.bray, zach.ware
priority: normal
severity: normal
stage: patch review
status: open
title: Fix test_asyncore tests on Cygwin
Added file: http://bugs.python.org/file46272/0001-Fix-test_asyncore-tests-on-Cygwin-by-working-around-.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29253>
_______________________________________

From report at bugs.python.org  Thu Jan 12 08:53:08 2017
From: report at bugs.python.org (Chi Hsuan Yen)
Date: Thu, 12 Jan 2017 13:53:08 +0000
Subject: [issue29082] In 2.7.13,
 _ctypes.LoadLibrary no longer accepts Unicode objects
In-Reply-To: <1482833341.48.0.70408057792.issue29082@psf.upfronthosting.co.za>
Message-ID: <1484229188.45.0.974040157037.issue29082@psf.upfronthosting.co.za>


Chi Hsuan Yen added the comment:

Here's a new patch using fake library names.

And thanks for those related issues about asdl_c.py!

----------
Added file: http://bugs.python.org/file46273/issue29082_3.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29082>
_______________________________________

From report at bugs.python.org  Thu Jan 12 09:16:02 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Thu, 12 Jan 2017 14:16:02 +0000
Subject: [issue29082] In 2.7.13,
 _ctypes.LoadLibrary no longer accepts Unicode objects
In-Reply-To: <1482833341.48.0.70408057792.issue29082@psf.upfronthosting.co.za>
Message-ID: <1484230562.05.0.560988990238.issue29082@psf.upfronthosting.co.za>


Changes by Serhiy Storchaka <storchaka+cpython at gmail.com>:


----------
assignee:  -> serhiy.storchaka

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29082>
_______________________________________

From report at bugs.python.org  Thu Jan 12 09:37:42 2017
From: report at bugs.python.org (Erik Bray)
Date: Thu, 12 Jan 2017 14:37:42 +0000
Subject: [issue25658] PyThread assumes pthread_key_t is an integer,
 which is against POSIX
In-Reply-To: <1447861816.88.0.632584646231.issue25658@psf.upfronthosting.co.za>
Message-ID: <1484231862.17.0.70537177236.issue25658@psf.upfronthosting.co.za>


Erik Bray added the comment:

Thanks Masayuki for the updated patch, and especially for the additional test cases.

Looking at the patch, it occurs to me that this solution seems to be the minimal needed to fix the issue that we were originally trying to fix, without adding too much additional complexity or new semantics to how TLS keys are used in the interpreter.  In other words, this preserves the existing usage with minimal changes except to better support opaque key types.  So I think it's a point in favor of this change that's managed to remain focused.

I'll work on updating the PEP draft to reflect the additions.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue25658>
_______________________________________

From report at bugs.python.org  Thu Jan 12 09:44:47 2017
From: report at bugs.python.org (Marc-Andre Lemburg)
Date: Thu, 12 Jan 2017 14:44:47 +0000
Subject: [issue18378] locale.getdefaultlocale() fails on Mac OS X with default
 language set to English
In-Reply-To: <1373113143.69.0.513997485887.issue18378@psf.upfronthosting.co.za>
Message-ID: <1484232287.67.0.936811293539.issue18378@psf.upfronthosting.co.za>


Marc-Andre Lemburg added the comment:

I think Ronald's patch issue18378-2015-07-25-py36.txt with added darwin check would be the best way forward.

In the current form, it would allow using 'UTF-8' as locale string on all platforms - which is not such a good idea.

----------
nosy: +lemburg

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue18378>
_______________________________________

From report at bugs.python.org  Thu Jan 12 10:00:53 2017
From: report at bugs.python.org (Roundup Robot)
Date: Thu, 12 Jan 2017 15:00:53 +0000
Subject: [issue29082] In 2.7.13,
 _ctypes.LoadLibrary no longer accepts Unicode objects
In-Reply-To: <1482833341.48.0.70408057792.issue29082@psf.upfronthosting.co.za>
Message-ID: <20170112150049.8851.50984.6F8B354D@psf.io>


Roundup Robot added the comment:

New changeset 4ce22d69e134 by Serhiy Storchaka in branch '2.7':
Issue #29082: Fixed loading libraries in ctypes by unicode names on Windows.
https://hg.python.org/cpython/rev/4ce22d69e134

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29082>
_______________________________________

From report at bugs.python.org  Thu Jan 12 10:37:23 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Thu, 12 Jan 2017 15:37:23 +0000
Subject: [issue28969] lru_cache is not threadsafe
In-Reply-To: <1481712564.52.0.530922236321.issue28969@psf.upfronthosting.co.za>
Message-ID: <1484235443.55.0.600944862137.issue28969@psf.upfronthosting.co.za>


Changes by Serhiy Storchaka <storchaka+cpython at gmail.com>:


----------
assignee: rhettinger -> serhiy.storchaka
stage: patch review -> commit review

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28969>
_______________________________________

From report at bugs.python.org  Thu Jan 12 11:42:13 2017
From: report at bugs.python.org (Suraj Kumar)
Date: Thu, 12 Jan 2017 16:42:13 +0000
Subject: [issue29254] Documentation Error
Message-ID: <1484239333.41.0.299474192201.issue29254@psf.upfronthosting.co.za>


New submission from Suraj Kumar:

Hi,

I have found one documentation mistake in section 3.1.2. Strings saying
"""Note how the start is always included, and the end always excluded. This makes sure that s[:i] + s[i:] is always equal to s:
>>>word = 'Python'
>>> word[:2] + word[2:]
'Python'
"""

I think instead of 'the end always excluded' it should be 'the end always included'.

If my observation is correct, please do the needful change.

Thanks & Regards,
Suraj

----------
assignee: docs at python
components: Documentation
messages: 285331
nosy: Suraj Kumar, docs at python
priority: normal
severity: normal
status: open
title: Documentation Error
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29254>
_______________________________________

From report at bugs.python.org  Thu Jan 12 11:45:20 2017
From: report at bugs.python.org (STINNER Victor)
Date: Thu, 12 Jan 2017 16:45:20 +0000
Subject: [issue29240] Implementation of the PEP 540: Add a new UTF-8 mode
In-Reply-To: <1484133592.17.0.263024020483.issue29240@psf.upfronthosting.co.za>
Message-ID: <1484239520.04.0.437867803089.issue29240@psf.upfronthosting.co.za>


STINNER Victor added the comment:

encodings.py: enhancement version of pep540_cli.py, add locale and filesystem encoding. Script to test the implementation of the PEP 540 (and PEP 538).

----------
Added file: http://bugs.python.org/file46274/encodings.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29240>
_______________________________________

From report at bugs.python.org  Thu Jan 12 12:04:49 2017
From: report at bugs.python.org (Ammar Askar)
Date: Thu, 12 Jan 2017 17:04:49 +0000
Subject: [issue29254] Documentation Error
In-Reply-To: <1484239333.41.0.299474192201.issue29254@psf.upfronthosting.co.za>
Message-ID: <1484240689.25.0.8791147234.issue29254@psf.upfronthosting.co.za>


Ammar Askar added the comment:

Given `slice[start:end]`:

What this line is trying to say is that the slice index is inclusive for the start parameter and exclusive for the end parameter.

In mathematical/interval notation this would look like [start, end)

As a quick example look at the word "Python"
Index  012345
Letter Python

If you do word[:2], the result is 'Py', notice how the t is excluded.

----------
nosy: +ammar2
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29254>
_______________________________________

From report at bugs.python.org  Thu Jan 12 12:13:54 2017
From: report at bugs.python.org (Greg Bengeult)
Date: Thu, 12 Jan 2017 17:13:54 +0000
Subject: [issue11681] -b option undocumented
In-Reply-To: <1301100825.87.0.69977993755.issue11681@psf.upfronthosting.co.za>
Message-ID: <1484241234.52.0.285488378364.issue11681@psf.upfronthosting.co.za>


Greg Bengeult added the comment:

Thanks for the :class: markup edits. I'm still learning this stuff.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue11681>
_______________________________________

From report at bugs.python.org  Thu Jan 12 12:20:46 2017
From: report at bugs.python.org (Vinay Sajip)
Date: Thu, 12 Jan 2017 17:20:46 +0000
Subject: [issue22343] Install bash activate script on Windows when using venv
In-Reply-To: <1409948648.7.0.892236136888.issue22343@psf.upfronthosting.co.za>
Message-ID: <1484241646.81.0.760351384047.issue22343@psf.upfronthosting.co.za>


Vinay Sajip added the comment:

I've pushed the changes, but due to some SMTP problem this issue wasn't updated by Roundup:

remote: SMTPServerDisconnected: Connection unexpectedly closed: timed out
remote: error: changegroup.roundup hook raised an exception: Connection unexpectedly closed: timed out

However, the changes appear to be in the repository. Interested parties might want to pull the changes and test things out on Windows so that the issue can be closed.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue22343>
_______________________________________

From report at bugs.python.org  Thu Jan 12 12:22:45 2017
From: report at bugs.python.org (Vinay Sajip)
Date: Thu, 12 Jan 2017 17:22:45 +0000
Subject: [issue22343] Install bash activate script on Windows when using venv
In-Reply-To: <1409948648.7.0.892236136888.issue22343@psf.upfronthosting.co.za>
Message-ID: <1484241765.38.0.906477156448.issue22343@psf.upfronthosting.co.za>


Vinay Sajip added the comment:

Changesets:

3.5     d6ae934da199
3.6     8f77c4f93806
default 7afe82ad34e7

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue22343>
_______________________________________

From report at bugs.python.org  Thu Jan 12 12:55:03 2017
From: report at bugs.python.org (Jamie Bliss)
Date: Thu, 12 Jan 2017 17:55:03 +0000
Subject: [issue29221] ABC Recursion Error on isinstance() with less than
 recursion limit class hierarchy depth
In-Reply-To: <1484021927.49.0.387886726852.issue29221@psf.upfronthosting.co.za>
Message-ID: <1484243702.99.0.45076991699.issue29221@psf.upfronthosting.co.za>


Changes by Jamie Bliss <astronouth7303 at gmail.com>:


----------
nosy: +astronouth7303

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29221>
_______________________________________

From report at bugs.python.org  Thu Jan 12 12:56:27 2017
From: report at bugs.python.org (Greg Bengeult)
Date: Thu, 12 Jan 2017 17:56:27 +0000
Subject: [issue28130] Document that time.tzset updates time module globals
In-Reply-To: <1473782506.71.0.242774514927.issue28130@psf.upfronthosting.co.za>
Message-ID: <1484243787.89.0.287957486674.issue28130@psf.upfronthosting.co.za>


Greg Bengeult added the comment:

How about this documentation patch for tzset()? Most of the verbiage comes from the Linux tzset() man page. I can also generate a similar patch for 2.7 if it's needed.

----------
nosy: +gbengeult
versions: +Python 3.7
Added file: http://bugs.python.org/file46275/tzset_rst.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28130>
_______________________________________

From report at bugs.python.org  Thu Jan 12 13:03:41 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Thu, 12 Jan 2017 18:03:41 +0000
Subject: [issue28969] lru_cache is not threadsafe
In-Reply-To: <1481712564.52.0.530922236321.issue28969@psf.upfronthosting.co.za>
Message-ID: <1484244221.55.0.798334574013.issue28969@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

Thanks Inada.

Committed in 9314aebc9674, 5724c3ddf25b and 9c852878a998. There were errors during pushing:

remote: adding changesets
remote: adding manifests
remote: adding file changes
remote: added 9 changesets with 15 changes to 5 files
remote: buildbot: change(s) sent successfully
remote: Traceback (most recent call last):
remote:   File "/srv/hg/repos/hooks/hgroundup.py", line 56, in update_issue
remote:     _update_issue(*args, **kwargs)
remote:   File "/srv/hg/repos/hooks/hgroundup.py", line 108, in _update_issue
remote:     s.login(username, password)
remote:   File "/usr/lib/python2.7/smtplib.py", line 610, in login
remote:     AUTH_PLAIN + " " + encode_plain(user, password))
remote:   File "/usr/lib/python2.7/smtplib.py", line 394, in docmd
remote:     return self.getreply()
remote:   File "/usr/lib/python2.7/smtplib.py", line 365, in getreply
remote:     + str(e))
remote: SMTPServerDisconnected: Connection unexpectedly closed: timed out
remote: error: changegroup.roundup hook raised an exception: Connection unexpectedly closed: timed out
remote: notified python-checkins at python.org of incoming changeset 9314aebc9674

remote: Traceback (most recent call last):
remote:   File "/srv/hg/repos/hooks/mail.py", line 166, in incoming
remote:     return _incoming(ui, repo, **kwargs)
remote:   File "/srv/hg/repos/hooks/mail.py", line 157, in _incoming
remote:     send(smtp, subj, sender, to, '\n'.join(body) + '\n')
remote:   File "/srv/hg/repos/hooks/mail.py", line 37, in send
remote:     smtp.sendmail(sender, to, msg.as_string())
remote:   File "/usr/lib/python2.7/smtplib.py", line 748, in sendmail
remote:     (code, resp) = self.data(msg)
remote:   File "/usr/lib/python2.7/smtplib.py", line 511, in data
remote:     (code, msg) = self.getreply()
remote:   File "/usr/lib/python2.7/smtplib.py", line 365, in getreply
remote:     + str(e))
remote: SMTPServerDisconnected: Connection unexpectedly closed: timed out
remote: error: incoming.notify hook raised an exception: Connection unexpectedly closed: timed out
...

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28969>
_______________________________________

From report at bugs.python.org  Thu Jan 12 13:14:40 2017
From: report at bugs.python.org (Guido van Rossum)
Date: Thu, 12 Jan 2017 18:14:40 +0000
Subject: [issue29246] typing.Union raises RecursionError when comparing Union
 to other type
In-Reply-To: <1484175780.67.0.919413735976.issue29246@psf.upfronthosting.co.za>
Message-ID: <1484244880.95.0.51649598623.issue29246@psf.upfronthosting.co.za>


Changes by Guido van Rossum <guido at python.org>:


----------
assignee:  -> levkivskyi
nosy: +levkivskyi

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29246>
_______________________________________

From report at bugs.python.org  Thu Jan 12 13:16:52 2017
From: report at bugs.python.org (desbma)
Date: Thu, 12 Jan 2017 18:16:52 +0000
Subject: [issue29212] Python 3.6 logging thread name regression with
 concurrent.future threads
In-Reply-To: <1483938963.09.0.315595404976.issue29212@psf.upfronthosting.co.za>
Message-ID: <1484245012.31.0.852875559143.issue29212@psf.upfronthosting.co.za>


desbma added the comment:

Thoughts anyone?

This is a minor bug, but for a common use case (easy to hit), and the fix is trivial.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29212>
_______________________________________

From report at bugs.python.org  Thu Jan 12 13:24:40 2017
From: report at bugs.python.org (Alexander Belopolsky)
Date: Thu, 12 Jan 2017 18:24:40 +0000
Subject: [issue28130] Document that time.tzset updates time module globals
In-Reply-To: <1473782506.71.0.242774514927.issue28130@psf.upfronthosting.co.za>
Message-ID: <1484245480.97.0.192208541081.issue28130@psf.upfronthosting.co.za>


Alexander Belopolsky added the comment:

Committed in ec3b08b361c0.  Thanks, Greg.

Leaving open until 2.7 docs are fixed as well.

----------
resolution:  -> fixed
stage:  -> resolved
versions: +Python 2.7 -Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28130>
_______________________________________

From report at bugs.python.org  Thu Jan 12 13:27:29 2017
From: report at bugs.python.org (Roundup Robot)
Date: Thu, 12 Jan 2017 18:27:29 +0000
Subject: [issue28130] Document that time.tzset updates time module globals
In-Reply-To: <1473782506.71.0.242774514927.issue28130@psf.upfronthosting.co.za>
Message-ID: <20170112182725.96191.57437.A022A36E@psf.io>


Roundup Robot added the comment:

New changeset ec3b08b361c0 by Alexander Belopolsky in branch 'default':
Closes #28130: Documented that time.tzset() updates time module globals.
https://hg.python.org/cpython/rev/ec3b08b361c0

----------
nosy: +python-dev
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28130>
_______________________________________

From report at bugs.python.org  Thu Jan 12 13:33:01 2017
From: report at bugs.python.org (Alexander Belopolsky)
Date: Thu, 12 Jan 2017 18:33:01 +0000
Subject: [issue28130] Document that time.tzset updates time module globals
In-Reply-To: <1473782506.71.0.242774514927.issue28130@psf.upfronthosting.co.za>
Message-ID: <1484245981.59.0.0196306314739.issue28130@psf.upfronthosting.co.za>


Changes by Alexander Belopolsky <alexander.belopolsky at gmail.com>:


----------
status: closed -> open

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28130>
_______________________________________

From report at bugs.python.org  Thu Jan 12 13:59:07 2017
From: report at bugs.python.org (Jamie Bliss)
Date: Thu, 12 Jan 2017 18:59:07 +0000
Subject: [issue29221] ABC Recursion Error on isinstance() with less than
 recursion limit class hierarchy depth
In-Reply-To: <1484021927.49.0.387886726852.issue29221@psf.upfronthosting.co.za>
Message-ID: <1484247547.34.0.717572745706.issue29221@psf.upfronthosting.co.za>


Jamie Bliss added the comment:

I consider it debatable as to if this is a bug in ABC (for using recursion in a way that limits class tree depth) or in CPython (for accounting recursion in such a way you can get significantly shallower stacks).

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29221>
_______________________________________

From report at bugs.python.org  Thu Jan 12 14:38:55 2017
From: report at bugs.python.org (Greg Bengeult)
Date: Thu, 12 Jan 2017 19:38:55 +0000
Subject: [issue28130] Document that time.tzset updates time module globals
In-Reply-To: <1473782506.71.0.242774514927.issue28130@psf.upfronthosting.co.za>
Message-ID: <1484249935.22.0.406829764042.issue28130@psf.upfronthosting.co.za>


Greg Bengeult added the comment:

Here's the 2.7 patch.

----------
Added file: http://bugs.python.org/file46276/tzset_rst_27.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28130>
_______________________________________

From report at bugs.python.org  Thu Jan 12 14:43:25 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Thu, 12 Jan 2017 19:43:25 +0000
Subject: [issue29082] In 2.7.13,
 _ctypes.LoadLibrary no longer accepts Unicode objects
In-Reply-To: <1482833341.48.0.70408057792.issue29082@psf.upfronthosting.co.za>
Message-ID: <1484250205.6.0.186292171311.issue29082@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

Committed with changed test. Thank you for your patch Chi Hsuan Yen.

----------
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29082>
_______________________________________

From report at bugs.python.org  Thu Jan 12 14:44:03 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Thu, 12 Jan 2017 19:44:03 +0000
Subject: [issue28969] lru_cache is not threadsafe
In-Reply-To: <1481712564.52.0.530922236321.issue28969@psf.upfronthosting.co.za>
Message-ID: <1484250243.01.0.681186921996.issue28969@psf.upfronthosting.co.za>


Changes by Serhiy Storchaka <storchaka+cpython at gmail.com>:


----------
resolution:  -> fixed
stage: commit review -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28969>
_______________________________________

From report at bugs.python.org  Thu Jan 12 14:44:49 2017
From: report at bugs.python.org (Thomas Nyberg)
Date: Thu, 12 Jan 2017 19:44:49 +0000
Subject: =?utf-8?q?=5Bissue24875=5D_pyvenv_doesn=C2=B4t_install_PIP_inside_a_new_v?=
 =?utf-8?q?env_with_--system-site-package?=
In-Reply-To: <1439699040.07.0.199956094974.issue24875@psf.upfronthosting.co.za>
Message-ID: <1484250289.95.0.873617505325.issue24875@psf.upfronthosting.co.za>


Changes by Thomas Nyberg <tomnyberg at gmail.com>:


Removed file: http://bugs.python.org/file46250/venv_site_packages.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24875>
_______________________________________

From report at bugs.python.org  Thu Jan 12 15:06:45 2017
From: report at bugs.python.org (Roundup Robot)
Date: Thu, 12 Jan 2017 20:06:45 +0000
Subject: [issue28130] Document that time.tzset updates time module globals
In-Reply-To: <1473782506.71.0.242774514927.issue28130@psf.upfronthosting.co.za>
Message-ID: <20170112200642.96977.65371.E7147C6E@psf.io>


Roundup Robot added the comment:

New changeset 8578757cc3a6 by Alexander Belopolsky in branch '2.7':
Closes #28130: Documented that time.tzset() updates time module globals.
https://hg.python.org/cpython/rev/8578757cc3a6

----------
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28130>
_______________________________________

From report at bugs.python.org  Thu Jan 12 15:36:52 2017
From: report at bugs.python.org (Vinay Sajip)
Date: Thu, 12 Jan 2017 20:36:52 +0000
Subject: =?utf-8?q?=5Bissue24875=5D_pyvenv_doesn=C2=B4t_install_PIP_inside_a_new_v?=
 =?utf-8?q?env_with_--system-site-package?=
In-Reply-To: <1439699040.07.0.199956094974.issue24875@psf.upfronthosting.co.za>
Message-ID: <1484253412.85.0.83434365588.issue24875@psf.upfronthosting.co.za>


Vinay Sajip added the comment:

It seems simpler to do it all in EnvBuilder.create:

    def create(self, env_dir):
        """
        Create a virtual environment in a directory.

        :param env_dir: The target directory to create an environment in.

        """
        env_dir = os.path.abspath(env_dir)
        context = self.ensure_directories(env_dir)
        # See issue 24875. We need system_site_packages to be False
        # until after pip is installed.
        true_system_site_packages = self.system_site_packages
        self.system_site_packages = False
        self.create_configuration(context)
        self.setup_python(context)
        if self.with_pip:
            self._setup_pip(context)
        if not self.upgrade:
            self.setup_scripts(context)
            self.post_setup(context)
        if true_system_site_packages:
            # We had set it to False before, now
            # restore it and rewrite the configuration
            self.system_site_packages = True
            self.create_configuration(context)

Can you patch that in and see if it works for you?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24875>
_______________________________________

From report at bugs.python.org  Thu Jan 12 15:39:48 2017
From: report at bugs.python.org (Decorater)
Date: Thu, 12 Jan 2017 20:39:48 +0000
Subject: [issue29252] self in classes missinterpreted as a string.
In-Reply-To: <1484224047.15.0.809127294576.issue29252@psf.upfronthosting.co.za>
Message-ID: <1484253588.73.0.865591059703.issue29252@psf.upfronthosting.co.za>


Decorater added the comment:

Also I sadly have to make this a dummy class just to add in an actual dictionary with the file extensions it supports because I cant use setattr on it. It is retarded that you cant use it for builtin types or classes if your code actually requires such.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29252>
_______________________________________

From report at bugs.python.org  Thu Jan 12 15:41:15 2017
From: report at bugs.python.org (Decorater)
Date: Thu, 12 Jan 2017 20:41:15 +0000
Subject: [issue29252] self in classes missinterpreted as a string.
In-Reply-To: <1484224047.15.0.809127294576.issue29252@psf.upfronthosting.co.za>
Message-ID: <1484253675.81.0.761452538817.issue29252@psf.upfronthosting.co.za>


Decorater added the comment:

Also in the code I had to subclass it 3 times, 1 time to preserver original functionality as if the hook was not inserted in the front of the path hooks, another time for 1 format of files it should be able to allow importing of, and the other time for another format of files.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29252>
_______________________________________

From report at bugs.python.org  Thu Jan 12 16:01:10 2017
From: report at bugs.python.org (Mark Lawrence)
Date: Thu, 12 Jan 2017 21:01:10 +0000
Subject: [issue17824] pty.spawn handles errors improperly
In-Reply-To: <1366772370.17.0.184801312009.issue17824@psf.upfronthosting.co.za>
Message-ID: <1484254870.1.0.833939540013.issue17824@psf.upfronthosting.co.za>


Changes by Mark Lawrence <breamoreboy at gmail.com>:


----------
nosy:  -BreamoreBoy

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue17824>
_______________________________________

From report at bugs.python.org  Thu Jan 12 16:11:29 2017
From: report at bugs.python.org (Thomas Nyberg)
Date: Thu, 12 Jan 2017 21:11:29 +0000
Subject: =?utf-8?q?=5Bissue24875=5D_pyvenv_doesn=C2=B4t_install_PIP_inside_a_new_v?=
 =?utf-8?q?env_with_--system-site-package?=
In-Reply-To: <1439699040.07.0.199956094974.issue24875@psf.upfronthosting.co.za>
Message-ID: <1484255488.99.0.328282214103.issue24875@psf.upfronthosting.co.za>


Thomas Nyberg added the comment:

Hi Vinay,

I just checked and yes that code works fine on my end (debian 8 box). Personally, I think either this or Mark's second patch would be fine.

Cheers,
Thomas

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24875>
_______________________________________

From report at bugs.python.org  Thu Jan 12 16:50:19 2017
From: report at bugs.python.org (Ivan Levkivskyi)
Date: Thu, 12 Jan 2017 21:50:19 +0000
Subject: [issue29246] typing.Union raises RecursionError when comparing Union
 to other type
In-Reply-To: <1484175780.67.0.919413735976.issue29246@psf.upfronthosting.co.za>
Message-ID: <1484257819.61.0.981842773249.issue29246@psf.upfronthosting.co.za>


Ivan Levkivskyi added the comment:

Thank you for catching this corner case!

I have made a PR upstream based on your patch: https://github.com/python/typing/pull/350

As I understand the fix will go in 3.6.1 and 3.5.4

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29246>
_______________________________________

From report at bugs.python.org  Thu Jan 12 18:00:38 2017
From: report at bugs.python.org (Craig Holmquist)
Date: Thu, 12 Jan 2017 23:00:38 +0000
Subject: [issue23407] os.walk always follows Windows junctions
In-Reply-To: <1423342763.81.0.179122512075.issue23407@psf.upfronthosting.co.za>
Message-ID: <1484262038.56.0.113337551266.issue23407@psf.upfronthosting.co.za>


Craig Holmquist added the comment:

Can you point me toward any documentation on the additional tags you want to support?  Searching for IO_REPARSE_TAG_IIS_CACHE mostly seems to yield header files that define it (and nothing at all on MSDN), and the non-Microsoft tags just yield a few results each.

(For comparison, the junction and symbolic link tags yield 10K+ results each.)

Junctions are created with each user's home directory so they exist on every Windows system, even if the user never explicitly creates them.  The additional tags seem like they're far less common and much less well-documented.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23407>
_______________________________________

From report at bugs.python.org  Thu Jan 12 18:35:07 2017
From: report at bugs.python.org (Nathaniel Smith)
Date: Thu, 12 Jan 2017 23:35:07 +0000
Subject: [issue29255] selects.KqueueSelector behaves incorrectly when no fds
 are registered
Message-ID: <1484264107.13.0.0489042381572.issue29255@psf.upfronthosting.co.za>


New submission from Nathaniel Smith:

When calling kevent(), selectors.KqueueSelector.select sets the "maxevents" argument to len(self._fd_to_key). So if there are no fds registered, it passes maxevents=0.

It turns out that the kevent() API has a special case behavior for maxevents=0: it returns immediately, ignoring timeout. I have no idea why kevent() works this way, but it's specifically called out in the man page:

   The nevents argument determines the size of eventlist.  When
   nevents is zero, kevent() will return immediately even if there is a
   timeout specified unlike select(2).

https://www.freebsd.org/cgi/man.cgi?query=kqueue&sektion=2

It happens that asyncio doesn't run into this because asyncio always has at least one fd registered with its selector, but this causes problems for other users of the selectors module, e.g.:
  https://github.com/dabeaz/curio/issues/156

I believe fix would just be to add some code like: "if max_ev == 0: max_ev = 1" to selectors.KqueueSelector.select.

----------
components: Library (Lib)
messages: 285352
nosy: njs
priority: normal
severity: normal
status: open
title: selects.KqueueSelector behaves incorrectly when no fds are registered
versions: Python 3.5, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29255>
_______________________________________

From report at bugs.python.org  Thu Jan 12 18:42:18 2017
From: report at bugs.python.org (Nathaniel Smith)
Date: Thu, 12 Jan 2017 23:42:18 +0000
Subject: [issue29256] Windows select() errors out when given no fds to select
 on, which breaks SelectSelector
Message-ID: <1484264538.85.0.902649207203.issue29256@psf.upfronthosting.co.za>


New submission from Nathaniel Smith:

If SelectSelector.select() is called when there are no fds registered, then it ends up calling select.select([], [], [], timeout).

On sensible operating systems, this is equivalent to time.sleep(timeout). On Windows, it raises an error. Asyncio manages to avoid hitting this due to the fact that it always has at least one fd registered, but it causes problems for other users of the selectors module, e.g.:
  https://github.com/dabeaz/curio/issues/75

I see two possible approaches to fixing this:

1) Modify SelectSelector to check for this case and call time.sleep() instead of select.select() when encountered.

2) Modify the select.select() wrapper so that it behaves consistently on all operating systems, by special-casing this situation on Windows.

Option (2) seems nicer to me.

----------
components: Library (Lib)
messages: 285353
nosy: njs
priority: normal
severity: normal
status: open
title: Windows select() errors out when given no fds to select on, which breaks SelectSelector
versions: Python 3.5, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29256>
_______________________________________

From report at bugs.python.org  Thu Jan 12 18:49:59 2017
From: report at bugs.python.org (Steve Dower)
Date: Thu, 12 Jan 2017 23:49:59 +0000
Subject: [issue29241] sys._enablelegacywindowsfsencoding() don't apply to
 os.fsencode and os.fsdecode
In-Reply-To: <1484137681.59.0.0301192140292.issue29241@psf.upfronthosting.co.za>
Message-ID: <1484264999.29.0.886435708973.issue29241@psf.upfronthosting.co.za>


Steve Dower added the comment:

Then we do in fact need to make os.fsencode/fsdecode either stop caching the encoding completely, or figure out a way to reset the cache when that function is called.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29241>
_______________________________________

From report at bugs.python.org  Thu Jan 12 19:23:43 2017
From: report at bugs.python.org (Ryan Hiebert)
Date: Fri, 13 Jan 2017 00:23:43 +0000
Subject: [issue11339] annotation for class being defined
In-Reply-To: <1298751820.01.0.369994267539.issue11339@psf.upfronthosting.co.za>
Message-ID: <1484267023.79.0.0711122438511.issue11339@psf.upfronthosting.co.za>


Changes by Ryan Hiebert <ryan at ryanhiebert.com>:


----------
nosy: +ryanhiebert

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue11339>
_______________________________________

From report at bugs.python.org  Thu Jan 12 19:38:24 2017
From: report at bugs.python.org (Eryk Sun)
Date: Fri, 13 Jan 2017 00:38:24 +0000
Subject: [issue23407] os.walk always follows Windows junctions
In-Reply-To: <1423342763.81.0.179122512075.issue23407@psf.upfronthosting.co.za>
Message-ID: <1484267904.97.0.473961050893.issue23407@psf.upfronthosting.co.za>


Eryk Sun added the comment:

I simply listed the tags that have the name-surrogate bit set out of those defined in km\ntifs.h. 

To keeps things simple it might be better to only include Microsoft tags (i.e. bit 31 is set). That way we don't have to deal with REPARSE_GUID_DATA_BUFFER struct that's used from non-Microsoft reparse points.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23407>
_______________________________________

From report at bugs.python.org  Thu Jan 12 19:48:36 2017
From: report at bugs.python.org (Gerald Britton)
Date: Fri, 13 Jan 2017 00:48:36 +0000
Subject: [issue29257] Possible error in discussion of Abstract Base Classes
 and abstract properties
Message-ID: <1484268516.3.0.750896526181.issue29257@psf.upfronthosting.co.za>


New submission from Gerald Britton:

I was rereading the 2.7 docs  about abstract base classes the other day.  I found this:

"This defines a read-only property; you can also define a read-write abstract property using the ?long? form of property declaration:"

along with an example.  so I copied the example and put in a little surrounding code:


from abc import ABCMeta, abstractproperty

class C:
    __metaclass__ = ABCMeta
    def getx(self): pass
    def setx(self, value): pass
    x = abstractproperty(getx, setx)
 
class D(C):
    @property
    def x(self):self._x
 
d = D()    
print(d)   

When I ran this, I expected an exception, since I defined a read/write abstract property but only implemented the read operation.  However, the example runs fine. That is the class D can be instantiated without error.  Of course I cannot set the property since I didn't implement that part.

Now, If I don't implement the property at all, I can' instantiate the class.  I get:

"TypeError: Can't instantiate abstract class D with abstract methods x"

which is what I would expect.  What I don't understand is why I don't get a similar error when I implement the read operation for the property but not the write operation.

If this actually doesn't work (catching the non-implementation at instantiation time), then why is it documented this way?  To me at least the doc implies that it *will* raise on the missing write property implementation.

If ABCs are working as intended, can the documentation be changed to reflect that as per my experience above?  If the documentation is correct, can the ABC implementation be modified to function that way?

----------
assignee: docs at python
components: Documentation
messages: 285356
nosy: docs at python, gbritton
priority: normal
severity: normal
status: open
title: Possible error in discussion of Abstract Base Classes and abstract properties
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29257>
_______________________________________

From report at bugs.python.org  Thu Jan 12 19:54:08 2017
From: report at bugs.python.org (INADA Naoki)
Date: Fri, 13 Jan 2017 00:54:08 +0000
Subject: [issue29240] Implementation of the PEP 540: Add a new UTF-8 mode
In-Reply-To: <1484133592.17.0.263024020483.issue29240@psf.upfronthosting.co.za>
Message-ID: <1484268848.22.0.563912738523.issue29240@psf.upfronthosting.co.za>


INADA Naoki added the comment:

How about locale.getpreferredencoding() returns 'utf-8' in utf8 mode?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29240>
_______________________________________

From report at bugs.python.org  Thu Jan 12 20:26:57 2017
From: report at bugs.python.org (INADA Naoki)
Date: Fri, 13 Jan 2017 01:26:57 +0000
Subject: [issue29062] hashlib documentation link error
In-Reply-To: <1482571303.3.0.781602557285.issue29062@psf.upfronthosting.co.za>
Message-ID: <1484270817.68.0.532574592541.issue29062@psf.upfronthosting.co.za>


INADA Naoki added the comment:

@christian.heimes Would you look merge-hashlib-blake2.patch?
This is blocker of our document translation project.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29062>
_______________________________________

From report at bugs.python.org  Thu Jan 12 21:41:34 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Fri, 13 Jan 2017 02:41:34 +0000
Subject: [issue29062] hashlib documentation link error
In-Reply-To: <1482571303.3.0.781602557285.issue29062@psf.upfronthosting.co.za>
Message-ID: <1484275294.7.0.266741937665.issue29062@psf.upfronthosting.co.za>


Xiang Zhang added the comment:

Although I am +1 for merging it before but these days I find it's not the only case. There are also two unittest.mock entries. This is just FYI, no opinion on this issue.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29062>
_______________________________________

From report at bugs.python.org  Thu Jan 12 22:47:58 2017
From: report at bugs.python.org (Nick Coghlan)
Date: Fri, 13 Jan 2017 03:47:58 +0000
Subject: [issue18378] locale.getdefaultlocale() fails on Mac OS X with default
 language set to English
In-Reply-To: <1373113143.69.0.513997485887.issue18378@psf.upfronthosting.co.za>
Message-ID: <1484279278.87.0.773654036758.issue18378@psf.upfronthosting.co.za>


Nick Coghlan added the comment:

SSH environment forwarding will propagate this "LC_CTYPE=UTF-8" setting from Mac OS X clients to Linux servers.

At present, that breaks in multiple ways, as CPython will interpret it as being the "C" locale (since Linux servers don't offer a "UTF-8" locale, even when they do offer "C.UTF-8")

PEPs 538 and 540 aim to help CPython itself to deal with that case, but that won't be sufficient to help code that tries to pass the nominal LC_CTYPE setting to the locale module.

Accepting "UTF-8" and interpreting it as functionally equivalent to C.UTF-8 will mean that this setting will at least work as desired on servers that offer C.UTF-8.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue18378>
_______________________________________

From report at bugs.python.org  Thu Jan 12 23:47:54 2017
From: report at bugs.python.org (Anthony Sottile)
Date: Fri, 13 Jan 2017 04:47:54 +0000
Subject: [issue29258] __init__.py required for pkgutil.walk_packages in python3
Message-ID: <1484282874.75.0.69511180509.issue29258@psf.upfronthosting.co.za>


New submission from Anthony Sottile:

PEP420 makes __init__.py files optional: https://docs.python.org/3/whatsnew/3.3.html#pep-420-implicit-namespace-packages

Though it seems without them, pkgutil.walk_packages does not function as desired: https://docs.python.org/3/library/pkgutil.html#pkgutil.walk_packages

Consider the following example:

```
$ tree foo
foo
??? bar
?   ??? baz.py
?   ??? __init__.py
??? __init__.py
??? womp.py
```

And a test script

# test.py
```
import pkgutil

import foo


for _, mod, _ in pkgutil.walk_packages(foo.__path__, foo.__name__ + '.'):
    print(mod)
```

In both python2 and python3 I get the following output:

```
$ python2.7 test.py
foo.bar
foo.bar.baz
foo.womp
$ python3.5 test.py
foo.bar
foo.bar.baz
foo.womp
```

Removing the __init__.py files and only using python3, I get this:

```
$ find -name '__init__.*' -delete
$ python3.5 test.py
foo.bar
```

The modules are definitely importable:

```
$ python3.5 -c 'import foo.bar.baz'
$
```

Originally asked as a question on stackoverflow: http://stackoverflow.com/questions/41203765/init-py-required-for-pkgutil-walk-packages-in-python3

----------
components: Library (Lib)
messages: 285361
nosy: Anthony Sottile
priority: normal
severity: normal
status: open
title: __init__.py required for pkgutil.walk_packages in python3
type: behavior
versions: Python 3.3, Python 3.4, Python 3.5, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29258>
_______________________________________

From report at bugs.python.org  Fri Jan 13 01:54:18 2017
From: report at bugs.python.org (Roundup Robot)
Date: Fri, 13 Jan 2017 06:54:18 +0000
Subject: [issue29195] Get rid of supporting outdated wrong keyword arguments
 in re methods
In-Reply-To: <1483790693.93.0.397432841389.issue29195@psf.upfronthosting.co.za>
Message-ID: <20170113065414.3622.19075.68F15123@psf.io>


Roundup Robot added the comment:

New changeset f8678135c042 by Serhiy Storchaka in branch 'default':
Issue #29195: Removed support of deprecated undocumented keyword arguments
https://hg.python.org/cpython/rev/f8678135c042

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29195>
_______________________________________

From report at bugs.python.org  Fri Jan 13 02:02:34 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Fri, 13 Jan 2017 07:02:34 +0000
Subject: [issue29195] Get rid of supporting outdated wrong keyword arguments
 in re methods
In-Reply-To: <1483790693.93.0.397432841389.issue29195@psf.upfronthosting.co.za>
Message-ID: <1484290954.63.0.609789997979.issue29195@psf.upfronthosting.co.za>


Changes by Serhiy Storchaka <storchaka+cpython at gmail.com>:


----------
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29195>
_______________________________________

From report at bugs.python.org  Fri Jan 13 02:06:33 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Fri, 13 Jan 2017 07:06:33 +0000
Subject: [issue29193] Remove support of format_string as keyword argument in
 string.Formatter().format()
In-Reply-To: <1483788911.55.0.222681922661.issue29193@psf.upfronthosting.co.za>
Message-ID: <1484291193.2.0.88000359003.issue29193@psf.upfronthosting.co.za>


Changes by Serhiy Storchaka <storchaka+cpython at gmail.com>:


----------
assignee:  -> serhiy.storchaka

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29193>
_______________________________________

From report at bugs.python.org  Fri Jan 13 02:11:07 2017
From: report at bugs.python.org (Roundup Robot)
Date: Fri, 13 Jan 2017 07:11:07 +0000
Subject: [issue29193] Remove support of format_string as keyword argument in
 string.Formatter().format()
In-Reply-To: <1483788911.55.0.222681922661.issue29193@psf.upfronthosting.co.za>
Message-ID: <20170113071104.3215.54225.80595427@psf.io>


Roundup Robot added the comment:

New changeset d571d8cd4258 by Serhiy Storchaka in branch 'default':
Issue #29193: A format string argument for string.Formatter.format()
https://hg.python.org/cpython/rev/d571d8cd4258

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29193>
_______________________________________

From report at bugs.python.org  Fri Jan 13 02:11:32 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Fri, 13 Jan 2017 07:11:32 +0000
Subject: [issue29193] Remove support of format_string as keyword argument in
 string.Formatter().format()
In-Reply-To: <1483788911.55.0.222681922661.issue29193@psf.upfronthosting.co.za>
Message-ID: <1484291492.56.0.99210507602.issue29193@psf.upfronthosting.co.za>


Changes by Serhiy Storchaka <storchaka+cpython at gmail.com>:


----------
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29193>
_______________________________________

From report at bugs.python.org  Fri Jan 13 02:19:24 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Fri, 13 Jan 2017 07:19:24 +0000
Subject: [issue29192] Remove deprecated features from http.cookies
In-Reply-To: <1483787527.13.0.320164851178.issue29192@psf.upfronthosting.co.za>
Message-ID: <1484291964.58.0.482134848178.issue29192@psf.upfronthosting.co.za>


Changes by Serhiy Storchaka <storchaka+cpython at gmail.com>:


----------
assignee:  -> serhiy.storchaka

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29192>
_______________________________________

From report at bugs.python.org  Fri Jan 13 02:23:30 2017
From: report at bugs.python.org (Roundup Robot)
Date: Fri, 13 Jan 2017 07:23:30 +0000
Subject: [issue29192] Remove deprecated features from http.cookies
In-Reply-To: <1483787527.13.0.320164851178.issue29192@psf.upfronthosting.co.za>
Message-ID: <20170113072328.55994.49021.8F119E6E@psf.io>


Roundup Robot added the comment:

New changeset 7bd9ec6e5da4 by Serhiy Storchaka in branch 'default':
Issue #29192: Removed deprecated features in the http.cookies module.
https://hg.python.org/cpython/rev/7bd9ec6e5da4

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29192>
_______________________________________

From report at bugs.python.org  Fri Jan 13 02:23:58 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Fri, 13 Jan 2017 07:23:58 +0000
Subject: [issue29192] Remove deprecated features from http.cookies
In-Reply-To: <1483787527.13.0.320164851178.issue29192@psf.upfronthosting.co.za>
Message-ID: <1484292238.91.0.717886270111.issue29192@psf.upfronthosting.co.za>


Changes by Serhiy Storchaka <storchaka+cpython at gmail.com>:


----------
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29192>
_______________________________________

From report at bugs.python.org  Fri Jan 13 02:31:38 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Fri, 13 Jan 2017 07:31:38 +0000
Subject: [issue28871] Destructor of ElementTree.Element is recursive
In-Reply-To: <1480892604.34.0.360751620771.issue28871@psf.upfronthosting.co.za>
Message-ID: <1484292698.85.0.690257230311.issue28871@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

Changes were reverted by 78bf34b6a713.

It is very uneasy to implement an alternative mechanism (without increasing the size of Element objects). It adds duplication of low level garbage collecting code. I think it is better to left all as is in 2.7. Yet one argument for moving to Python 3.

----------
resolution:  -> fixed
stage: needs patch -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28871>
_______________________________________

From report at bugs.python.org  Fri Jan 13 02:35:09 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Fri, 13 Jan 2017 07:35:09 +0000
Subject: [issue29219] TracebackException(capture_locals=True) may fail with
 RecursionError
In-Reply-To: <1484012610.3.0.9642464783.issue29219@psf.upfronthosting.co.za>
Message-ID: <1484292909.61.0.977082132291.issue29219@psf.upfronthosting.co.za>


Changes by Serhiy Storchaka <storchaka+cpython at gmail.com>:


----------
assignee:  -> serhiy.storchaka

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29219>
_______________________________________

From report at bugs.python.org  Fri Jan 13 02:44:51 2017
From: report at bugs.python.org (Roundup Robot)
Date: Fri, 13 Jan 2017 07:44:51 +0000
Subject: [issue29219] TracebackException(capture_locals=True) may fail with
 RecursionError
In-Reply-To: <1484012610.3.0.9642464783.issue29219@psf.upfronthosting.co.za>
Message-ID: <20170113074448.56146.49591.FE4FFC20@psf.io>


Roundup Robot added the comment:

New changeset d9a64d00a439 by Serhiy Storchaka in branch '3.5':
Issue #29219: Fixed infinite recursion in the repr of uninitialized
https://hg.python.org/cpython/rev/d9a64d00a439

New changeset d9539a5c2315 by Serhiy Storchaka in branch '2.7':
Issue #29219: Fixed infinite recursion in the repr of uninitialized
https://hg.python.org/cpython/rev/d9539a5c2315

New changeset 17ca7c4dd583 by Serhiy Storchaka in branch '3.6':
Issue #29219: Fixed infinite recursion in the repr of uninitialized
https://hg.python.org/cpython/rev/17ca7c4dd583

New changeset 0503024831ad by Serhiy Storchaka in branch 'default':
Issue #29219: Fixed infinite recursion in the repr of uninitialized
https://hg.python.org/cpython/rev/0503024831ad

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29219>
_______________________________________

From report at bugs.python.org  Fri Jan 13 02:45:51 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Fri, 13 Jan 2017 07:45:51 +0000
Subject: [issue29219] TracebackException(capture_locals=True) may fail with
 RecursionError
In-Reply-To: <1484012610.3.0.9642464783.issue29219@psf.upfronthosting.co.za>
Message-ID: <1484293551.82.0.406479442589.issue29219@psf.upfronthosting.co.za>


Changes by Serhiy Storchaka <storchaka+cpython at gmail.com>:


----------
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29219>
_______________________________________

From report at bugs.python.org  Fri Jan 13 02:50:25 2017
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 13 Jan 2017 07:50:25 +0000
Subject: [issue28871] Destructor of ElementTree.Element is recursive
In-Reply-To: <1480892604.34.0.360751620771.issue28871@psf.upfronthosting.co.za>
Message-ID: <1484293825.39.0.00221706199077.issue28871@psf.upfronthosting.co.za>


STINNER Victor added the comment:

> Yet one argument for moving to Python 3.

Yep, thanks ;-)

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28871>
_______________________________________

From report at bugs.python.org  Fri Jan 13 02:52:08 2017
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 13 Jan 2017 07:52:08 +0000
Subject: [issue18378] locale.getdefaultlocale() fails on Mac OS X with default
 language set to English
In-Reply-To: <1373113143.69.0.513997485887.issue18378@psf.upfronthosting.co.za>
Message-ID: <1484293928.93.0.98354793309.issue18378@psf.upfronthosting.co.za>


Changes by STINNER Victor <victor.stinner at gmail.com>:


----------
nosy:  -haypo

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue18378>
_______________________________________

From report at bugs.python.org  Fri Jan 13 03:14:13 2017
From: report at bugs.python.org (Christian Heimes)
Date: Fri, 13 Jan 2017 08:14:13 +0000
Subject: [issue29062] hashlib documentation link error
In-Reply-To: <1482571303.3.0.781602557285.issue29062@psf.upfronthosting.co.za>
Message-ID: <1484295253.21.0.529524309158.issue29062@psf.upfronthosting.co.za>


Christian Heimes added the comment:

Go ahead if it makes your work easier. I kept the file separate because the blake2 documentation is maintained externally. It's not going to change any time soon, though. Let's merge it.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29062>
_______________________________________

From report at bugs.python.org  Fri Jan 13 03:28:24 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Fri, 13 Jan 2017 08:28:24 +0000
Subject: [issue29258] __init__.py required for pkgutil.walk_packages in python3
In-Reply-To: <1484282874.75.0.69511180509.issue29258@psf.upfronthosting.co.za>
Message-ID: <1484296104.15.0.643225120295.issue29258@psf.upfronthosting.co.za>


Changes by Serhiy Storchaka <storchaka+cpython at gmail.com>:


----------
nosy: +brett.cannon, eric.snow, ncoghlan
versions:  -Python 3.3, Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29258>
_______________________________________

From report at bugs.python.org  Fri Jan 13 03:32:24 2017
From: report at bugs.python.org (Evan)
Date: Fri, 13 Jan 2017 08:32:24 +0000
Subject: [issue22343] Install bash activate script on Windows when using venv
In-Reply-To: <1409948648.7.0.892236136888.issue22343@psf.upfronthosting.co.za>
Message-ID: <1484296344.15.0.522975678563.issue22343@psf.upfronthosting.co.za>


Evan added the comment:

Works for me on the master branch. Thanks, Vinay.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue22343>
_______________________________________

From report at bugs.python.org  Fri Jan 13 03:48:56 2017
From: report at bugs.python.org (Marc-Andre Lemburg)
Date: Fri, 13 Jan 2017 08:48:56 +0000
Subject: [issue18378] locale.getdefaultlocale() fails on Mac OS X with default
 language set to English
In-Reply-To: <1484279278.87.0.773654036758.issue18378@psf.upfronthosting.co.za>
Message-ID: <6199a640-7e2b-4f5e-945b-b3c419781abb@egenix.com>


Marc-Andre Lemburg added the comment:

On 13.01.2017 04:47, Nick Coghlan wrote:
> Accepting "UTF-8" and interpreting it as functionally equivalent to C.UTF-8 will mean that this setting will at least work as desired on servers that offer C.UTF-8.

I don't think that's within the scope of this patch. "UTF-8" is not
a valid locale setting on Linux and so Python should not allow
passing this through the locale normalization process on Linux.

Please also note that SSH does not forward arbitrary env vars.
Only a select few are forwarded and all others have to be
configured. The locale vars are not among the default ones
(see the ssh man page for details).

Aisde: While looking into this I found that the locale module
aliases C.UTF-8 to en_US.UTF-8. This was added as part of
issue #20076 and originates from the X.org locale.alias file.
Time machine and all that :-)

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue18378>
_______________________________________

From report at bugs.python.org  Fri Jan 13 04:19:45 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Fri, 13 Jan 2017 09:19:45 +0000
Subject: [issue29099] sqlite3 timestamp converter cannot handle timezone
In-Reply-To: <1482966269.67.0.620298847284.issue29099@psf.upfronthosting.co.za>
Message-ID: <1484299185.09.0.337597973036.issue29099@psf.upfronthosting.co.za>


Xiang Zhang added the comment:

> I think the old code unlikely will be broken if preserve an exception type and don't change conditions for raising an error.

Currently there could be no error when the object gets a timezone. The timezone is simply discarded and only when microseconds comes to 0 there is a ValueError. I don't think the user code would prepare to catch the ValueError. I don't oppose make timestamp more strict but just not sure if it's suitable.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29099>
_______________________________________

From report at bugs.python.org  Fri Jan 13 04:23:59 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Fri, 13 Jan 2017 09:23:59 +0000
Subject: [issue29099] sqlite3 timestamp converter cannot handle timezone
In-Reply-To: <1482966269.67.0.620298847284.issue29099@psf.upfronthosting.co.za>
Message-ID: <1484299439.07.0.50419521463.issue29099@psf.upfronthosting.co.za>


Xiang Zhang added the comment:

timestamptz-2.patch make timestamp and timestamptz specilized for their own objects.

----------
Added file: http://bugs.python.org/file46277/timestamptz-2.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29099>
_______________________________________

From report at bugs.python.org  Fri Jan 13 04:34:13 2017
From: report at bugs.python.org (Ezio Melotti)
Date: Fri, 13 Jan 2017 09:34:13 +0000
Subject: [issue2771] Test issue
In-Reply-To: <1210005645.74.0.283923986194.issue2771@psf.upfronthosting.co.za>
Message-ID: <1484300053.13.0.490435424001.issue2771@psf.upfronthosting.co.za>


Ezio Melotti added the comment:

test pr status

----------
pull_requests: +16

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue2771>
_______________________________________

From report at bugs.python.org  Fri Jan 13 05:03:11 2017
From: report at bugs.python.org (Marc-Andre Lemburg)
Date: Fri, 13 Jan 2017 10:03:11 +0000
Subject: [issue29241] sys._enablelegacywindowsfsencoding() don't apply to
 os.fsencode and os.fsdecode
In-Reply-To: <1484137681.59.0.0301192140292.issue29241@psf.upfronthosting.co.za>
Message-ID: <1484301791.96.0.120262933434.issue29241@psf.upfronthosting.co.za>


Marc-Andre Lemburg added the comment:

Adding Victor, who implemented the fs codec.

AFAIK, it's not possible to change the encoding after interpreter initialization, since it will have been already used for many different things by the time you get to executing code.

----------
nosy: +haypo, lemburg

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29241>
_______________________________________

From report at bugs.python.org  Fri Jan 13 05:19:04 2017
From: report at bugs.python.org (Vinay Sajip)
Date: Fri, 13 Jan 2017 10:19:04 +0000
Subject: [issue22343] Install bash activate script on Windows when using venv
In-Reply-To: <1409948648.7.0.892236136888.issue22343@psf.upfronthosting.co.za>
Message-ID: <1484302744.35.0.458252637448.issue22343@psf.upfronthosting.co.za>


Changes by Vinay Sajip <vinay_sajip at yahoo.co.uk>:


----------
assignee:  -> vinay.sajip
resolution:  -> fixed
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue22343>
_______________________________________

From report at bugs.python.org  Fri Jan 13 05:21:51 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Fri, 13 Jan 2017 10:21:51 +0000
Subject: [issue29099] sqlite3 timestamp converter cannot handle timezone
In-Reply-To: <1482966269.67.0.620298847284.issue29099@psf.upfronthosting.co.za>
Message-ID: <1484302911.47.0.669066956696.issue29099@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

I think ProgrammingError is not correct type of an exception. It is used for programming errors such as using uninitialized or closed objects or supplying incorrect number of bindings. But this error can be caused by invalid data in a database created not by Python. Currently converters raise ValueError or TypeError for invalid data. ValueError is raised for datetime without microseconds but with timezone offset. I think ValueError is more appropriate type for such errors.

Perhaps even TypeError should be converted to ValueError.

    try:
        # parsing date or datetime
    except (ValueError, TypeError):
        raise ValueError(...)

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29099>
_______________________________________

From report at bugs.python.org  Fri Jan 13 05:25:52 2017
From: report at bugs.python.org (Armin Rigo)
Date: Fri, 13 Jan 2017 10:25:52 +0000
Subject: [issue29006] 2.7.13 _sqlite more prone to "database table is locked"
In-Reply-To: <1482095939.25.0.891516594682.issue29006@psf.upfronthosting.co.za>
Message-ID: <1484303152.67.0.212297213552.issue29006@psf.upfronthosting.co.za>


Armin Rigo added the comment:

Gian-Carlo is right: I can modify the 2.6 tests in the same way as I described, and then I get the same error with python2.6.  So it seems that all of 2.6 was prone to the same issue, and it was never found, but went away in 2.7 accidentally.  That seems to mean that reverting 030e100f048a was not really necessary (beyond the issue with PyPy/Jython/IronPython).

According to http://bugs.python.org/issue23129 it was not a good idea to revert 030e100f048a.  But I'm surprized that the test added with 030e100f048a continues to pass with the revertion.  Maybe we should investigate why it does.  (Again, don't rely on me for that, because I don't know sqlite.)

If 030e100f048a stays in, I'll probably figure out a hack to avoid this pitfall with PyPy.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29006>
_______________________________________

From report at bugs.python.org  Fri Jan 13 05:28:31 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Fri, 13 Jan 2017 10:28:31 +0000
Subject: [issue29099] sqlite3 timestamp converter cannot handle timezone
In-Reply-To: <1482966269.67.0.620298847284.issue29099@psf.upfronthosting.co.za>
Message-ID: <1484303311.63.0.442015254469.issue29099@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

> Perhaps even TypeError should be converted to ValueError.

But this is different issue of course.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29099>
_______________________________________

From report at bugs.python.org  Fri Jan 13 05:31:28 2017
From: report at bugs.python.org (Roundup Robot)
Date: Fri, 13 Jan 2017 10:31:28 +0000
Subject: [issue29062] hashlib documentation link error
In-Reply-To: <1482571303.3.0.781602557285.issue29062@psf.upfronthosting.co.za>
Message-ID: <20170113103125.56631.16719.36DA1B6E@psf.io>


Roundup Robot added the comment:

New changeset 799ed3122456 by INADA Naoki in branch '3.6':
Issue #29062: Merge hashlib-blake2.rst into hashlib.rst
https://hg.python.org/cpython/rev/799ed3122456

New changeset 380e63b7fc82 by INADA Naoki in branch 'default':
Issue #29062: Merge hashlib-blake2.rst into hashlib.rst
https://hg.python.org/cpython/rev/380e63b7fc82

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29062>
_______________________________________

From report at bugs.python.org  Fri Jan 13 05:32:30 2017
From: report at bugs.python.org (INADA Naoki)
Date: Fri, 13 Jan 2017 10:32:30 +0000
Subject: [issue29062] hashlib documentation link error
In-Reply-To: <1482571303.3.0.781602557285.issue29062@psf.upfronthosting.co.za>
Message-ID: <1484303550.8.0.772295482579.issue29062@psf.upfronthosting.co.za>


INADA Naoki added the comment:

Thanks.

----------
resolution:  -> fixed
stage:  -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29062>
_______________________________________

From report at bugs.python.org  Fri Jan 13 05:36:07 2017
From: report at bugs.python.org (Armin Rigo)
Date: Fri, 13 Jan 2017 10:36:07 +0000
Subject: [issue29006] 2.7.13 _sqlite more prone to "database table is locked"
In-Reply-To: <1482095939.25.0.891516594682.issue29006@psf.upfronthosting.co.za>
Message-ID: <1484303767.63.0.336659924486.issue29006@psf.upfronthosting.co.za>


Armin Rigo added the comment:

...hah, of course the commit dd13098a5dc2 also reverted away the new test.  That test fails.  Sorry about that, and feel free to redo that commit.  It's just one more case in which the implicit refcounting is used, but I guess as it fixes a real issue it's a good idea anyway and should be fixed first in PyPy.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29006>
_______________________________________

From report at bugs.python.org  Fri Jan 13 05:50:52 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Fri, 13 Jan 2017 10:50:52 +0000
Subject: [issue29099] sqlite3 timestamp converter cannot handle timezone
In-Reply-To: <1482966269.67.0.620298847284.issue29099@psf.upfronthosting.co.za>
Message-ID: <1484304652.24.0.888843779004.issue29099@psf.upfronthosting.co.za>


Xiang Zhang added the comment:

I am okay with ValueError?actually I use it in the patch originally) but I am not in favour of catching the errors of the parser. The parser raises errors due to invalid data, not timezone. I think propagate it to users could make the reason more obvious.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29099>
_______________________________________

From report at bugs.python.org  Fri Jan 13 06:07:06 2017
From: report at bugs.python.org (Vinay Sajip)
Date: Fri, 13 Jan 2017 11:07:06 +0000
Subject: =?utf-8?q?=5Bissue24875=5D_pyvenv_doesn=C2=B4t_install_PIP_inside_a_new_v?=
 =?utf-8?q?env_with_--system-site-package?=
In-Reply-To: <1439699040.07.0.199956094974.issue24875@psf.upfronthosting.co.za>
Message-ID: <1484305626.2.0.877663607507.issue24875@psf.upfronthosting.co.za>


Vinay Sajip added the comment:

Just a sanity check ... I can't reproduce the issue with the latest versions of Python (I haven't made any changes to the venv code):

On Python 3.5:

 vinay at ubuntu:~/projects/python/3.5$ ./python
Python 3.5.3rc1+ (3.5:d9a64d00a439, Jan 13 2017, 10:26:23) 
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> 
 vinay at ubuntu:~/projects/python/3.5$ ./python -m venv --system-site-packages /tmp/venv35
 vinay at ubuntu:~/projects/python/3.5$ /tmp/venv35/bin/pip --version
pip 9.0.1 from /tmp/venv35/lib/python3.5/site-packages (python 3.5)

Now try on 3.6:

 vinay at ubuntu:~/projects/python/3.5$ cd ../3.6
 vinay at ubuntu:~/projects/python/3.6$ ./python
Python 3.6.0+ (3.6:17ca7c4dd583, Jan 13 2017, 10:24:18) 
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> 
 vinay at ubuntu:~/projects/python/3.6$ ./python -m venv --system-site-packages /tmp/venv36
 vinay at ubuntu:~/projects/python/3.6$ /tmp/venv36/bin/pip --version
pip 9.0.1 from /tmp/venv36/lib/python3.6/site-packages (python 3.6)

Now try on default (to-be-3.7):

 vinay at ubuntu:~/projects/python/default$ cd ../default
 vinay at ubuntu:~/projects/python/default$ ./python
Python 3.7.0a0 (default:0503024831ad, Jan 13 2017, 10:22:09) 
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> 
 vinay at ubuntu:~/projects/python/default$ ./python -m venv --system-site-packages /tmp/venv37
 vinay at ubuntu:~/projects/python/default$ /tmp/venv37/bin/pip --version
pip 9.0.1 from /tmp/venv37/lib/python3.7/site-packages (python 3.7)

Let's try installing a package: pick a small, one-file one.

 vinay at ubuntu:~/projects/python/default$ /tmp/venv37/bin/pip install python-gnupg
Collecting python-gnupg
  Downloading python_gnupg-0.3.9-py2.py3-none-any.whl
Installing collected packages: python-gnupg
  Installing from a newer Wheel-Version (1.1)
Successfully installed python-gnupg-0.3.9
 vinay at ubuntu:~/projects/python/default$ ls -l /tmp/venv37/lib/python3.7/site-packages/
total 84
-rw-rw-r--  1 vinay vinay   126 Jan 13 11:00 easy_install.py
-rw-rw-r--  1 vinay vinay 51019 Jan 13 11:01 gnupg.py
drwxrwxr-x 11 vinay vinay  4096 Jan 13 11:00 pip
drwxrwxr-x  2 vinay vinay  4096 Jan 13 11:00 pip-9.0.1.dist-info
drwxrwxr-x  5 vinay vinay  4096 Jan 13 11:00 pkg_resources
drwxrwxr-x  2 vinay vinay  4096 Jan 13 11:01 __pycache__
drwxrwxr-x  2 vinay vinay  4096 Jan 13 11:01 python_gnupg-0.3.9.dist-info
drwxrwxr-x  5 vinay vinay  4096 Jan 13 11:00 setuptools
drwxrwxr-x  2 vinay vinay  4096 Jan 13 11:00 setuptools-28.8.0.dist-info

As you can see, gnupg.py is in the list, showing that the package was correctly installed in the virtualenv.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24875>
_______________________________________

From report at bugs.python.org  Fri Jan 13 06:13:23 2017
From: report at bugs.python.org (Vinay Sajip)
Date: Fri, 13 Jan 2017 11:13:23 +0000
Subject: [issue29213] python -m venv activate.bat has weird mix of line endings
In-Reply-To: <1483971138.02.0.292455070123.issue29213@psf.upfronthosting.co.za>
Message-ID: <1484306003.42.0.117324751075.issue29213@psf.upfronthosting.co.za>


Vinay Sajip added the comment:

I've added a patch, can you confirm if it resolves the issue?

----------
assignee:  -> vinay.sajip
keywords: +patch
stage:  -> patch review
Added file: http://bugs.python.org/file46278/issue-29213-01.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29213>
_______________________________________

From report at bugs.python.org  Fri Jan 13 06:17:07 2017
From: report at bugs.python.org (Armin Rigo)
Date: Fri, 13 Jan 2017 11:17:07 +0000
Subject: [issue29006] 2.7.13 _sqlite more prone to "database table is locked"
In-Reply-To: <1482095939.25.0.891516594682.issue29006@psf.upfronthosting.co.za>
Message-ID: <1484306227.63.0.000171717140162.issue29006@psf.upfronthosting.co.za>


Armin Rigo added the comment:

Managed to write a patch in PyPy that seems to pass all tests including the new one, including on Windows.  I know think that dd13098a5dc2 should be backed out (i.e. 030e100f048a should be kept).

Reference to the PyPy changes: https://bitbucket.org/pypy/pypy/commits/235e8a3889790042b3f148bcf04891b27f97a1fc

Maybe something similar should be added to CPython, to avoid the unexpected "database table is locked" case; but such a change should probably be done only in trunk, because the <= 2.6 experience seems to suggest it is rare enough in practice.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29006>
_______________________________________

From report at bugs.python.org  Fri Jan 13 06:20:33 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Fri, 13 Jan 2017 11:20:33 +0000
Subject: [issue29210] Remove the support of the exclude argument in
 tarfile.TarFile.add()
In-Reply-To: <1483899747.86.0.0120992057455.issue29210@psf.upfronthosting.co.za>
Message-ID: <1484306433.3.0.243756436689.issue29210@psf.upfronthosting.co.za>


Changes by Serhiy Storchaka <storchaka+cpython at gmail.com>:


----------
assignee:  -> serhiy.storchaka

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29210>
_______________________________________

From report at bugs.python.org  Fri Jan 13 06:23:12 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Fri, 13 Jan 2017 11:23:12 +0000
Subject: [issue29099] sqlite3 timestamp converter cannot handle timezone
In-Reply-To: <1482966269.67.0.620298847284.issue29099@psf.upfronthosting.co.za>
Message-ID: <1484306592.55.0.919149042063.issue29099@psf.upfronthosting.co.za>


Xiang Zhang added the comment:

timestamptz-3.patch uses ValueError instead of ProgrammingError and treat suffix Z as UTC timezone.

----------
Added file: http://bugs.python.org/file46279/timestamptz-3.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29099>
_______________________________________

From report at bugs.python.org  Fri Jan 13 06:25:43 2017
From: report at bugs.python.org (Roundup Robot)
Date: Fri, 13 Jan 2017 11:25:43 +0000
Subject: [issue29210] Remove the support of the exclude argument in
 tarfile.TarFile.add()
In-Reply-To: <1483899747.86.0.0120992057455.issue29210@psf.upfronthosting.co.za>
Message-ID: <20170113112540.112302.75053.40C3D9FD@psf.io>


Roundup Robot added the comment:

New changeset 4e23948f2df2 by Serhiy Storchaka in branch 'default':
Issue #29210:  Removed support of deprecated argument "exclude" in
https://hg.python.org/cpython/rev/4e23948f2df2

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29210>
_______________________________________

From report at bugs.python.org  Fri Jan 13 06:26:23 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Fri, 13 Jan 2017 11:26:23 +0000
Subject: [issue29210] Remove the support of the exclude argument in
 tarfile.TarFile.add()
In-Reply-To: <1483899747.86.0.0120992057455.issue29210@psf.upfronthosting.co.za>
Message-ID: <1484306783.93.0.337656087044.issue29210@psf.upfronthosting.co.za>


Changes by Serhiy Storchaka <storchaka+cpython at gmail.com>:


----------
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29210>
_______________________________________

From report at bugs.python.org  Fri Jan 13 07:19:31 2017
From: report at bugs.python.org (Mark Haase)
Date: Fri, 13 Jan 2017 12:19:31 +0000
Subject: =?utf-8?q?=5Bissue24875=5D_pyvenv_doesn=C2=B4t_install_PIP_inside_a_new_v?=
 =?utf-8?q?env_with_--system-site-package?=
In-Reply-To: <1439699040.07.0.199956094974.issue24875@psf.upfronthosting.co.za>
Message-ID: <1484309971.47.0.444378796297.issue24875@psf.upfronthosting.co.za>


Mark Haase added the comment:

Hey Thomas and Vinay, thanks for looking into this. Although I discovered this problem on my MacOS laptop 6 months ago, today I also reproduced this issue on my Ubuntu desktop.

```
/home/mhaase/Downloads $ mkdir temp
/home/mhaase/Downloads $ cd temp
/home/mhaase/Downloads/temp $ pyvenv --system-site-packages pyvenv
/home/mhaase/Downloads/temp $ source pyvenv/bin/activate
(pyvenv) /home/mhaase/Downloads/temp $ pip --version
pip 9.0.1 from /usr/local/lib/python2.7/dist-packages (python 2.7)
(pyvenv) /home/mhaase/Downloads/temp $ python --version
Python 3.5.2
(pyvenv) /home/mhaase/Downloads/temp $ cat /etc/lsb-release 
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=16.04
DISTRIB_CODENAME=xenial
DISTRIB_DESCRIPTION="Ubuntu 16.04.1 LTS"
(pyvenv) /home/mhaase/Downloads/temp $ uname -av
Linux prodigy 4.4.0-57-generic #78-Ubuntu SMP Fri Dec 9 23:50:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
```

This bug is really annoying if you work with scientific Python (numpy, scipy, sklearn), as those are big, stable packages that I would prefer to install once on my system and then access from multiple venvs. The standardization of linux wheels in 2016 made this slightly less problematic (don't need to compile all of numpy just to create a venv for an ipython notebook), but this is still an annoying bug. Unfortunately, I think it lacks the upvotes to get attention from the PyPA developers.

Maybe it would help to post this on the https://groups.google.com/forum/#!forum/pypa-dev mailing list?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24875>
_______________________________________

From report at bugs.python.org  Fri Jan 13 07:24:35 2017
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 13 Jan 2017 12:24:35 +0000
Subject: [issue29259] Add tp_fastcall to PyTypeObject: support FASTCALL
 calling convention for all callable objects
Message-ID: <1484310274.99.0.832491900546.issue29259@psf.upfronthosting.co.za>


New submission from STINNER Victor:

A new FASTCALL calling convention was added to Python 3.6. It allows to avoid the creation of a temporary tuple to pass positional arguments and a temporary dictionary to pass keyword arguments. A new METH_FASTCALL calling convention was added for C functions. Most functions now support fastcall, except objects with a __call__() method which have to go through slot_tp_call() which still requires a tuple and dictionary.

I tried multiple implementations to support fast calls to call the __call__() method, but I had practical and technical issues.

First, I tried to reuse the tp_call field to PyTypeObject: it can be a regular call (tuple/dict for arguments) or a fast call. I added a flag to the tp_flags field. It was tricky to support class inheritance, decide to set or clear the flag. But the real blocker issue is fAthat it is obviously breaks the backward compatibility: existing code calling directly tp_call with the regular calling convention will crash immediatly, and the error is not catched during compilation, even if the code is recompiled.

I propose a different design: add a new tp_fastcall field to PyTypeObject and use a wrapper for tp_call when tp_fastcall is defined. If a type defines tp_fastcall but not, the tp_call wrapper "simply" calls tp_fastcall. Advantages:

* The wrapper is trivial
* Minor changes to PyType_Ready() to support inheritance (simple logic)
* Fully backward compatible
* If tp_call is called directly without keyword arguments, there is no overhead but a speedup!

Inheritance:

* If a type only defines tp_call, tp_fastcall is not inherited from the parent: tp_fastcall is set to NULL.
* If a type only defines tp_fastcall: tp_fastcall is always use (tp_call uses the wrapper)
* If a type defines tp_call and tp_fastcall, PyObject_Call() uses tp_call whereas _PyObject_FastCallDict() uses tp_fastcall.

Functions of the C API will be modified to use tp_fastcall if available.

The plan is then to patch most Python types to replace their tp_call with tp_fastcall. First, most important (common) types like Python and C functions, descriptors, and the various kinds of wrappers should be patched. Later, we should maybe discuss on a case by case basis to decide if it's worth it.

I will try to run benchmark before any kind.

----------
messages: 285388
nosy: haypo, inada.naoki, serhiy.storchaka
priority: normal
pull_requests: 17
severity: normal
status: open
title: Add tp_fastcall to PyTypeObject: support FASTCALL calling convention for all callable objects
type: performance
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29259>
_______________________________________

From report at bugs.python.org  Fri Jan 13 07:26:01 2017
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 13 Jan 2017 12:26:01 +0000
Subject: [issue29259] Add tp_fastcall to PyTypeObject: support FASTCALL
 calling convention for all callable objects
In-Reply-To: <1484310274.99.0.832491900546.issue29259@psf.upfronthosting.co.za>
Message-ID: <1484310361.33.0.974275125116.issue29259@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Naoki wants to use FASTCALL for CALL_METHOD:
http://bugs.python.org/issue26110#msg283093

tp_fastcall should allow to easily implement it.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29259>
_______________________________________

From report at bugs.python.org  Fri Jan 13 07:33:21 2017
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 13 Jan 2017 12:33:21 +0000
Subject: [issue29259] Add tp_fastcall to PyTypeObject: support FASTCALL
 calling convention for all callable objects
In-Reply-To: <1484310274.99.0.832491900546.issue29259@psf.upfronthosting.co.za>
Message-ID: <1484310801.82.0.291994874073.issue29259@psf.upfronthosting.co.za>


STINNER Victor added the comment:

I started to work on FASTCALL, because I dislike the "cached tuple" hack used in some performance critical code, and the hack causes various kinds of tricky but severe issues (can lead to segfault).

Thanks to tp_fastcall, it becomes possible to drop the "cached tuple" hack from property_descr_get() *and* keep good performances.

First, a benchmark to show the performance gain of using "cached tuple". I modified property_descr_get() to use Python 3.4 code which doesn't have the optimization:

$ ./python -m perf compare_to py34.json ref.json 
Median +- std dev: [py34] 75.0 ns +- 1.7 ns -> [ref] 50.0 ns +- 0.9 ns: 1.50x faster (-33%)

It's MUCH faster, good job. But it requires complex and fragile code. Ok, let's see with operator.itemgetter() supporting tp_fastcall, Python modified to use tp_fastcall and without the "cached arg" hack:

$ ./python -m perf compare_to ref.json fastcall_wrapper.json 
Median +- std dev: [ref] 50.0 ns +- 0.9 ns -> [fastcall_wrapper] 48.2 ns +- 1.5 ns: 1.04x faster (-4%)

It's a little bit faster, but that's not the point. The point is that it isn't slower and it doesn't require to modify C code to benefit of the optimization! Just to be clear, another benchmark result on property_descr_get() without "cache args", without fastcall (py34) and with fastcall ("fastcall_wrapper"):

$ ./python -m perf compare_to py34.json fastcall_wrapper.json 
Median +- std dev: [py34] 75.0 ns +- 1.7 ns -> [fastcall_wrapper] 48.2 ns +- 1.5 ns: 1.56x faster (-36%)

Summary:

* tp_fastcall avoids to remove the "cached args" hack which will fix severe issue in corner cases
* tp_fastcall makes existing code faster for free. I mean, builtin types should be modified to support tp_fastcall, most all code *calling* these types don't need any change.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29259>
_______________________________________

From report at bugs.python.org  Fri Jan 13 07:39:06 2017
From: report at bugs.python.org (Xavier de Gaye)
Date: Fri, 13 Jan 2017 12:39:06 +0000
Subject: [issue26858] android: setting SO_REUSEPORT fails
In-Reply-To: <1461677842.81.0.574068471289.issue26858@psf.upfronthosting.co.za>
Message-ID: <1484311146.07.0.632469087834.issue26858@psf.upfronthosting.co.za>


Xavier de Gaye added the comment:

The changes made in issue #28174 fix the problem for the Android x86 platform and for the armv7 platform at Android API level 24, but the problem is still there on the armv7 platform at Android API level 21.

======================================================================
ERROR: test_create_datagram_endpoint_sockopts (test.test_asyncio.test_base_events.BaseEventLoopWithS
electorTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/test/test_asyncio/test_base_events.py", line 1593,
 in test_create_datagram_endpoint_sockopts
    socket.SOL_SOCKET, socket.SO_REUSEPORT))
OSError: [Errno 92] Protocol not available

======================================================================
ERROR: test_create_server_reuse_port (test.test_asyncio.test_events.EPollEventLoopTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/test/test_asyncio/test_events.py", line 913, in te
st_create_server_reuse_port
    socket.SOL_SOCKET, socket.SO_REUSEPORT))
OSError: [Errno 92] Protocol not available

======================================================================
ERROR: test_create_server_reuse_port (test.test_asyncio.test_events.PollEventLoopTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/test/test_asyncio/test_events.py", line 913, in te
st_create_server_reuse_port
    socket.SOL_SOCKET, socket.SO_REUSEPORT))
OSError: [Errno 92] Protocol not available

======================================================================
ERROR: test_create_server_reuse_port (test.test_asyncio.test_events.SelectEventLoopTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/test/test_asyncio/test_events.py", line 913, in te
st_create_server_reuse_port
    socket.SOL_SOCKET, socket.SO_REUSEPORT))
OSError: [Errno 92] Protocol not available

----------------------------------------------------------------------
Ran 1476 tests in 382.973s

FAILED (errors=4, skipped=2)
test test_asyncio failed
test_asyncio failed in 7 min 25 sec

1 test failed:
    test_asyncio

Total duration: 7 min 27 sec
Tests result: FAILURE

----------
components: +asyncio -Cross-Build
resolution: fixed -> 
stage: resolved -> needs patch
status: closed -> open
versions: +Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue26858>
_______________________________________

From report at bugs.python.org  Fri Jan 13 07:46:24 2017
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 13 Jan 2017 12:46:24 +0000
Subject: [issue29241] sys._enablelegacywindowsfsencoding() don't apply to
 os.fsencode and os.fsdecode
In-Reply-To: <1484137681.59.0.0301192140292.issue29241@psf.upfronthosting.co.za>
Message-ID: <1484311584.38.0.0800548635489.issue29241@psf.upfronthosting.co.za>


STINNER Victor added the comment:

My experience with changing the Python "filesystem encoding" (sys.getfilesystemencoding()) at runtime: it doesn't work.

The filesystem encoding must be set as soon as possible and must never change later. As soon as possible: before the first call to os.fsdecode(), which is implemented in C as Py_DecodeLocale(). For example, the encoding must be set before Python imports the first module.

The filesystem encoding must be set before Python decodes *any* operating system data: command line arguments, any filename or path, environment variables, etc.

Hopefully, Windows provides most operating system data as Unicode directly: command line arguments and environment variables are exposed as Unicode for example.

os.fsdecode() and os.fsencode() have an important property:
assert os.fsencode(os.fsdecode(data)) == data

On Windows, the other property is maybe more imporant:
assert os.fsdecode(os.fsencode(data)) == data

If the property becomes false, for example if the filesystem encoding is changed at runtime, you get mojibake. Example:

* os.fsdecode() decodes the filename b'h\xc3\xa9llo' from UTF-8 => 'h\xe9llo'
* sys._enablelegacywindowsfsencoding()
* os.fsencode() encodes the filename to cp1252 => you get 'h\xc3\xa9llo'
 instead of 'h\xe9llo', say hello to mojibake

--

Sorry, I didn't play with sys._enablelegacywindowsfsencoding() on Windows. I don't know if it would "work" if sys._enablelegacywindowsfsencoding() is the first instruction of an application. I expect that Python almost decodes nothing at startup on Windows, so it may work.

sys._enablelegacywindowsfsencoding() is a private method, so it shouldn't be used.

Maybe we could add a "fuse" (flag only sets to 1, cannot be reset to 0) to raise an exception if sys._enablelegacywindowsfsencoding() is called "too late", after the first call to os.fsdecode() / Py_DecodeLocale()?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29241>
_______________________________________

From report at bugs.python.org  Fri Jan 13 07:51:04 2017
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 13 Jan 2017 12:51:04 +0000
Subject: [issue26858] android: setting SO_REUSEPORT fails
In-Reply-To: <1461677842.81.0.574068471289.issue26858@psf.upfronthosting.co.za>
Message-ID: <1484311864.34.0.506217401393.issue26858@psf.upfronthosting.co.za>


STINNER Victor added the comment:

IMHO it's ok to skip a few asyncio tests on Android. It's safe if it doesn't touch the code of asyncio itself.

It don't really care if the Android version is not checked yet. Moreover, if we really want to check the version, we should check the *runtime* verison, not the *build* version of Android. We need the discussed platform.android_ver() function which doesn't exist yet. So it's fine to postpone the version check in asyncio tests.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue26858>
_______________________________________

From report at bugs.python.org  Fri Jan 13 08:40:28 2017
From: report at bugs.python.org (Michael Felt)
Date: Fri, 13 Jan 2017 13:40:28 +0000
Subject: [issue26439] ctypes.util.find_library fails when ldconfig/glibc not
 available (e.g., AIX)
In-Reply-To: <1456413023.18.0.874865601403.issue26439@psf.upfronthosting.co.za>
Message-ID: <1484314828.47.0.352824271967.issue26439@psf.upfronthosting.co.za>


Michael Felt added the comment:

I would like to say:
a) I am happy this is being considered for Python3-3.7

b) I still believe it is a "bug" plain and simple - it (find_library()) cannot work in "normal" situations. Why "IBM" or others never addressed this is beyond me - but - imho - that is a poor argument for not fixing it.

FYI: As I have documented in ... this does not mean that ctypes.CDLL is incapable of loading a library. If you know the 'magic', aka what is not being provided by this "implementation dependent" module - it is possible to load either a .so file (actually a file by any name as long as it is a shared archive) - or an AIX-style .a archive with shared libraries stored internally as "archive members" - again, any member name is acceptable.

The bug: The "default code" behind an "else:" block depends on the presence of the program "/sbin/ldconfig" to create a list of shared libraries. Normally, this program is not installed on an AIX system (certainly not production - maybe a system with gcc environment and using the gcc ld rather than the AIX ld (the latter being normal).

Using python -m trace -C /tmp --count Lib/src/util.py The following (key) counts are reported.

This is all that is being called - so when /sbin/ldonfig is not installed (which is usual) - so this code can only return 'None'.

Environment: AIX 5.3, AIX 6.1, AIX 7.1 - all identical

root at x064:[/]ls -l /sbin/ld*
ls: 0653-341 The file /sbin/ld* does not exist.

michael at x071:[/home/michael]ls /sbin/ld*
ls: cannot access '/sbin/ld*': A file or directory in the path name does not exist.
(GNU coreutils ls here)

root at x062:[/]ls /sbin/ld*
ls: 0653-341 The file /sbin/ld* does not exist.

So, obviously the sys.popen() call below will always 'fail'.


The basic trace from util.py:
    1:     if os.name == "posix":
               # find and load_version
    1:         print find_library("m")
    1:         print find_library("c")
    1:         print find_library("bz2")


And how this is called:

           else:

    1:         def _findSoname_ldconfig(name):
    3:             import struct
    3:             if struct.calcsize('l') == 4:
                       machine = os.uname()[4] + '-32'
                   else:
    3:                 machine = os.uname()[4] + '-64'
    3:             mach_map = {
    3:                 'x86_64-64': 'libc6,x86-64',
    3:                 'ppc64-64': 'libc6,64bit',
    3:                 'sparc64-64': 'libc6,64bit',
    3:                 's390x-64': 'libc6,64bit',
    3:                 'ia64-64': 'libc6,IA-64',
                       }
    3:             abi_type = mach_map.get(machine, 'libc6')

                   # XXX assuming GLIBC's ldconfig (with option -p)
    3:             expr = r'\s+(lib%s\.[^\s]+)\s+\(%s' % (re.escape(name), abi_type)
    3:             f = os.popen('LC_ALL=C LANG=C /sbin/ldconfig -p 2>/dev/null')
    3:             try:
    3:                 data = f.read()
                   finally:
    3:                 f.close()
    3:             res = re.search(expr, data)
    3:             if not res:
    3:                 return None
                   return res.group(1)

    1:         def find_library(name):
    3:             return _findSoname_ldconfig(name) or _get_soname(_findLib_gcc(name))


And with python2-2.7.13 - the counts start the same - but the result is the same - by definition 'None' because subprocess.Popen() also
does not have any output to search...

    1:     if os.name == "posix":
               # find and load_version
    1:         print find_library("m")
    1:         print find_library("c")
    1:         print find_library("bz2")

           else:

    1:         def _findSoname_ldconfig(name):
    3:             import struct
    3:             if struct.calcsize('l') == 4:
    3:                 machine = os.uname()[4] + '-32'
                   else:
                       machine = os.uname()[4] + '-64'
    3:             mach_map = {
    3:                 'x86_64-64': 'libc6,x86-64',
    3:                 'ppc64-64': 'libc6,64bit',
    3:                 'sparc64-64': 'libc6,64bit',
    3:                 's390x-64': 'libc6,64bit',
    3:                 'ia64-64': 'libc6,IA-64',
                       }
    3:             abi_type = mach_map.get(machine, 'libc6')

                   # XXX assuming GLIBC's ldconfig (with option -p)
    3:             expr = r'\s+(lib%s\.[^\s]+)\s+\(%s' % (re.escape(name), abi_type)

    3:             env = dict(os.environ)
    3:             env['LC_ALL'] = 'C'
    3:             env['LANG'] = 'C'
    3:             null = open(os.devnull, 'wb')
    3:             try:
    3:                 with null:
    3:                     p = subprocess.Popen(['/sbin/ldconfig', '-p'],
    3:                                           stderr=null,
    3:                                           stdout=subprocess.PIPE,
    3:                                           env=env)
    3:             except OSError:  # E.g. command not found
    3:                 return None
                   [data, _] = p.communicate()
                   res = re.search(expr, data)
                   if not res:
                       return None
                   return res.group(1)

========
In closing:

a) this is an "issue" aka bug, plain and simple - even it is has been ignored as such (other issues only complained about poorer performance because /sbin/ldconfig was not available). Please do not say - not fixing it because noone ever complained before. Otherwise, what is the point of ever accepting bug-reports aka issues if they can just be ignored.

b) I want to thank Martin for his help on many (PEP8 et al) improvements to my initial proposals for a patch.

c) more important to me right now is that this be recognized as a bug - that should have been reported and resolved years ago. Personally, I do not care why noone ever reported it BUT I would like to see it properly identified that the default code is a specific implementation that is no way related to a normal AIX system - and an AIX-specific implementation is needed for normal operation of python2 and/or python3.

Thank you for your time and thought!

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue26439>
_______________________________________

From report at bugs.python.org  Fri Jan 13 08:47:30 2017
From: report at bugs.python.org (Michael Felt)
Date: Fri, 13 Jan 2017 13:47:30 +0000
Subject: [issue26439] ctypes.util.find_library fails when ldconfig/glibc not
 available (e.g., AIX)
In-Reply-To: <1456413023.18.0.874865601403.issue26439@psf.upfronthosting.co.za>
Message-ID: <1484315250.24.0.840447030893.issue26439@psf.upfronthosting.co.za>


Michael Felt added the comment:

OOPS: I have a ... above, meant to be a link to a message.

I also needed to 'patch' util.py - with 32-bit build with something like this:
--- src/python-2.7.13/Lib/ctypes/util.py        2016-12-17 20:05:05 +0000
+++ python-2.7.13.0/Lib/ctypes/util.py  2017-01-13 13:29:12 +0000
@@ -299,6 +299,10 @@
             print cdll.LoadLibrary("libcrypto.dylib")
             print cdll.LoadLibrary("libSystem.dylib")
             print cdll.LoadLibrary("System.framework/System")
+        elif sys.platform[:3] == "aix":
+            from ctypes import CDLL
+            RTLD_MEMBER =  0x00040000
+            print CDLL("libc.a(shr.o)", RTLD_MEMBER)
         else:
             print cdll.LoadLibrary("libm.so")
             print cdll.LoadLibrary("libcrypt.so")


And 64-bit with:

    1:         elif sys.platform[:3] == "aix":
    1:             from ctypes import CDLL
    1:             RTLD_MEMBER =  0x00040000
    1:             print CDLL("libc_64.a(shr.o)", RTLD_MEMBER)

The respective output is:

root at x064:[/data/prj/python/python-2.7.13.0]./python Lib/ctypes/util.py
None
None
None
<CDLL 'libc.a(shr.o)', handle 10 at 300fe0b0>

root at x064:[/data/prj/python/python-2.7.12.0]./python Lib/ctypes/util.py
None
None
None
<CDLL 'libc.a(shr_64.o)', handle d at 7000000001941d0>

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue26439>
_______________________________________

From report at bugs.python.org  Fri Jan 13 09:04:05 2017
From: report at bugs.python.org (Vinay Sajip)
Date: Fri, 13 Jan 2017 14:04:05 +0000
Subject: =?utf-8?q?=5Bissue24875=5D_pyvenv_doesn=C2=B4t_install_PIP_inside_a_new_v?=
 =?utf-8?q?env_with_--system-site-package?=
In-Reply-To: <1439699040.07.0.199956094974.issue24875@psf.upfronthosting.co.za>
Message-ID: <1484316245.85.0.963661760443.issue24875@psf.upfronthosting.co.za>


Vinay Sajip added the comment:

I see why I couldn't reproduce the problem - there's no pip installed in the site-packages of the Pythons I was testing with, so the problem didn't show up.

Mark, can you apply the version of EnvBuilder.create that I posted in a message above and see if that resolves the issue for you on both OS X and Ubuntu? Thomas has confirmed that it works for him.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24875>
_______________________________________

From report at bugs.python.org  Fri Jan 13 09:16:40 2017
From: report at bugs.python.org (Steve Dower)
Date: Fri, 13 Jan 2017 14:16:40 +0000
Subject: [issue29241] sys._enablelegacywindowsfsencoding() don't apply to
 os.fsencode and os.fsdecode
In-Reply-To: <1484137681.59.0.0301192140292.issue29241@psf.upfronthosting.co.za>
Message-ID: <1484317000.53.0.731563254539.issue29241@psf.upfronthosting.co.za>


Steve Dower added the comment:

Windows doesn't use the fs encoding at all until Python code requests/provides something in bytes. Except for the caching in fsencode/fsdecode, there's no problem setting it once at the start of your program (and it can only be set once - there's no parameter and it cannot be undone).

What I'm most interested in is whether caching the encoding in fsencode/fsdecode is actually an optimization - if not, remove it, and if so make a way to reset it. I'll get around to this sooner or later but I don't want to stop someone else from doing it.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29241>
_______________________________________

From report at bugs.python.org  Fri Jan 13 09:31:05 2017
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 13 Jan 2017 14:31:05 +0000
Subject: [issue29260] Use designated initializers to define PyTypeObject types
Message-ID: <1484317865.58.0.725398845848.issue29260@psf.upfronthosting.co.za>


New submission from STINNER Victor:

Currently, PyTypeObject fields are set in order in the C code. Typical example:

PyTypeObject PyMethod_Type = {
    PyVarObject_HEAD_INIT(&PyType_Type, 0)
    "method",
    sizeof(PyMethodObject),
    0,
    (destructor)method_dealloc,                 /* tp_dealloc */
    0,                                          /* tp_print */
    0,                                          /* tp_getattr */
    0,                                          /* tp_setattr */
    0,                                          /* tp_reserved */
    (reprfunc)method_repr,                      /* tp_repr */
    0,                                          /* tp_as_number */
    0,                                          /* tp_as_sequence */
    0,                                          /* tp_as_mapping */
    (hashfunc)method_hash,                      /* tp_hash */
    method_call,                                /* tp_call */
    0,                                          /* tp_str */
    method_getattro,                            /* tp_getattro */
    PyObject_GenericSetAttr,                    /* tp_setattro */
    0,                                          /* tp_as_buffer */
    Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC, /* tp_flags */
    method_doc,                                 /* tp_doc */
    (traverseproc)method_traverse,              /* tp_traverse */
    0,                                          /* tp_clear */
    method_richcompare,                         /* tp_richcompare */
    offsetof(PyMethodObject, im_weakreflist), /* tp_weaklistoffset */
    0,                                          /* tp_iter */
    0,                                          /* tp_iternext */
    method_methods,                             /* tp_methods */
    method_memberlist,                          /* tp_members */
    method_getset,                              /* tp_getset */
    0,                                          /* tp_base */
    0,                                          /* tp_dict */
    method_descr_get,                           /* tp_descr_get */
    0,                                          /* tp_descr_set */
    0,                                          /* tp_dictoffset */
    0,                                          /* tp_init */
    0,                                          /* tp_alloc */
    method_new,                                 /* tp_new */
};

The aligned comment is an old practice required to be able to see to which field correspond a line.

This syntax usually produces a lot of zeros.

The C standard (C99?) allows to use a more readable syntax:

PyTypeObject PyMethod_Type = {
    PyVarObject_HEAD_INIT(&PyType_Type, 0)
    .tp_name = "method",
    .tp_basicsize = sizeof(PyMethodObject),
    .tp_dealloc = (destructor)method_dealloc,
    .tp_repr = (reprfunc)method_repr,
    .tp_hash = (hashfunc)method_hash,
    .tp_call = method_call,
    .tp_getattro = method_getattro,
    ...
};

It seems like it's possible to start with positional fields and then switch to named fields, so PyVarObject_HEAD_INIT() still works. Maybe maybe PyVarObject_HEAD_INIT() should also evolve? Or maybe we need a new macro using the ".tp_xxx=...," syntax?

INADA Naoki proposed to use this syntax in my pull request which adds a new tp_fastcall field which requires to add many zeros:
https://github.com/python/cpython/pull/65#pullrequestreview-16566423

Example of change:

@@ -370,6 +370,17 @@ PyTypeObject PyMethod_Type = {
     0,                                          /* tp_init */
     0,                                          /* tp_alloc */
     method_new,                                 /* tp_new */
+    0,                                          /* tp_free */
+    0,                                          /* tp_is_gc */
+    0,                                          /* tp_bases */
+    0,                                          /* tp_mro */
+    0,                                          /* tp_cache */
+    0,                                          /* tp_subclasses */
+    0,                                          /* tp_weaklist */
+    0,                                          /* tp_del */
+    0,                                          /* tp_version_tag */
+    0,                                          /* tp_finalize */
+    (fastternaryfunc)method_fastcall,           /* tp_fastcall */
 };

----------
messages: 285399
nosy: haypo, inada.naoki
priority: normal
severity: normal
status: open
title: Use designated initializers to define PyTypeObject types
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29260>
_______________________________________

From report at bugs.python.org  Fri Jan 13 09:47:35 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Fri, 13 Jan 2017 14:47:35 +0000
Subject: [issue29259] Add tp_fastcall to PyTypeObject: support FASTCALL
 calling convention for all callable objects
In-Reply-To: <1484310274.99.0.832491900546.issue29259@psf.upfronthosting.co.za>
Message-ID: <1484318855.58.0.766914460402.issue29259@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

* If a type only defines tp_fastcall: tp_fastcall is always use (tp_call uses the wrapper)

Is tp_call set to the wrapper rather then inheriting? What if tp_call is defined in a superclass?

> * If a type defines tp_call and tp_fastcall, PyObject_Call() uses tp_call whereas _PyObject_FastCallDict() uses tp_fastcall.

I would consider this as a bug. It would be weird if different ways of calling cause executing different code.

What about dynamically changed Python types? What if you set or delete the __call__ attribute of Python class?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29259>
_______________________________________

From report at bugs.python.org  Fri Jan 13 09:56:59 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Fri, 13 Jan 2017 14:56:59 +0000
Subject: [issue29260] Use designated initializers to define PyTypeObject types
In-Reply-To: <1484317865.58.0.725398845848.issue29260@psf.upfronthosting.co.za>
Message-ID: <1484319419.92.0.343568642602.issue29260@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

The widespread use of PyType_FromSpec() will supersede this issue. But there is a caveat (issue26979).

----------
nosy: +serhiy.storchaka

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29260>
_______________________________________

From report at bugs.python.org  Fri Jan 13 10:03:02 2017
From: report at bugs.python.org (Xavier de Gaye)
Date: Fri, 13 Jan 2017 15:03:02 +0000
Subject: [issue26858] android: setting SO_REUSEPORT fails
In-Reply-To: <1461677842.81.0.574068471289.issue26858@psf.upfronthosting.co.za>
Message-ID: <1484319782.83.0.135840609973.issue26858@psf.upfronthosting.co.za>


Xavier de Gaye added the comment:

Skipping those tests on Android has already been proposed with the existing patches in this issue.
The reason why the changes made by issue 28174 fix only partly the problem should be understood.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue26858>
_______________________________________

From report at bugs.python.org  Fri Jan 13 10:06:25 2017
From: report at bugs.python.org (Dave Jones)
Date: Fri, 13 Jan 2017 15:06:25 +0000
Subject: [issue29261] Missing venv/scripts/common after "make install"
Message-ID: <1484319985.48.0.948212892156.issue29261@psf.upfronthosting.co.za>


New submission from Dave Jones:

While attempting to diagnose something (unrelated to this issue) under python 3.6, I used the following steps to clone and build a non-root python installation:

    $ mkdir py36
    $ hg clone https://hg.python.org/cpython
    $ cd cpython
    $ hg update 3.6
    $ ./configure --prefix=/home/pi/py36
    $ make
    $ make install

Afterwards I attempted to set up a venv for further testing:

    $ /home/pi/py36/bin/python3 -m venv py36env

This succeeded but, unfortunately, activating the venv proved impossible as the "activate" script was missing from its "bin" sub-dir. 

After a bit of investigation it appears that "make install" had copied "Lib/venv/scripts/posix" but not "Lib/venv/scripts/common" into the installation, hence the templates for "activate.csh" and "activate.fish" are present, but not the "activate" template.

I had a search around for related issues but only found #22343 which explains why "activate" is no longer present (because it's moved from scripts/posix/ to scripts/common/) but I don't think is responsible for common not getting copied by "make install".

It appears adding "venv/scripts/common" to LIBSUBDIRS in Makefile.pre.in fixes the issue (as in the attached patch) but I've no idea if this covers all necessary platforms (I'd guess Windows doesn't use the Makefile if #22343 fixed things for it?).

----------
components: Installation
files: missing_venv_common.diff
keywords: patch
messages: 285403
nosy: Dave Jones
priority: normal
severity: normal
status: open
title: Missing venv/scripts/common after "make install"
type: behavior
versions: Python 3.6, Python 3.7
Added file: http://bugs.python.org/file46280/missing_venv_common.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29261>
_______________________________________

From report at bugs.python.org  Fri Jan 13 10:08:22 2017
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 13 Jan 2017 15:08:22 +0000
Subject: [issue29259] Add tp_fastcall to PyTypeObject: support FASTCALL
 calling convention for all callable objects
In-Reply-To: <1484310274.99.0.832491900546.issue29259@psf.upfronthosting.co.za>
Message-ID: <1484320102.12.0.657780295125.issue29259@psf.upfronthosting.co.za>


STINNER Victor added the comment:

> * If a type only defines tp_fastcall: tp_fastcall is always use (tp_call uses the wrapper)
> 
> Is tp_call set to the wrapper rather then inheriting?

It is set to the wrapper. Defining tp_fastcall should work as defining tp_call: it should override the tp_call and tp_fastcall slots.


>  What if tp_call is defined in a superclass?

What is a superclass? Do you have an example?



> * If a type defines tp_call and tp_fastcall, PyObject_Call() uses tp_call whereas _PyObject_FastCallDict() uses tp_fastcall.
>
> I would consider this as a bug. It would be weird if different ways of calling cause executing different code.

It's a micro-optimization to avoid arguments unpacking/packing, conversions needed when you switch from the regular calling convention to the fast call convention, or the opposite.

I don't think that types defining tp_call and tp_fastcall will be common.


> What about dynamically changed Python types?

What do you mean? Do you have an example?


> What if you set or delete the __call__ attribute of Python class?

I don't know how these things work :-) Let me try on a patched Python (using tp_fastcall):

$ ./python
>>> class A:
...  def __call__(self): print("A")
... 
>>> a=A()
>>> a()
A

>>> A.__call__=lambda self: print("B!")
>>> a()
B!

>>> del A.__call__
>>> a()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'A' object is not callable

It seems like "it just works", but I don't know how it works internally :-)

a() uses a dynamic lookup of a.__class__.__call__, no?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29259>
_______________________________________

From report at bugs.python.org  Fri Jan 13 10:13:29 2017
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 13 Jan 2017 15:13:29 +0000
Subject: [issue29259] Add tp_fastcall to PyTypeObject: support FASTCALL
 calling convention for all callable objects
In-Reply-To: <1484310274.99.0.832491900546.issue29259@psf.upfronthosting.co.za>
Message-ID: <1484320409.95.0.815546451722.issue29259@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Ok, here is a first patch: tp_fastcall.patch

* Add tp_fastcall to PyTypeObject
* Add fastcall_wrapper() and use it for tp_call in PyType_Ready()
* Modify _PyObject_FastCallDict() and _PyObject_FastCallKeywords() to use tp_fastcall if defined
* Modify a lot of types to define tp_fastcall instead of tp_call (to use FASTCALL calling convention)
* Add a few new helper functions:

  - _PyObject_FastCall_Prepend() -- similar to _PyObject_Call_Prepend()
  - _Py_FastCall_FromArgs() -- ugly name, sorry about that, it should be renamed :-)
  - PyArg_UnpackStack() -- similar to PyArg_UnpackTuple()
  - _PyArg_NoStackKeywords() -- similar to _PyArg_NoKeywords

Hum, I should make sure that these new functions are not public. By the way, I should probably rename PyArg_UnpackStack() to _PyArg_UnpackStack().

----------
keywords: +patch
Added file: http://bugs.python.org/file46281/tp_fastcall.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29259>
_______________________________________

From report at bugs.python.org  Fri Jan 13 10:20:55 2017
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 13 Jan 2017 15:20:55 +0000
Subject: [issue26858] android: setting SO_REUSEPORT fails
In-Reply-To: <1461677842.81.0.574068471289.issue26858@psf.upfronthosting.co.za>
Message-ID: <1484320855.32.0.30161851318.issue26858@psf.upfronthosting.co.za>


STINNER Victor added the comment:

> The reason why the changes made by issue 28174 fix only partly the problem should be understood.

The change c1c247cf3488 catchs OSError on sock.setsockopt() in asyncio/base_events.py, whereas the test fails while calling sock.getsockopt() in test_asyncio/tets_events.py. Extract of the test:

    def test_create_server_reuse_port(self):
        proto = MyProto(self.loop)
        f = self.loop.create_server(
            lambda: proto, '0.0.0.0', 0)
        server = self.loop.run_until_complete(f)
        self.assertEqual(len(server.sockets), 1)
        sock = server.sockets[0]
        self.assertFalse(
            sock.getsockopt(
                socket.SOL_SOCKET, socket.SO_REUSEPORT))
        server.close()

The change c1c247cf3488 doesn't touch the code of this unit test.

test_base_events.py also calls directly sock.getsockopt(socket.SOL_SOCKET, socket.SO_REUSEPORT).

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue26858>
_______________________________________

From report at bugs.python.org  Fri Jan 13 10:27:23 2017
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 13 Jan 2017 15:27:23 +0000
Subject: [issue29240] Implementation of the PEP 540: Add a new UTF-8 mode
In-Reply-To: <1484133592.17.0.263024020483.issue29240@psf.upfronthosting.co.za>
Message-ID: <1484321243.82.0.609122535366.issue29240@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Oh, I just noticed that os.environ uses the hardcoded error handler "surrogateescape": it should be replaced with sys.getfilesystemencodeerrors() to support UTF-8 Strict mode.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29240>
_______________________________________

From report at bugs.python.org  Fri Jan 13 10:36:32 2017
From: report at bugs.python.org (INADA Naoki)
Date: Fri, 13 Jan 2017 15:36:32 +0000
Subject: [issue29260] Use designated initializers to define PyTypeObject types
In-Reply-To: <1484317865.58.0.725398845848.issue29260@psf.upfronthosting.co.za>
Message-ID: <1484321792.69.0.531537217374.issue29260@psf.upfronthosting.co.za>


INADA Naoki added the comment:

https://www.python.org/dev/peps/pep-0007/#c-dialect says
"designated initializers (especially nice for type declarations)"

So I think it's allowed explicitly.
To minimize diff, I think we can start using it when adding
slot at bottom of type declaration, or creating completely
new type.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29260>
_______________________________________

From report at bugs.python.org  Fri Jan 13 10:39:20 2017
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 13 Jan 2017 15:39:20 +0000
Subject: [issue29260] Use designated initializers to define PyTypeObject types
In-Reply-To: <1484317865.58.0.725398845848.issue29260@psf.upfronthosting.co.za>
Message-ID: <1484321960.38.0.109073089896.issue29260@psf.upfronthosting.co.za>


STINNER Victor added the comment:

> The widespread use of PyType_FromSpec() will supersede this issue.

Oh, I didn't know PyType_FromSpec(). It is only used for a very few types in the Python stdlib.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29260>
_______________________________________

From report at bugs.python.org  Fri Jan 13 10:45:47 2017
From: report at bugs.python.org (flying sheep)
Date: Fri, 13 Jan 2017 15:45:47 +0000
Subject: [issue29262] Provide a way to check for *real* typing.Union instances
Message-ID: <1484322347.31.0.5780880295.issue29262@psf.upfronthosting.co.za>


New submission from flying sheep:

typing.Union prevents the use of `isinstance` and `issubclass` via a hook.

This is presumably to prevent errors that would arise if someone tried to do issubclass(A, Union[A, B]) or isinstance(A(), Union[A, B]), because Union isn?t specified in the PEP to allow a check like this, and doesn?t implement it. (Instead it throws said error)

However, as far as I can see there is no blessed way to check if an object was returned by Union.__getitem__(). A simple way that works is:

sig = inspect.signature(f)
ann = sig.parameters['arg1'].annotation
is_an_union = isinstance(ann, typing._Union)

but _Union is a private class, and an implementation detail.

is there a blessed way to do this? If not, one should be added.

----------
components: Library (Lib)
messages: 285410
nosy: flying sheep
priority: normal
severity: normal
status: open
title: Provide a way to check for *real* typing.Union instances
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29262>
_______________________________________

From report at bugs.python.org  Fri Jan 13 10:51:14 2017
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 13 Jan 2017 15:51:14 +0000
Subject: [issue29259] Add tp_fastcall to PyTypeObject: support FASTCALL
 calling convention for all callable objects
In-Reply-To: <1484310274.99.0.832491900546.issue29259@psf.upfronthosting.co.za>
Message-ID: <1484322674.89.0.692607587334.issue29259@psf.upfronthosting.co.za>


STINNER Victor added the comment:

tp_fastcall.patch modifies property_descr_get() to use tp_fastcall if available, or use the cached tuple. Once performances are validated (no regression), the cached tuple as to pass position arguments, should go away.

By the way, msg285390 uses the following micro-benchmark:
---
import perf
bench = perf.Runner()
bench.timeit("namedtuple.attr",
             "a.a",
             "from collections import namedtuple as n; a = n('n', 'a b c')(1, 2, 3)",
             duplicate=20)
---

It tests property_descr_get() with operator.itemgetter(). It would be nice to benchmark property_descr_get() with other functions.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29259>
_______________________________________

From report at bugs.python.org  Fri Jan 13 10:58:52 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Fri, 13 Jan 2017 15:58:52 +0000
Subject: [issue29260] Use designated initializers to define PyTypeObject types
In-Reply-To: <1484317865.58.0.725398845848.issue29260@psf.upfronthosting.co.za>
Message-ID: <1484323132.45.0.0189726419548.issue29260@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

Designated initializers already are used in _asynciomodule.c. But this is one of C99 features that are not compatible with C++. Using them makes harder possible using C++ compiler in future.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29260>
_______________________________________

From report at bugs.python.org  Fri Jan 13 11:06:52 2017
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 13 Jan 2017 16:06:52 +0000
Subject: [issue29260] Use designated initializers to define PyTypeObject types
In-Reply-To: <1484317865.58.0.725398845848.issue29260@psf.upfronthosting.co.za>
Message-ID: <1484323612.0.0.751003643165.issue29260@psf.upfronthosting.co.za>


STINNER Victor added the comment:

> Designated initializers already are used in _asynciomodule.c.

Oh, I didn't notice. Since I don't recall any bug report on compilation failing on this module, I guess that it's fine to start to use them in more code.

> But this is one of C99 features that are not compatible with C++. Using them makes harder possible using C++ compiler in future.

Hum, what kind of issue? With which compiler? What is it in the PEP 7 in this case?

----------
nosy: +benjamin.peterson, yselivanov

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29260>
_______________________________________

From report at bugs.python.org  Fri Jan 13 11:09:52 2017
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 13 Jan 2017 16:09:52 +0000
Subject: [issue29263] Implement LOAD_METHOD/CALL_METHOD for C functions
Message-ID: <1484323792.44.0.217164349535.issue29263@psf.upfronthosting.co.za>


New submission from STINNER Victor:

The issue #29259 implements tp_fastcall on method_descriptor (PyMethodDescr_Type). According to http://bugs.python.org/issue26110#msg283093 it would allow to implement LOAD_METHOD and CALL_METHOD for C functions.

----------
messages: 285414
nosy: haypo, inada.naoki, yselivanov
priority: normal
severity: normal
status: open
title: Implement LOAD_METHOD/CALL_METHOD for C functions
type: performance
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29263>
_______________________________________

From report at bugs.python.org  Fri Jan 13 11:10:34 2017
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 13 Jan 2017 16:10:34 +0000
Subject: [issue26110] Speedup method calls 1.2x
In-Reply-To: <1452791347.05.0.923215900831.issue26110@psf.upfronthosting.co.za>
Message-ID: <1484323834.48.0.262274772722.issue26110@psf.upfronthosting.co.za>


STINNER Victor added the comment:

I just created the issue #29263: "Implement LOAD_METHOD/CALL_METHOD for C functions".

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue26110>
_______________________________________

From report at bugs.python.org  Fri Jan 13 11:10:46 2017
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 13 Jan 2017 16:10:46 +0000
Subject: [issue29263] Implement LOAD_METHOD/CALL_METHOD for C functions
In-Reply-To: <1484323792.44.0.217164349535.issue29263@psf.upfronthosting.co.za>
Message-ID: <1484323846.73.0.794448956582.issue29263@psf.upfronthosting.co.za>


Changes by STINNER Victor <victor.stinner at gmail.com>:


----------
dependencies: +Add tp_fastcall to PyTypeObject: support FASTCALL calling convention for all callable objects

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29263>
_______________________________________

From report at bugs.python.org  Fri Jan 13 11:22:59 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Fri, 13 Jan 2017 16:22:59 +0000
Subject: [issue29259] Add tp_fastcall to PyTypeObject: support FASTCALL
 calling convention for all callable objects
In-Reply-To: <1484310274.99.0.832491900546.issue29259@psf.upfronthosting.co.za>
Message-ID: <1484324579.75.0.0099156725903.issue29259@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

PyArg_UnpackStack() is declared as in the limited API since 3.3. If you want to add PyArg_UnpackStack() to the limited API, define it as added in 3.7.

For compatibility with extensions built with older Pythons you should define new type flag and read tp_fastcall only if the flag is set. See for example Py_TPFLAGS_HAVE_FINALIZE.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29259>
_______________________________________

From report at bugs.python.org  Fri Jan 13 11:37:21 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Fri, 13 Jan 2017 16:37:21 +0000
Subject: [issue29260] Use designated initializers to define PyTypeObject types
In-Reply-To: <1484317865.58.0.725398845848.issue29260@psf.upfronthosting.co.za>
Message-ID: <1484325441.34.0.545575100322.issue29260@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

There is nothing about this in PEP 7, but I think it would be better to use common subset of C and C++. This would allow to migrate to the subset of C++ in future. It is sad that PEP 7 allows C99 features not compatible with C++.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29260>
_______________________________________

From report at bugs.python.org  Fri Jan 13 11:47:07 2017
From: report at bugs.python.org (Thomas Nyberg)
Date: Fri, 13 Jan 2017 16:47:07 +0000
Subject: =?utf-8?q?=5Bissue24875=5D_pyvenv_doesn=C2=B4t_install_PIP_inside_a_new_v?=
 =?utf-8?q?env_with_--system-site-package?=
In-Reply-To: <1439699040.07.0.199956094974.issue24875@psf.upfronthosting.co.za>
Message-ID: <1484326027.83.0.026660687438.issue24875@psf.upfronthosting.co.za>


Thomas Nyberg added the comment:

Hi Vinay,

You should probably upload a patch with the changes you made (however trivial) if you want that version considered. Makes it easier to comment and can then point to it for others to consider.

Cheers,
Thomas

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24875>
_______________________________________

From report at bugs.python.org  Fri Jan 13 11:48:41 2017
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 13 Jan 2017 16:48:41 +0000
Subject: [issue29259] Add tp_fastcall to PyTypeObject: support FASTCALL
 calling convention for all callable objects
In-Reply-To: <1484310274.99.0.832491900546.issue29259@psf.upfronthosting.co.za>
Message-ID: <1484326121.82.0.264560375132.issue29259@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Ok, I pushed a new commit "Add Py_TPFLAGS_HAVE_FASTCALL flag" in my Git branch.

(By the way, I'm using the Git branch as a working copy, it can change anytime to get a nice list of commits. If you want reliable changes, use patches attached to this issue.)


Serhiy Storchaka!:
> For compatibility with extensions built with older Pythons you should define new type flag and read tp_fastcall only if the flag is set. See for example Py_TPFLAGS_HAVE_FINALIZE.

Wait, there is maybe an issue with the stable API in my design :-/

Let's say that a builtin type NEW defines tp_fastcall and uses fastcall_wrapper() for tp_call. A type OLD inherits from NEW and don't override tp_call (not tp_fastcall). Type OLD comes from an extension compiled with Python 3.6 and use the stable ABI. Does the type OLD contain the field tp_fastcall in memory? Is it possible to copy the tp_fastcall filed from type NEW into the type OLD?

If not, the fastcall_wrapper() should be smarter to not get tp_fastcall from the type OLD but follow the MRO to find the first type with the Py_TPFLAGS_HAVE_FASTCALL flag. Is it possible that this short loop to find has a cost on performances?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29259>
_______________________________________

From report at bugs.python.org  Fri Jan 13 11:53:33 2017
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 13 Jan 2017 16:53:33 +0000
Subject: [issue29259] Add tp_fastcall to PyTypeObject: support FASTCALL
 calling convention for all callable objects
In-Reply-To: <1484310274.99.0.832491900546.issue29259@psf.upfronthosting.co.za>
Message-ID: <1484326413.4.0.620244199293.issue29259@psf.upfronthosting.co.za>


STINNER Victor added the comment:

I ran benchmarks on tp_fastcall.patch. Hum, it seems like there is a performance issue somewhere :-( Almost all benchmarks are much slower with the patch.

haypo at speed-python$ python3 -m perf compare_to 2017-01-11_00-07-default-b9404639a18c.json tp_fastcall_ref_b9404639a18c.json -G
Slower (63):
- unpickle_pure_python: 671 us +- 16 us -> 952 us +- 9 us: 1.42x slower (+42%)
- pickle_pure_python: 1.02 ms +- 0.02 ms -> 1.40 ms +- 0.02 ms: 1.37x slower (+37%)
- telco: 16.6 ms +- 0.6 ms -> 22.7 ms +- 0.3 ms: 1.37x slower (+37%)
- xml_etree_generate: 215 ms +- 3 ms -> 290 ms +- 7 ms: 1.35x slower (+35%)
- hexiom: 17.7 ms +- 0.2 ms -> 23.6 ms +- 0.1 ms: 1.33x slower (+33%)
- xml_etree_process: 182 ms +- 3 ms -> 240 ms +- 4 ms: 1.32x slower (+32%)
- genshi_text: 69.0 ms +- 0.8 ms -> 90.4 ms +- 0.8 ms: 1.31x slower (+31%)
- logging_simple: 27.5 us +- 0.4 us -> 35.8 us +- 0.4 us: 1.30x slower (+30%)
(...)

Faster (1):
- spectral_norm: 271 ms +- 2 ms -> 257 ms +- 2 ms: 1.06x faster (-5%)

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29259>
_______________________________________

From report at bugs.python.org  Fri Jan 13 12:02:05 2017
From: report at bugs.python.org (JGoutin)
Date: Fri, 13 Jan 2017 17:02:05 +0000
Subject: [issue29241] sys._enablelegacywindowsfsencoding() don't apply to
 os.fsencode and os.fsdecode
In-Reply-To: <1484137681.59.0.0301192140292.issue29241@psf.upfronthosting.co.za>
Message-ID: <1484326925.99.0.619121642157.issue29241@psf.upfronthosting.co.za>


JGoutin added the comment:

Personally, I call "sys._enablelegacywindowsfsencoding()" for only one reason :
Temporary fixing issues with some third party libraries which use C code for files I/O (With filename as "mbcs" encoded bytes internally).

Theses libraries generally call "filename.encode(sys.getfilesystemencoding())" or "os.fsencode(filename)" before sending filenames from Python to C code.

Actually, I didn't see any side effect for using this function. Maybe because I call it at start before anything else.

Using the environment variable is not easy in my case.


I can look if encoding caching in fsencode is efficient (On Windows). And eventually propose a code change for this.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29241>
_______________________________________

From report at bugs.python.org  Fri Jan 13 12:12:55 2017
From: report at bugs.python.org (Zachary Ware)
Date: Fri, 13 Jan 2017 17:12:55 +0000
Subject: [issue29261] Missing venv/scripts/common after "make install"
In-Reply-To: <1484319985.48.0.948212892156.issue29261@psf.upfronthosting.co.za>
Message-ID: <1484327575.41.0.915754462058.issue29261@psf.upfronthosting.co.za>


Zachary Ware added the comment:

LGTM.

----------
assignee:  -> vinay.sajip
nosy: +vinay.sajip, zach.ware
stage:  -> commit review

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29261>
_______________________________________

From report at bugs.python.org  Fri Jan 13 12:13:17 2017
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 13 Jan 2017 17:13:17 +0000
Subject: [issue29241] sys._enablelegacywindowsfsencoding() don't apply to
 os.fsencode and os.fsdecode
In-Reply-To: <1484137681.59.0.0301192140292.issue29241@psf.upfronthosting.co.za>
Message-ID: <1484327597.3.0.999575598609.issue29241@psf.upfronthosting.co.za>


STINNER Victor added the comment:

> Temporary fixing issues with some third party libraries which use C code for files I/O (With filename as "mbcs" encoded bytes internally).
>
> Theses libraries generally call "filename.encode(sys.getfilesystemencoding())" or "os.fsencode(filename)" before sending filenames from Python to C code.

Hum, Python lacks a function to encode to/decode from the ANSI code page, something like codecs.code_page_encode() / code_page_decode() with CP_ACP. It would allow to get the same encoding in UTF-8 and legacy modes.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29241>
_______________________________________

From report at bugs.python.org  Fri Jan 13 12:14:27 2017
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 13 Jan 2017 17:14:27 +0000
Subject: [issue29241] sys._enablelegacywindowsfsencoding() don't apply to
 os.fsencode and os.fsdecode
In-Reply-To: <1484137681.59.0.0301192140292.issue29241@psf.upfronthosting.co.za>
Message-ID: <1484327667.95.0.304109895074.issue29241@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Hum, it was long time ago since I worked on Windows. Well, Python has a "mbcs" codec which uses the ANSI code page which exists like "forever". These libraries should be patched to use "mbcs" instead of sys.getfilesystemencoding().

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29241>
_______________________________________

From report at bugs.python.org  Fri Jan 13 12:15:44 2017
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 13 Jan 2017 17:15:44 +0000
Subject: [issue29259] Add tp_fastcall to PyTypeObject: support FASTCALL
 calling convention for all callable objects
In-Reply-To: <1484326413.4.0.620244199293.issue29259@psf.upfronthosting.co.za>
Message-ID: <CAMpsgwbz1z5eO3BV0r-tFVKy++cAdbF474q7p3+xJeks2ZajfQ@mail.gmail.com>


STINNER Victor added the comment:

Strange, I'm unable to reproduce the result on a different computer:

haypo at smithers$ ./python -m perf compare_to ref.json tp_fastcall.json  -v
Median +- std dev: [ref] 20.0 ms +- 0.5 ms -> [tp_fastcall] 20.0 ms +-
0.6 ms: 1.00x slower (+0%)
Not significant!

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29259>
_______________________________________

From report at bugs.python.org  Fri Jan 13 12:16:05 2017
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 13 Jan 2017 17:16:05 +0000
Subject: [issue29259] Add tp_fastcall to PyTypeObject: support FASTCALL
 calling convention for all callable objects
In-Reply-To: <CAMpsgwbz1z5eO3BV0r-tFVKy++cAdbF474q7p3+xJeks2ZajfQ@mail.gmail.com>
Message-ID: <CAMpsgwauOKVov2WeUcSNQwCTrWQ7+M6SjZf+hVtdhzmtbvzyjg@mail.gmail.com>


STINNER Victor added the comment:

> Strange, I'm unable to reproduce the result on a different computer:
>
> haypo at smithers$ ./python -m perf compare_to ref.json tp_fastcall.json  -v
> Median +- std dev: [ref] 20.0 ms +- 0.5 ms -> [tp_fastcall] 20.0 ms +-
> 0.6 ms: 1.00x slower (+0%)
> Not significant!

Oops, I forgot to mention that I tried the telco benchmark.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29259>
_______________________________________

From report at bugs.python.org  Fri Jan 13 12:22:31 2017
From: report at bugs.python.org (Segev Finer)
Date: Fri, 13 Jan 2017 17:22:31 +0000
Subject: [issue19764] subprocess: use PROC_THREAD_ATTRIBUTE_HANDLE_LIST with
 STARTUPINFOEX on Windows Vista
In-Reply-To: <1385370115.99.0.757096187315.issue19764@psf.upfronthosting.co.za>
Message-ID: <1484328151.17.0.58765112017.issue19764@psf.upfronthosting.co.za>


Segev Finer added the comment:

I have read some of https://github.com/rprichard/win32-console-docs and it documents quite a bunch of nastiness with PROC_THREAD_ATTRIBUTE_HANDLE_LIST in Windows Vista/7. Windows is so much fun sometimes :P

Essentially console handles in Windows before Windows 8 are user mode handles and not real kernel handles. Those user mode handles are inherited by a different mechanism than kernel handles and regardless of PROC_THREAD_ATTRIBUTE_HANDLE_LIST, and if passed in PROC_THREAD_ATTRIBUTE_HANDLE_LIST will cause it to fail in weird ways. Those user mode console handles have the lower two bits set. The lower two bits in Windows are reserved for tagging such special handles.

Also in all versions you can't pass in an empty handle list, but a list with just a NULL handle works fine.

See: https://github.com/rprichard/win32-console-docs/blob/master/src/HandleTests/CreateProcess_InheritList.cc

I attached a version of the patch with a hack around those issues based on what I read, but I can't test that it actually fixes the issues since I don't have a Windows Vista or 7 system around.

----------
Added file: http://bugs.python.org/file46282/windows-subprocess-close-fds-v3-vista7-hack.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue19764>
_______________________________________

From report at bugs.python.org  Fri Jan 13 12:27:39 2017
From: report at bugs.python.org (Brian Vandenberg)
Date: Fri, 13 Jan 2017 17:27:39 +0000
Subject: [issue29264] sparc/ffi.c:440 error: 'asm' undeclared
Message-ID: <1484328459.12.0.655564355741.issue29264@psf.upfronthosting.co.za>


New submission from Brian Vandenberg:

When building pythong 3.6.0 on solaris 10/sparc I'm seeing the following error:

Modules/_ctypes/libffi/src/sparc/ffi.c:440:8: error: 'asm' undeclared (first use in this function)
(...)

If I force it to use -std=gnu99 then it seems to compile fine.

----------
components: ctypes
messages: 285428
nosy: phantal
priority: normal
severity: normal
status: open
title: sparc/ffi.c:440 error: 'asm' undeclared
type: compile error
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29264>
_______________________________________

From report at bugs.python.org  Fri Jan 13 12:31:51 2017
From: report at bugs.python.org (JGoutin)
Date: Fri, 13 Jan 2017 17:31:51 +0000
Subject: [issue29241] sys._enablelegacywindowsfsencoding() don't apply to
 os.fsencode and os.fsdecode
In-Reply-To: <1484137681.59.0.0301192140292.issue29241@psf.upfronthosting.co.za>
Message-ID: <1484328711.35.0.681196300149.issue29241@psf.upfronthosting.co.za>


JGoutin added the comment:

Yes, I reported this encoding issue to some of them.

But, there is still some problems : 
- Some libraries are not updated frequently (Or not still maintained), and still use fsencode.
- Tests and CI don't see this problem if they don't have a test case for filename with accents or other uncommon characters in english.

This problem will not be easy to eliminate totally...

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29241>
_______________________________________

From report at bugs.python.org  Fri Jan 13 12:48:34 2017
From: report at bugs.python.org (Brian Vandenberg)
Date: Fri, 13 Jan 2017 17:48:34 +0000
Subject: [issue29265] test suite is attempting to spawn 258 child processes to
 run tests
Message-ID: <1484329714.36.0.262857115906.issue29265@psf.upfronthosting.co.za>


New submission from Brian Vandenberg:

I'm attempting to build python 3.6.0 on sparc/solaris 10.  After the initial configure/compile complete I ran "make test" and I see:

$ make test
running build
running build_ext
(...)
running build_scripts
copying and adjusting (...)
changing mode of (...)
renaming (...)
(...)
Run tests in parallel using 258 child processes


I'm fairly sure the issue stems from the fact that each core on the machine has 8 "threads" and there's 32 cores (for a total of 256 virtual cores).

Each core can execute 8 parallel tasks only in very specific circumstances.  It's intended for use by things like lapack/atlas where you might be doing many computations on the same set of data.

Outside of these more restricted circumstances each core can only handle 2 parallel tasks (or so I gathered from the documentation), so at best this machine could handle 64 backgrounded jobs though I normally restrict my builds to the actual core count or less.

The most common way to get a "realistic" core count on these machines from shell scripts is:

$ core_count=`kstat -m cpu_info | grep core_id | sort -u | wc -l`

... though I'm not sure how the test suite is determining the core count.  I didn't see any mention of "kstat" anywhere.

----------
components: Tests
messages: 285430
nosy: phantal
priority: normal
severity: normal
status: open
title: test suite is attempting to spawn 258 child processes to run tests
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29265>
_______________________________________

From report at bugs.python.org  Fri Jan 13 12:51:08 2017
From: report at bugs.python.org (Brian Vandenberg)
Date: Fri, 13 Jan 2017 17:51:08 +0000
Subject: [issue29265] test suite is attempting to spawn 258 child processes to
 run tests
In-Reply-To: <1484329714.36.0.262857115906.issue29265@psf.upfronthosting.co.za>
Message-ID: <1484329868.52.0.737061995633.issue29265@psf.upfronthosting.co.za>


Brian Vandenberg added the comment:

I forgot to mention, this wasn't an issue in 3.5.1 though I never did check how many jobs it was using.

I ran into other issues building that version and moved to a newer version because at least one of them (logging test race condition) was fixed after 3.5.1.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29265>
_______________________________________

From report at bugs.python.org  Fri Jan 13 12:54:06 2017
From: report at bugs.python.org (Brian Vandenberg)
Date: Fri, 13 Jan 2017 17:54:06 +0000
Subject: [issue29265] test suite is attempting to spawn 258 child processes to
 run tests
In-Reply-To: <1484329714.36.0.262857115906.issue29265@psf.upfronthosting.co.za>
Message-ID: <1484330046.39.0.412254264169.issue29265@psf.upfronthosting.co.za>


Brian Vandenberg added the comment:

This is odd.  I just went back and re-ran 3.5.1 to see how many cores and it's having the same problem now.  So, scratch that last coment.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29265>
_______________________________________

From report at bugs.python.org  Fri Jan 13 13:02:48 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Fri, 13 Jan 2017 18:02:48 +0000
Subject: [issue29265] test suite is attempting to spawn 258 child processes to
 run tests
In-Reply-To: <1484329714.36.0.262857115906.issue29265@psf.upfronthosting.co.za>
Message-ID: <1484330568.5.0.0861719476642.issue29265@psf.upfronthosting.co.za>


Changes by Serhiy Storchaka <storchaka+cpython at gmail.com>:


----------
nosy: +haypo

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29265>
_______________________________________

From report at bugs.python.org  Fri Jan 13 13:17:54 2017
From: report at bugs.python.org (Lisa Roach)
Date: Fri, 13 Jan 2017 18:17:54 +0000
Subject: [issue28911] Clarify the behaviour of assert_called_once_with
In-Reply-To: <1481233700.68.0.637550951007.issue28911@psf.upfronthosting.co.za>
Message-ID: <1484331474.46.0.558237777572.issue28911@psf.upfronthosting.co.za>


Lisa Roach added the comment:

It took me a little while to wrap my brain around this, but you are definitely right that the documentation is not sufficient, your changes are an improvement. 

My wonder is, should we change the documentation or be looking at the code itself? I have always interpreted the method as asserting that only one call was made with the matching signature. If I want to test for call count I can just assert mock.call_count == 1.

The code could instead look like this: 

def assert_called_once_with(_mock_self, *args, **kwargs):
    self = _mock_self
    if not self.mock_calls.count(call(*args, **kwargs)) == 1:
        msg = ("Expected '%s' to be called once with %r %r. Called %s times."          % (self._mock_name or 'mock', args, kwargs, self.mock_calls.count(call(*args,  **kwargs))))
        raise AssertionError(msg)


Then again, if users have been using this to assert that the call_count is one (which is likely since it is in the examples documentation), this change would break backwards functionality.

Thoughts?

----------
nosy: +lisroach

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28911>
_______________________________________

From report at bugs.python.org  Fri Jan 13 13:43:15 2017
From: report at bugs.python.org (Brian Vandenberg)
Date: Fri, 13 Jan 2017 18:43:15 +0000
Subject: [issue29266] test_create_connection_service_name fails if "http"
 isn't listed in /etc/services
Message-ID: <1484332995.5.0.803330803747.issue29266@psf.upfronthosting.co.za>


New submission from Brian Vandenberg:

One of our solaris machines doesn't have an entry in /etc/services for "http".  This is causing test_create_connection_service_name to fail.

In my case I can just ignore that particular failure, but as a fix you might consider overtly getting valid service names from the host to test.  Additionally, if you don't already have it you might consider adding a test that validates behavior when using a non-existent service.

----------
components: Tests
messages: 285434
nosy: phantal
priority: normal
severity: normal
status: open
title: test_create_connection_service_name fails if "http" isn't listed in /etc/services
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29266>
_______________________________________

From report at bugs.python.org  Fri Jan 13 13:52:29 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Fri, 13 Jan 2017 18:52:29 +0000
Subject: [issue29197] Remove os.path.splitunc()
In-Reply-To: <1483809458.59.0.701425626614.issue29197@psf.upfronthosting.co.za>
Message-ID: <1484333549.78.0.452151251275.issue29197@psf.upfronthosting.co.za>


Changes by Serhiy Storchaka <storchaka+cpython at gmail.com>:


----------
assignee:  -> serhiy.storchaka

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29197>
_______________________________________

From report at bugs.python.org  Fri Jan 13 13:53:57 2017
From: report at bugs.python.org (Brian Vandenberg)
Date: Fri, 13 Jan 2017 18:53:57 +0000
Subject: [issue29267] Cannot override some flags in CFLAGS from the
 command-line
Message-ID: <1484333637.07.0.801577004436.issue29267@psf.upfronthosting.co.za>


New submission from Brian Vandenberg:

Due to issue 29264 I was attempting to override the build default "-std=c99" with:

    /path/to/configure (...) CFLAGS=-std=gnu99

... however, the configure script is written like this:

    CFLAGS_NODIST="$CFLAGS_NODIST -std=c99"

This causes it to always override my attempt to override the build default.

----------
components: Build
messages: 285435
nosy: phantal
priority: normal
severity: normal
status: open
title: Cannot override some flags in CFLAGS from the command-line
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29267>
_______________________________________

From report at bugs.python.org  Fri Jan 13 13:55:45 2017
From: report at bugs.python.org (Roundup Robot)
Date: Fri, 13 Jan 2017 18:55:45 +0000
Subject: [issue29197] Remove os.path.splitunc()
In-Reply-To: <1483809458.59.0.701425626614.issue29197@psf.upfronthosting.co.za>
Message-ID: <20170113185542.3089.58172.2F736476@psf.io>


Roundup Robot added the comment:

New changeset 2be38927ea61 by Serhiy Storchaka in branch 'default':
Issue #29197: Removed deprecated function ntpath.splitunc().
https://hg.python.org/cpython/rev/2be38927ea61

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29197>
_______________________________________

From report at bugs.python.org  Fri Jan 13 13:57:02 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Fri, 13 Jan 2017 18:57:02 +0000
Subject: [issue29197] Remove os.path.splitunc()
In-Reply-To: <1483809458.59.0.701425626614.issue29197@psf.upfronthosting.co.za>
Message-ID: <1484333822.79.0.711030612498.issue29197@psf.upfronthosting.co.za>


Changes by Serhiy Storchaka <storchaka+cpython at gmail.com>:


----------
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29197>
_______________________________________

From report at bugs.python.org  Fri Jan 13 14:18:44 2017
From: report at bugs.python.org (Roundup Robot)
Date: Fri, 13 Jan 2017 19:18:44 +0000
Subject: [issue29261] Missing venv/scripts/common after "make install"
In-Reply-To: <1484319985.48.0.948212892156.issue29261@psf.upfronthosting.co.za>
Message-ID: <20170113191840.56631.4017.845A92ED@psf.io>


Roundup Robot added the comment:

New changeset 077681e35577 by Vinay Sajip in branch '3.5':
Fixes #29261: added venv/scripts/common to LIBSUBDIRS.
https://hg.python.org/cpython/rev/077681e35577

New changeset f20b2073dd4a by Vinay Sajip in branch '3.6':
Fixes #29261: merged fix from 3.5.
https://hg.python.org/cpython/rev/f20b2073dd4a

New changeset b76c42656639 by Vinay Sajip in branch 'default':
Closed #29261: merged fix from 3.6.
https://hg.python.org/cpython/rev/b76c42656639

----------
nosy: +python-dev
resolution:  -> fixed
stage: commit review -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29261>
_______________________________________

From report at bugs.python.org  Fri Jan 13 14:23:44 2017
From: report at bugs.python.org (Vinay Sajip)
Date: Fri, 13 Jan 2017 19:23:44 +0000
Subject: =?utf-8?q?=5Bissue24875=5D_pyvenv_doesn=C2=B4t_install_PIP_inside_a_new_v?=
 =?utf-8?q?env_with_--system-site-package?=
In-Reply-To: <1439699040.07.0.199956094974.issue24875@psf.upfronthosting.co.za>
Message-ID: <1484335424.61.0.662473262939.issue24875@psf.upfronthosting.co.za>


Vinay Sajip added the comment:

Patch uploaded herewith.

----------
assignee:  -> vinay.sajip
stage:  -> patch review
Added file: http://bugs.python.org/file46283/issue-24875-01.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24875>
_______________________________________

From report at bugs.python.org  Fri Jan 13 14:43:48 2017
From: report at bugs.python.org (Brian Vandenberg)
Date: Fri, 13 Jan 2017 19:43:48 +0000
Subject: [issue29268] test_spwd fails on solaris using NIS users
Message-ID: <1484336628.67.0.439917676346.issue29268@psf.upfronthosting.co.za>


New submission from Brian Vandenberg:

On all our solaris 10 machines when I run a simple test program it never reports a failure when calling getspnam:

   #include <shadow.h>
   #include <stdio.h>
   int main( int, char** ) {
     spwd *asdf = getspnam( "some_user" );
     if( NULL == sdf ) {
       perror( "getspnam" );
     }
     return 0;
   }

If I run the above program on our linux boxes it fails as expected, but on our solaris machines it produces the same information you'd see running "ypcat passwd | grep some_user".

I suspect either there's a bug in the solaris implementation of getpwnam() or perhaps there's a configuration issue on our solaris machines, though it's also possible this is just how it behaves in Solaris (at least with NIS).

As to whether anything should change for test_spwd -- I suspect this will get closed as "won't fix" but at least this report may help anyone else running into this failure.

----------
components: Tests
messages: 285439
nosy: phantal
priority: normal
severity: normal
status: open
title: test_spwd fails on solaris using NIS users
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29268>
_______________________________________

From report at bugs.python.org  Fri Jan 13 15:55:24 2017
From: report at bugs.python.org (Brian Vandenberg)
Date: Fri, 13 Jan 2017 20:55:24 +0000
Subject: [issue29269] test_socket failing in solaris
Message-ID: <1484340924.46.0.120687210952.issue29269@psf.upfronthosting.co.za>


New submission from Brian Vandenberg:

I started looking into this failure to see if I could figure out why but it looks like I'd have to spend more time than I have available to figure out the cause.

Environment/setup:
* air-gapped network (no internet access)
* sparc / Solaris 10
* Built with gcc 6.3.0
* Altered configure script to change -std=c99 to -std=gnu99 (see issue 29264)
* The only configure flags used were --prefix and --with-universal-archs=all


When I run test_socket I see the following 4 failures; please note, I'm hand typing the results so I may typo something:


ERROR: testCount (test.test_socket.SendfileUsingSendfileTest)
Traceback:
  File "(...)/test_socket.py", line 5248, in testCount
  File "(...)/test_socket.py", line 5151, in recv_data
MemoryError

Error: testCount (test.test_socket.SendfileUsingSendfileTest)
Traceback:
  File "(...)/test_socket.py", line 277, in _tearDown
  File "(...)/test_socket.py", line 289, in clientRun
  File "(...)/test_socket.py", line 5241, in _testCount
  File "(...)/Lib/socket.py", line 296, in _sendfile_use_sendfile
socket.timeout: timed out

Error: testWithTimeout (test.test_socket.SendfileUsingSendfileTest)
Traceback:
  File "(...)/test_socket.py", line 5318, in testWithTimeout
    data = self.recv_data(conn)
  File "(...)/test_socket.py", line 5151, in recv_data
    chunk = conn.recv(self.BUFSIZE)
MemoryError

Error: testWithTimeout (test.test_socket.SendfileUsingSendfileTest)
Traceback:
  File "(...)/test_socket.py", line 277, in _tearDown
    raise exc
  File "(...)/test_socket.py", line 289, in clientRun
    test_func()
  File "(...)/test_socket.py", line 5313, in _testWithTimeout
    sent = meth(file)
  File "(...)/Lib/socket.py", line 296, in _sendfile_use_sendfile
socket.timeout: timed out

Error: testCountWithOffset (test.test_socket.SendfileUsingSendfileTest)
Traceback:
  File "(...)/test_socket.py", line 5287, in testCountWithOffset
    self.assertEqual(len(data), count)
AssertionError: 4376231 != 100007

Ran 539 tests in 69.166s

FAILED (failures=1, errors=4, skipped=324)
test test_socket failed

----------
components: Tests
messages: 285440
nosy: phantal
priority: normal
severity: normal
status: open
title: test_socket failing in solaris
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29269>
_______________________________________

From report at bugs.python.org  Fri Jan 13 15:59:38 2017
From: report at bugs.python.org (R. David Murray)
Date: Fri, 13 Jan 2017 20:59:38 +0000
Subject: [issue29265] os.cpu_count is problematic on sparc/solaris
In-Reply-To: <1484329714.36.0.262857115906.issue29265@psf.upfronthosting.co.za>
Message-ID: <1484341178.17.0.664121397124.issue29265@psf.upfronthosting.co.za>


R. David Murray added the comment:

You don't know it, but you are actually reporting a possible sparc/Solaris specific (I think) bug against os.cpu_count.  There was some discussion about how cpu_count might be problematic in this regard.

It doesn't cause any real problem with the tests, though.  I routinely run with -j40 on my 2 cpu test box because the test run completes faster that way due to the way many tests spend time waiting for various things.

----------
components: +Library (Lib) -Tests
nosy: +r.david.murray

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29265>
_______________________________________

From report at bugs.python.org  Fri Jan 13 16:07:18 2017
From: report at bugs.python.org (R. David Murray)
Date: Fri, 13 Jan 2017 21:07:18 +0000
Subject: [issue29266] test_create_connection_service_name fails if "http"
 isn't listed in /etc/services
In-Reply-To: <1484332995.5.0.803330803747.issue29266@psf.upfronthosting.co.za>
Message-ID: <1484341638.39.0.475701454853.issue29266@psf.upfronthosting.co.za>


R. David Murray added the comment:

It looks like that test already does so (it expects an OSError when passed the service name 'nonsense').

----------
nosy: +r.david.murray

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29266>
_______________________________________

From report at bugs.python.org  Fri Jan 13 16:07:32 2017
From: report at bugs.python.org (R. David Murray)
Date: Fri, 13 Jan 2017 21:07:32 +0000
Subject: [issue29266] test_create_connection_service_name fails if "http"
 isn't listed in /etc/services
In-Reply-To: <1484332995.5.0.803330803747.issue29266@psf.upfronthosting.co.za>
Message-ID: <1484341652.31.0.365054040343.issue29266@psf.upfronthosting.co.za>


Changes by R. David Murray <rdmurray at bitdance.com>:


----------
components: +asyncio
nosy: +gvanrossum, yselivanov

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29266>
_______________________________________

From report at bugs.python.org  Fri Jan 13 16:30:06 2017
From: report at bugs.python.org (Martin Panter)
Date: Fri, 13 Jan 2017 21:30:06 +0000
Subject: [issue18235] _sysconfigdata.py wrong on AIX installations
In-Reply-To: <1371429201.94.0.372851990361.issue18235@psf.upfronthosting.co.za>
Message-ID: <1484343006.1.0.529545565621.issue18235@psf.upfronthosting.co.za>


Martin Panter added the comment:

So are you saying you tried patching Python 2 and/or 3, but did not see any relevant change at all?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue18235>
_______________________________________

From report at bugs.python.org  Fri Jan 13 16:30:15 2017
From: report at bugs.python.org (Martin Panter)
Date: Fri, 13 Jan 2017 21:30:15 +0000
Subject: [issue25825] AIX shared library extension modules installation
 broken: wrong dir names
In-Reply-To: <1449602974.51.0.754649695687.issue25825@psf.upfronthosting.co.za>
Message-ID: <1484343015.11.0.0797917733775.issue25825@psf.upfronthosting.co.za>


Changes by Martin Panter <vadmium+py at gmail.com>:


----------
title: AIX shared library extension modules installation broken -> AIX shared library extension modules installation broken: wrong dir names

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue25825>
_______________________________________

From report at bugs.python.org  Fri Jan 13 16:30:20 2017
From: report at bugs.python.org (Martin Panter)
Date: Fri, 13 Jan 2017 21:30:20 +0000
Subject: [issue16189] config/ld_so_aix not found: old dir name
In-Reply-To: <1349884360.41.0.476274191409.issue16189@psf.upfronthosting.co.za>
Message-ID: <1484343020.42.0.147464099425.issue16189@psf.upfronthosting.co.za>


Changes by Martin Panter <vadmium+py at gmail.com>:


----------
title: ld_so_aix not found -> config/ld_so_aix not found: old dir name

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue16189>
_______________________________________

From report at bugs.python.org  Fri Jan 13 17:10:22 2017
From: report at bugs.python.org (Dave Jones)
Date: Fri, 13 Jan 2017 22:10:22 +0000
Subject: [issue29270] super call in ctypes sub-class fails in 3.6
Message-ID: <1484345422.14.0.967631121608.issue29270@psf.upfronthosting.co.za>


New submission from Dave Jones:

While investigating a bug report in one of my libraries (https://github.com/waveform80/picamera/issues/355) I've come across a behaviour that appears in Python 3.6 but not prior versions. Specifically, calling super() in a sub-class of a ctypes scalar type appears to fail at the class definition stage. A minimal test case is as follows:

    import ctypes as ct

    class SuperTest(ct.c_uint32):
        def __repr__(self):
            return super().__repr__()

This works happily under python 3.2, 3.4, and 3.5 (that I've tested), and also under 2.7 (with the appropriate modification to super's arguments). However, under 3.6 it elicits the following exception:

    Traceback (most recent call last):
      File "py36_ctypes.py", line 3, in <module>
        class SuperTest(ct.c_uint32):
    TypeError: __class__ set to <class '__main__.SuperTest'> defining 'SuperTest' as <class '__main__.SuperTest'>

Reading through the "What's New" list in 3.6, I thought this might be something to do with the PEP-487 implementation (given it modified class construction), but having read through the PEP and associated patches I'm not so sure as I can't see anything that affects the setting of the "__class__" attribute (but don't rule it out on that basis; I'm no expert!).

I'll admit that sub-classing one of ctypes' scalar types is a little odd, but given this works in prior versions and there doesn't appear to be anything in the documentation banning the practice (that I've found?) this might constitute a bug?

----------
components: ctypes
messages: 285444
nosy: Dave Jones
priority: normal
severity: normal
status: open
title: super call in ctypes sub-class fails in 3.6
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29270>
_______________________________________

From report at bugs.python.org  Fri Jan 13 17:11:24 2017
From: report at bugs.python.org (Yury Selivanov)
Date: Fri, 13 Jan 2017 22:11:24 +0000
Subject: [issue29271] Task.current_task(None) returns unexpected result
Message-ID: <1484345484.86.0.127501941011.issue29271@psf.upfronthosting.co.za>


New submission from Yury Selivanov:

Quoting Alberto Berdi from python-tulip mailing list:

working with the Python 3.6 asyncio implemented in C, i've got what is
for me a strange behavior of the asyncio.Task.current task
function. Executing the following test:

 import asyncio

 async def coro():
     print(asyncio.Task.current_task())
     print(asyncio.Task.current_task(None))
     print(asyncio.Task.current_task(loop=asyncio.get_event_loop()))

 loop = asyncio.get_event_loop()
 loop.run_until_complete(coro())

in Py3.5, I obtain the result:

<Task pending coro=<coro() running at test_task.py:4> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:176]>
<Task pending coro=<coro() running at test_task.py:5> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:176]>
<Task pending coro=<coro() running at test_task.py:6> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:176]>

In Py3.6, i get:

<Task pending coro=<coro() running at test_task.py:4> cb=[_run_until_complete_cb() at /usr/lib/python3.6/asyncio/base_events.py:176]>
None
<Task pending coro=<coro() running at test_task.py:6> cb=[_run_until_complete_cb() at /usr/lib/python3.6/asyncio/base_events.py:176]>

What's going on here? It's still right to expect the same result from the three calls or something has changed?
The documentation for the Task.current_task() function seems unchanged between the two version.

My python3.6 has version:

$ python3.6 -VV
Python 3.6.0 (default, Dec 29 2016, 04:29:02) 
[GCC 6.2.1 20161215]

----------
assignee: yselivanov
components: asyncio
messages: 285445
nosy: gvanrossum, yselivanov
priority: normal
severity: normal
stage: patch review
status: open
title: Task.current_task(None) returns unexpected result
type: behavior
versions: Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29271>
_______________________________________

From report at bugs.python.org  Fri Jan 13 17:17:41 2017
From: report at bugs.python.org (Yury Selivanov)
Date: Fri, 13 Jan 2017 22:17:41 +0000
Subject: [issue29271] Task.current_task(None) returns unexpected result
In-Reply-To: <1484345484.86.0.127501941011.issue29271@psf.upfronthosting.co.za>
Message-ID: <1484345861.93.0.284558957179.issue29271@psf.upfronthosting.co.za>


Changes by Yury Selivanov <yselivanov at gmail.com>:


----------
keywords: +patch
nosy: +inada.naoki
Added file: http://bugs.python.org/file46284/task.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29271>
_______________________________________

From report at bugs.python.org  Fri Jan 13 17:26:22 2017
From: report at bugs.python.org (Alberto Berti)
Date: Fri, 13 Jan 2017 22:26:22 +0000
Subject: [issue29271] Task.current_task(None) returns unexpected result
In-Reply-To: <1484345484.86.0.127501941011.issue29271@psf.upfronthosting.co.za>
Message-ID: <1484346382.66.0.918290002892.issue29271@psf.upfronthosting.co.za>


Changes by Alberto Berti <alberto at metapensiero.it>:


----------
nosy: +azazel

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29271>
_______________________________________

From report at bugs.python.org  Fri Jan 13 17:28:00 2017
From: report at bugs.python.org (Lele Gaifax)
Date: Fri, 13 Jan 2017 22:28:00 +0000
Subject: [issue29271] Task.current_task(None) returns unexpected result
In-Reply-To: <1484345484.86.0.127501941011.issue29271@psf.upfronthosting.co.za>
Message-ID: <1484346480.69.0.893338341218.issue29271@psf.upfronthosting.co.za>


Changes by Lele Gaifax <lelegaifax at gmail.com>:


----------
nosy: +lelit

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29271>
_______________________________________

From report at bugs.python.org  Fri Jan 13 17:28:44 2017
From: report at bugs.python.org (Brian Vandenberg)
Date: Fri, 13 Jan 2017 22:28:44 +0000
Subject: [issue29265] os.cpu_count is problematic on sparc/solaris
In-Reply-To: <1484329714.36.0.262857115906.issue29265@psf.upfronthosting.co.za>
Message-ID: <1484346524.56.0.793212125127.issue29265@psf.upfronthosting.co.za>


Brian Vandenberg added the comment:

> It doesn't cause any real problem with the tests, though.  I routinely run with -j40 on my 2 cpu test box because the test run completes faster that way due to the way many tests spend time waiting for various things.

In my case it did because it caused enough file descriptors to be allocated that it hit the cap for max open file handles.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29265>
_______________________________________

From report at bugs.python.org  Fri Jan 13 17:31:21 2017
From: report at bugs.python.org (Guido van Rossum)
Date: Fri, 13 Jan 2017 22:31:21 +0000
Subject: [issue29271] Task.current_task(None) returns unexpected result
In-Reply-To: <1484345484.86.0.127501941011.issue29271@psf.upfronthosting.co.za>
Message-ID: <1484346681.88.0.496822909157.issue29271@psf.upfronthosting.co.za>


Changes by Guido van Rossum <guido at python.org>:


----------
nosy:  -gvanrossum

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29271>
_______________________________________

From report at bugs.python.org  Fri Jan 13 17:36:52 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Fri, 13 Jan 2017 22:36:52 +0000
Subject: [issue29270] super call in ctypes sub-class fails in 3.6
In-Reply-To: <1484345422.14.0.967631121608.issue29270@psf.upfronthosting.co.za>
Message-ID: <1484347012.08.0.87412473895.issue29270@psf.upfronthosting.co.za>


Changes by Serhiy Storchaka <storchaka+cpython at gmail.com>:


----------
nosy: +ncoghlan, serhiy.storchaka
versions: +Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29270>
_______________________________________

From report at bugs.python.org  Fri Jan 13 17:57:20 2017
From: report at bugs.python.org (Lele Gaifax)
Date: Fri, 13 Jan 2017 22:57:20 +0000
Subject: [issue29271] Task.current_task(None) returns unexpected result
In-Reply-To: <1484345484.86.0.127501941011.issue29271@psf.upfronthosting.co.za>
Message-ID: <1484348240.3.0.388926011615.issue29271@psf.upfronthosting.co.za>


Lele Gaifax added the comment:

FWIW, the problem afflicts other functions, for example the following script exhibits a similar difference:

import asyncio

async def coro():
    print(asyncio.Task.all_tasks())
    print(asyncio.Task.all_tasks(None))

loop = asyncio.get_event_loop()
loop.run_until_complete(coro())

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29271>
_______________________________________

From report at bugs.python.org  Fri Jan 13 18:01:27 2017
From: report at bugs.python.org (Lele Gaifax)
Date: Fri, 13 Jan 2017 23:01:27 +0000
Subject: [issue29271] Task.current_task(None) returns unexpected result
In-Reply-To: <1484345484.86.0.127501941011.issue29271@psf.upfronthosting.co.za>
Message-ID: <1484348487.01.0.204031125845.issue29271@psf.upfronthosting.co.za>


Lele Gaifax added the comment:

Yay, Yury is too fast!! :-)

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29271>
_______________________________________

From report at bugs.python.org  Fri Jan 13 18:38:40 2017
From: report at bugs.python.org (Michael Felt)
Date: Fri, 13 Jan 2017 23:38:40 +0000
Subject: [issue18235] _sysconfigdata.py wrong on AIX installations
In-Reply-To: <1371429201.94.0.372851990361.issue18235@psf.upfronthosting.co.za>
Message-ID: <1484350720.43.0.419881553959.issue18235@psf.upfronthosting.co.za>


Michael Felt added the comment:

I did not go through a whole build process.

Changing:


    if _PYTHON_BUILD:
        vars['BLDSHARED'] = vars['LDSHARED']
to

    if _PYTHON_BUILD:
        vars['LDSHARED'] = vars['BLDSHARED']

is not going to help if both variables are wrong in _sysconfigdata*.py

Also,

The patch is not applied in any version - yet the values in _sysconfigdata*.py are different, i.e., version dependent (without the patch).

If I errored in my assumption that the file being patched is reading _sysconfigdata*.py then I will need to patch and build from scratch to see the effect on _sysconfigdata.py

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue18235>
_______________________________________

From report at bugs.python.org  Fri Jan 13 18:53:52 2017
From: report at bugs.python.org (Eryk Sun)
Date: Fri, 13 Jan 2017 23:53:52 +0000
Subject: [issue29270] super call in ctypes sub-class fails in 3.6
In-Reply-To: <1484345422.14.0.967631121608.issue29270@psf.upfronthosting.co.za>
Message-ID: <1484351632.83.0.957451441914.issue29270@psf.upfronthosting.co.za>


Eryk Sun added the comment:

In 3.6, type_new in Objects/typeobject.c sets the __classcell__ in the dict if it's a cell object. It happens that CreateSwappedType in Modules/_ctypes/_ctypes.c re-uses the dict to create the swapped type (e.g. big endian), which in turn updates the __classcell__. Thus in builtin___build_class__ in Python/bltinmodule.c, the check `cell_cls != cls` ends up being true, which leads to the observed TypeError. CreateSwappedType should be able to avoid this by either deleting "__classcell__" from the dict or creating a copy without it before passing it to type_new.

----------
nosy: +eryksun

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29270>
_______________________________________

From report at bugs.python.org  Fri Jan 13 19:46:50 2017
From: report at bugs.python.org (STINNER Victor)
Date: Sat, 14 Jan 2017 00:46:50 +0000
Subject: [issue29259] Add tp_fastcall to PyTypeObject: support FASTCALL
 calling convention for all callable objects
In-Reply-To: <1484310274.99.0.832491900546.issue29259@psf.upfronthosting.co.za>
Message-ID: <1484354810.72.0.473063623424.issue29259@psf.upfronthosting.co.za>


STINNER Victor added the comment:

> Slower (63):
> - unpickle_pure_python: 671 us +- 16 us -> 952 us +- 9 us: 1.42x slower (+42%)
> ...

This benchmark was run with LTO+PGO. It seems like PGO produced less efficient machine code with the patch. Maybe a missed optimization.

Oh, I just realized that I announced that objects with a __call__() are faster, but it's not the case yet! Such object still gets slot_tp_call() as tp_call which isn't a fastcall!

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29259>
_______________________________________

From report at bugs.python.org  Fri Jan 13 20:37:00 2017
From: report at bugs.python.org (Brian Vandenberg)
Date: Sat, 14 Jan 2017 01:37:00 +0000
Subject: [issue29272] test_logging hangs if /etc/hosts only aliases
 "localhost" to ::1
Message-ID: <1484357820.6.0.682177445577.issue29272@psf.upfronthosting.co.za>


New submission from Brian Vandenberg:

On some of the linux boxes on our (air-gapped, if that matters) network it looks like some of them were mis-configured and their /etc/hosts file looks something like this:

$ cat /etc/hosts
127.0.0.1 snoopy.the.internal.domain snoopy localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6

... whereas for other linux machines not exhibiting this problem:

$ cat /etc/hosts
127.0.0.1 woodstock.the.internal.domain woodstock localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6

It appears this is causing test_logging to hang while running SocketHandlerTest.

Unfortunately I don't have administrative privileges so I'm not able to prove I'm right.  I only have circumstantial evidence, but no machine I tested (about 20 machines) contradicts what I stated above.

----------
components: Tests
messages: 285452
nosy: phantal
priority: normal
severity: normal
status: open
title: test_logging hangs if /etc/hosts only aliases "localhost" to ::1
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29272>
_______________________________________

From report at bugs.python.org  Fri Jan 13 20:58:00 2017
From: report at bugs.python.org (Bryan B)
Date: Sat, 14 Jan 2017 01:58:00 +0000
Subject: [issue26296] colorys rgb_to_hls algorithm error
In-Reply-To: <1454680812.05.0.172060571572.issue26296@psf.upfronthosting.co.za>
Message-ID: <1484359080.74.0.640012635916.issue26296@psf.upfronthosting.co.za>


Bryan B added the comment:

Adding myself to this since I'm going to fix another hiccup in this file and I might as well clean this up too.

----------
nosy: +aarqon

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue26296>
_______________________________________

From report at bugs.python.org  Fri Jan 13 21:15:13 2017
From: report at bugs.python.org (Eryk Sun)
Date: Sat, 14 Jan 2017 02:15:13 +0000
Subject: [issue29270] super call in ctypes sub-class fails in 3.6
In-Reply-To: <1484345422.14.0.967631121608.issue29270@psf.upfronthosting.co.za>
Message-ID: <1484360113.52.0.11235082757.issue29270@psf.upfronthosting.co.za>


Eryk Sun added the comment:

Here's a patch that deletes __classcell__ from the dict before calling type_new.

----------
keywords: +patch
Added file: http://bugs.python.org/file46285/issue_29270_01.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29270>
_______________________________________

From report at bugs.python.org  Fri Jan 13 21:18:46 2017
From: report at bugs.python.org (Eryk Sun)
Date: Sat, 14 Jan 2017 02:18:46 +0000
Subject: [issue29270] super call in ctypes sub-class fails in 3.6
In-Reply-To: <1484345422.14.0.967631121608.issue29270@psf.upfronthosting.co.za>
Message-ID: <1484360326.94.0.0494635057452.issue29270@psf.upfronthosting.co.za>


Changes by Eryk Sun <eryksun at gmail.com>:


----------
stage:  -> patch review

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29270>
_______________________________________

From report at bugs.python.org  Fri Jan 13 21:19:10 2017
From: report at bugs.python.org (Bryan B)
Date: Sat, 14 Jan 2017 02:19:10 +0000
Subject: [issue26296] colorsys rgb_to_hls algorithm error
In-Reply-To: <1454680812.05.0.172060571572.issue26296@psf.upfronthosting.co.za>
Message-ID: <1484360350.75.0.536231034098.issue26296@psf.upfronthosting.co.za>


Bryan B added the comment:

Well, the other issue was resolved by updating Python on my computer to 3.6 ;)

Setting up the entire Python build and test environment for an issue this small seems a little excessive, especially for a module that seems seldomly used. I'm gonna have to be that guy and let this one sit as well.

Sorry :(

----------
title: colorys rgb_to_hls algorithm error -> colorsys rgb_to_hls algorithm error

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue26296>
_______________________________________

From report at bugs.python.org  Fri Jan 13 21:39:31 2017
From: report at bugs.python.org (INADA Naoki)
Date: Sat, 14 Jan 2017 02:39:31 +0000
Subject: [issue29271] Task.current_task(None) returns unexpected result
In-Reply-To: <1484345484.86.0.127501941011.issue29271@psf.upfronthosting.co.za>
Message-ID: <1484361571.78.0.272317030819.issue29271@psf.upfronthosting.co.za>


INADA Naoki added the comment:

LGTM

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29271>
_______________________________________

From report at bugs.python.org  Sat Jan 14 00:36:13 2017
From: report at bugs.python.org (INADA Naoki)
Date: Sat, 14 Jan 2017 05:36:13 +0000
Subject: [issue29263] Implement LOAD_METHOD/CALL_METHOD for C functions
In-Reply-To: <1484323792.44.0.217164349535.issue29263@psf.upfronthosting.co.za>
Message-ID: <1484372173.8.0.704820340742.issue29263@psf.upfronthosting.co.za>


INADA Naoki added the comment:

This is proof of concept patch to support LOAD_METHOD & CALL_METHOD for METH_FASTCALL methods.
(This patch should be applied after fastcall.patch)

$ ./python -m perf timeit --compare-to `pwd`/python-fastcall -s "d = b''" -- "d.decode()"
python-fastcall: ..................... 91.0 ns +- 1.0 ns
python: ..................... 80.3 ns +- 0.3 ns

Median +- std dev: [python-fastcall] 91.0 ns +- 1.0 ns -> [python] 80.3 ns +- 0.3 ns: 1.13x faster

$ ./python -m perf timeit --compare-to `pwd`/python-fastcall -s "d = b''" -- "d.decode('ascii')"
python-fastcall: ..................... 116 ns +- 1 ns
python: ..................... 106 ns +- 1 ns

Median +- std dev: [python-fastcall] 116 ns +- 1 ns -> [python] 106 ns +- 1 ns: 1.10x faster


Since PyCFunction is lighter than PyMethodObject, performance benefit seems smaller than
Python method (up to 20%).

Sadly speaking, there are only few METH_FASTCALL in builtin type.

----------
keywords: +patch
Added file: http://bugs.python.org/file46286/loadmethod-fastcall.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29263>
_______________________________________

From report at bugs.python.org  Sat Jan 14 01:16:35 2017
From: report at bugs.python.org (Craig Holmquist)
Date: Sat, 14 Jan 2017 06:16:35 +0000
Subject: [issue23407] os.walk always follows Windows junctions
In-Reply-To: <1423342763.81.0.179122512075.issue23407@psf.upfronthosting.co.za>
Message-ID: <1484374595.62.0.96657378215.issue23407@psf.upfronthosting.co.za>


Craig Holmquist added the comment:

FWIW, the only name-surrogate tags in the user-mode SDK headers (specifically winnt.h) are IO_REPARSE_TAG_MOUNT_POINT and IO_REPARSE_TAG_SYMLINK, as of at least the Windows 8.1 SDK.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23407>
_______________________________________

From report at bugs.python.org  Sat Jan 14 01:36:15 2017
From: report at bugs.python.org (INADA Naoki)
Date: Sat, 14 Jan 2017 06:36:15 +0000
Subject: [issue29263] Implement LOAD_METHOD/CALL_METHOD for C functions
In-Reply-To: <1484323792.44.0.217164349535.issue29263@psf.upfronthosting.co.za>
Message-ID: <1484375775.07.0.901089033373.issue29263@psf.upfronthosting.co.za>


INADA Naoki added the comment:

Maybe, we should do

* Make clinic use more METH_FASTCALL
* Use clinic more in builtin methods

before trying this optimization.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29263>
_______________________________________

From report at bugs.python.org  Sat Jan 14 01:54:13 2017
From: report at bugs.python.org (Martin Panter)
Date: Sat, 14 Jan 2017 06:54:13 +0000
Subject: [issue29062] hashlib documentation link error
In-Reply-To: <1482571303.3.0.781602557285.issue29062@psf.upfronthosting.co.za>
Message-ID: <1484376853.11.0.439051021782.issue29062@psf.upfronthosting.co.za>


Martin Panter added the comment:

Looks like Doc/tools/susp-ignored.csv needs updating:

$ make -C Doc/ suspicious
[. . .]
writing output... [ 49%] library/hashlib                                        
WARNING: [library/hashlib:502] ":vatrogasac" found in ">>> cookie = b'user:vatrogasac'"
WARNING: [library/hashlib:502] ":vatrogasac" found in "user:vatrogasac,349cf904533767ed2d755279a8df84d0"
WARNING: [library/hashlib:502] ":policajac" found in ">>> compare_digest(b'user:policajac', sig)"
WARNING: [library/hashlib:646] ":LEAF" found in "... h00 = blake2b(buf[0:LEAF_SIZE], fanout=FANOUT, depth=DEPTH,"
[. . .]
WARNING: Found 4/327 unused rules:
library/hashlib-blake2,,:vatrogasac,>>> cookie = b'user:vatrogasac'
library/hashlib-blake2,,:vatrogasac,user:vatrogasac,349cf904533767ed2d755279a8df84d0
library/hashlib-blake2,,:policajac,>>> compare_digest(b'user:policajac', sig)
library/hashlib-blake2,,:LEAF,h00 = blake2b(buf[0:LEAF_SIZE], fanout=FANOUT, depth=DEPTH,
build finished with problems, 7 warnings.
make[1]: *** [build] Error 1
make[1]: Leaving directory `/media/disk/home/proj/python/cpython/Doc'
Suspicious check complete; look for any errors in the above output or in build/suspicious/suspicious.csv.  If all issues are false positives, append that file to tools/susp-ignored.csv.
make: *** [suspicious] Error 1

----------
status: closed -> open

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29062>
_______________________________________

From report at bugs.python.org  Sat Jan 14 02:02:13 2017
From: report at bugs.python.org (Larry Hastings)
Date: Sat, 14 Jan 2017 07:02:13 +0000
Subject: [issue29006] 2.7.13 _sqlite more prone to "database table is locked"
In-Reply-To: <1482095939.25.0.891516594682.issue29006@psf.upfronthosting.co.za>
Message-ID: <1484377333.91.0.522259415075.issue29006@psf.upfronthosting.co.za>


Larry Hastings added the comment:

Hoping to tag in less than 48 hours...!

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29006>
_______________________________________

From report at bugs.python.org  Sat Jan 14 02:03:09 2017
From: report at bugs.python.org (Larry Hastings)
Date: Sat, 14 Jan 2017 07:03:09 +0000
Subject: [issue29125] Shell injection via TIX_LIBRARY when using tkinter.tix
In-Reply-To: <1483210830.52.0.920210756798.issue29125@psf.upfronthosting.co.za>
Message-ID: <1484377389.47.0.519851092309.issue29125@psf.upfronthosting.co.za>


Larry Hastings added the comment:

Could one of you recent tagees (Terry, Zach) review the patch?  Hoping to tag 3.5.3 final in less than 48 hours, and I want to cherry-pick the fix for this...!

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29125>
_______________________________________

From report at bugs.python.org  Sat Jan 14 02:13:13 2017
From: report at bugs.python.org (INADA Naoki)
Date: Sat, 14 Jan 2017 07:13:13 +0000
Subject: [issue29259] Add tp_fastcall to PyTypeObject: support FASTCALL
 calling convention for all callable objects
In-Reply-To: <1484310274.99.0.832491900546.issue29259@psf.upfronthosting.co.za>
Message-ID: <1484377993.03.0.938024528988.issue29259@psf.upfronthosting.co.za>


INADA Naoki added the comment:

1. _PyCFunction_FastCallKeywords() calls _PyStack_AsDict(), then calls _PyCFunction_FastCallDict().
2. _PyCFunction_FastCallDict() calls _Py_FastCall_FromArgs() when METH_FASTCALL.
3. _Py_FastCall_FromArgs() calls _PyStack_UnpackDict(), then calls fastcall().

Can you remove this _PyStack_AsDict() & _PyStack_UnpackDict() by creating shortcut in _PyCFunction_FastCallKeywords()?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29259>
_______________________________________

From report at bugs.python.org  Sat Jan 14 02:25:16 2017
From: report at bugs.python.org (Roundup Robot)
Date: Sat, 14 Jan 2017 07:25:16 +0000
Subject: [issue29145] failing overflow checks in replace_*
In-Reply-To: <1483468602.77.0.231507148809.issue29145@psf.upfronthosting.co.za>
Message-ID: <20170114072512.26938.63603.A4A3915D@psf.io>


Roundup Robot added the comment:

New changeset dd2c7d497878 by Martin Panter in branch '3.5':
Issues #1621, #29145: Test for str.join() overflow
https://hg.python.org/cpython/rev/dd2c7d497878

New changeset 0c6ea411af17 by Martin Panter in branch 'default':
Issue #29145: Merge test from 3.6
https://hg.python.org/cpython/rev/0c6ea411af17

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29145>
_______________________________________

From report at bugs.python.org  Sat Jan 14 02:25:16 2017
From: report at bugs.python.org (Roundup Robot)
Date: Sat, 14 Jan 2017 07:25:16 +0000
Subject: [issue22980] C extension naming doesn't take bitness into account
In-Reply-To: <1417530925.56.0.55888060474.issue22980@psf.upfronthosting.co.za>
Message-ID: <20170114072512.26938.19441.543AB1AC@psf.io>


Roundup Robot added the comment:

New changeset 80fc40a9ae47 by Martin Panter in branch '3.5':
Issue #22980: Skip a sysconfig test if _ctypes is not available.
https://hg.python.org/cpython/rev/80fc40a9ae47

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue22980>
_______________________________________

From report at bugs.python.org  Sat Jan 14 02:25:17 2017
From: report at bugs.python.org (Roundup Robot)
Date: Sat, 14 Jan 2017 07:25:17 +0000
Subject: [issue1621] Do not assume signed integer overflow behavior
In-Reply-To: <1197593027.35.0.00314874350765.issue1621@psf.upfronthosting.co.za>
Message-ID: <20170114072513.26938.3730.0B74BE33@psf.io>


Roundup Robot added the comment:

New changeset dd2c7d497878 by Martin Panter in branch '3.5':
Issues #1621, #29145: Test for str.join() overflow
https://hg.python.org/cpython/rev/dd2c7d497878

New changeset eb6eafafdb44 by Martin Panter in branch 'default':
Issue #1621: Overflow should not be possible in listextend()
https://hg.python.org/cpython/rev/eb6eafafdb44

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue1621>
_______________________________________

From report at bugs.python.org  Sat Jan 14 02:25:54 2017
From: report at bugs.python.org (Martin Panter)
Date: Sat, 14 Jan 2017 07:25:54 +0000
Subject: [issue29273] test___all__ alters utf8 locale setting
Message-ID: <1484378754.56.0.202012710496.issue29273@psf.upfronthosting.co.za>


New submission from Martin Panter:

On a Linux computer I have LANG=en_US.utf8 set. Python 3 initializes the locale with this setting (see revision 43e32b2b4004):

>>> locale.setlocale(locale.LC_CTYPE)  # Get setting initialized by Python
'en_US.utf8'

In Lib/test/test___all__.py, there is a special case to restore the locale after loading the ?readline? module (last changed in revision 37ed6eed0595). It does this by calling getlocale() and later setlocale(). However the getlocale() call translates the setting to UTF-8 in uppercase with a dash, and setlocale() keeps it that way:

>>> locale.getlocale(locale.LC_CTYPE)
('en_US', 'UTF-8')
>>> locale.setlocale(locale.LC_CTYPE, _)
'en_US.UTF-8'

This means that the test framework may complain:

Warning -- locale was modified by test___all__

One subtlety is that I only see the warning when the ?tkinter? module is unavailable. Otherwise, the test imports Lib/idlelib/iomenu.py, which calls setlocale(locale.LC_CTYPE, "") and restores the original ?utf8? spelling.

The easiest fix would be to use setlocale(locale.LC_CTYPE) instead of getlocale() in test___all__. However I think it may be safe to remove this code entirely. The original version of the it was added by Guido in revision ba522c74a931 (year 2002). Three hours later is revision c7d5801bbae8, where Guido changed the ?readline? module to restore the locale after Readline library initialization. Since the module is fixed, I don?t think we need the test workaround, and I propose to remove it in Python 3, where it causes the above conflict.

----------
components: Tests
files: readline-getlocale.patch
keywords: patch
messages: 285467
nosy: martin.panter
priority: normal
severity: normal
stage: patch review
status: open
title: test___all__ alters utf8 locale setting
type: behavior
versions: Python 3.5, Python 3.6, Python 3.7
Added file: http://bugs.python.org/file46287/readline-getlocale.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29273>
_______________________________________

From report at bugs.python.org  Sat Jan 14 02:30:59 2017
From: report at bugs.python.org (Martin Panter)
Date: Sat, 14 Jan 2017 07:30:59 +0000
Subject: =?utf-8?b?W2lzc3VlMjkyNzRdIENoYW5nZSDigJx0ZXN0cyBjYXNlc+KAnSDihpIg4oCc?=
 =?utf-8?q?test_cases=E2=80=9D?=
Message-ID: <1484379059.0.0.00296348115264.issue29274@psf.upfronthosting.co.za>


New submission from Martin Panter:

The ?unittest? documentation has ?tests cases? written a few times. This doesn?t seem right to me, but I thought I should get a second opinion in case I missed something.

----------
assignee: docs at python
components: Documentation
files: tests-cases.patch
keywords: patch
messages: 285468
nosy: docs at python, martin.panter
priority: normal
severity: normal
stage: patch review
status: open
title: Change ?tests cases? ? ?test cases?
versions: Python 2.7, Python 3.5, Python 3.6, Python 3.7
Added file: http://bugs.python.org/file46288/tests-cases.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29274>
_______________________________________

From report at bugs.python.org  Sat Jan 14 03:20:58 2017
From: report at bugs.python.org (Nick Coghlan)
Date: Sat, 14 Jan 2017 08:20:58 +0000
Subject: [issue29270] super call in ctypes sub-class fails in 3.6
In-Reply-To: <1484345422.14.0.967631121608.issue29270@psf.upfronthosting.co.za>
Message-ID: <1484382058.73.0.704788643812.issue29270@psf.upfronthosting.co.za>


Nick Coghlan added the comment:

Eryk's diagnosis sounds right to me, and the suggested patch will get this back to working as well as it did in Python 3.5.

However, it's worth noting that that state itself was already broken when it comes to zero-argument super() support on the type definition with reversed endianness:

```
>>> import ctypes as ct
>>> class SuperText(ct.c_uint32):
...     def __repr__(self):
...         return super().__repr__()
... 
>>> SuperText.__ctype_le__(1)
<SuperText object at 0x7fde526daea0>
>>> SuperText.__ctype_be__(1)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 3, in __repr__
TypeError: super(type, obj): obj must be an instance or subtype of type
```

The apparently nonsensical error message comes from both the original type and the type with swapped endianness having the same representation:

>>> SuperText.__ctype_le__
<class '__main__.SuperText'>
>>> SuperText.__ctype_be__
<class '__main__.SuperText'>

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29270>
_______________________________________

From report at bugs.python.org  Sat Jan 14 03:50:29 2017
From: report at bugs.python.org (Armin Rigo)
Date: Sat, 14 Jan 2017 08:50:29 +0000
Subject: [issue29006] 2.7.13 _sqlite more prone to "database table is locked"
In-Reply-To: <1482095939.25.0.891516594682.issue29006@psf.upfronthosting.co.za>
Message-ID: <1484383829.23.0.283526344724.issue29006@psf.upfronthosting.co.za>


Armin Rigo added the comment:

larry: unless someone else comments, I think now that the current status of 3.5.3 is fine enough (nothing was done in this branch, and the problem I describe and just fixed in PyPy can be left for later).

The revert dd13098a5dc2 needs to be itself reverted in the 2.7 branch.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29006>
_______________________________________

From report at bugs.python.org  Sat Jan 14 05:17:21 2017
From: report at bugs.python.org (Roundup Robot)
Date: Sat, 14 Jan 2017 10:17:21 +0000
Subject: [issue15527] Double parens in functions references
In-Reply-To: <1343833557.96.0.948912415493.issue15527@psf.upfronthosting.co.za>
Message-ID: <20170114101718.10896.68923.7AF5B961@psf.io>


Roundup Robot added the comment:

New changeset be4da80b493e by Martin Panter in branch '2.7':
Issue #15527: remove double parens by changing markup.
https://hg.python.org/cpython/rev/be4da80b493e

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue15527>
_______________________________________

From report at bugs.python.org  Sat Jan 14 05:45:14 2017
From: report at bugs.python.org (Terry J. Reedy)
Date: Sat, 14 Jan 2017 10:45:14 +0000
Subject: [issue29125] Shell injection via TIX_LIBRARY when using tkinter.tix
In-Reply-To: <1483210830.52.0.920210756798.issue29125@psf.upfronthosting.co.za>
Message-ID: <1484390714.65.0.239029995403.issue29125@psf.upfronthosting.co.za>


Terry J. Reedy added the comment:

In the original code, python interpolates tixlib into the string sent to and executed by tcl exec.  With the patch, tcl exec does the interpolation.  Not knowing anything in particular about tcl's exec, I found a value for tixlib that appears to validate Serhiy's claim that tcl exec does not rescan.

C:\Users\Terry>py -3.5
Python 3.5.2 (v3.5.2:4def2a2901a5, Jun 25 2016, 22:18:55) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import tkinter as tk
>>> tka = tk.Tk().tk
>>> txlib =  '} python -c "print(999)"'
>>> tka.setvar('TIX_LIBRARY', txlib)
>>> tka.eval('global autopath; lappend auto_path {%s}' % txlib)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
_tkinter.TclError: extra characters after close-quote

>>> tka.eval('global autopath; lappend auto_path $TIX_LIBRARY')
'{C:\\Programs\\Python35\\tcl\\tcl8.6} C:/Programs/Python35/tcl C:/Programs/lib C:/Programs/Python35/tcl/tk8.6 C:/Programs/Python35/tcl/tk8.6/ttk \\}\\ python\\ -c\\ \\"print(999)\\"'

I don't understand exactly why (or when) TclError is raised. but it is only raised when python does the interpolation.  And for this string, only when '}' is present.  Without the '}', there is no exception and the interpolated string is simply appended, as with the new $TIX_LIBRARY code.

test_tix, such as it is, passes with the patch.  So unless I missed something the patch appears to be both safe and useful.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29125>
_______________________________________

From report at bugs.python.org  Sat Jan 14 06:20:43 2017
From: report at bugs.python.org (Dave Jones)
Date: Sat, 14 Jan 2017 11:20:43 +0000
Subject: [issue29270] super call in ctypes sub-class fails in 3.6
In-Reply-To: <1484345422.14.0.967631121608.issue29270@psf.upfronthosting.co.za>
Message-ID: <1484392843.14.0.149968463936.issue29270@psf.upfronthosting.co.za>


Dave Jones added the comment:

I confess I'm going to have to read a bit more about Python internals before I can understand Eryk's analysis (this is my first encounter with "cell objects"), but many thanks for the rapid analysis and patch!

I'm not too concerned about the state being broken with reversed endianness; I don't think that's going to affect any of my use-cases in the near future, but it's certainly useful to know in case it does come up.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29270>
_______________________________________

From report at bugs.python.org  Sat Jan 14 06:35:41 2017
From: report at bugs.python.org (Eryk Sun)
Date: Sat, 14 Jan 2017 11:35:41 +0000
Subject: [issue29270] super call in ctypes sub-class fails in 3.6
In-Reply-To: <1484345422.14.0.967631121608.issue29270@psf.upfronthosting.co.za>
Message-ID: <1484393741.25.0.132354122256.issue29270@psf.upfronthosting.co.za>


Eryk Sun added the comment:

OK, this is completely broken and needs a more thoughtful solution than my simpleminded hack. Here's a practical example of the problem, tested in 3.5.2:

    class MyInt(ctypes.c_int):
        def __repr__(self):
            return super().__repr__()

    class Struct(ctypes.BigEndianStructure):
        _fields_ = (('i', MyInt),)

    >>> s = Struct()
    >>> s.i
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "<stdin>", line 3, in __repr__
    TypeError: super(type, obj): obj must be an instance or subtype of type

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29270>
_______________________________________

From report at bugs.python.org  Sat Jan 14 07:04:52 2017
From: report at bugs.python.org (Nick Coghlan)
Date: Sat, 14 Jan 2017 12:04:52 +0000
Subject: [issue29270] super call in ctypes sub-class fails in 3.6
In-Reply-To: <1484345422.14.0.967631121608.issue29270@psf.upfronthosting.co.za>
Message-ID: <1484395492.47.0.935156904699.issue29270@psf.upfronthosting.co.za>


Nick Coghlan added the comment:

Yeah, re-using Python-level method objects in different types is genuinely invalid when combined with __class__ or zero-argument super(), as there's no way to make the __class__ closure refer to two different classes at runtime - it will always refer back to the original defining class.

And while ctypes could be updated to do the right thing for functions, classmethod, staticmethod, abstractmethod, property, etc, there's nothing systematic it can do that would work for arbitrary descriptors (any of which may have a zero-argument-super-using method definition hidden inside their internal state).

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29270>
_______________________________________

From report at bugs.python.org  Sat Jan 14 07:05:06 2017
From: report at bugs.python.org (Roundup Robot)
Date: Sat, 14 Jan 2017 12:05:06 +0000
Subject: [issue29062] hashlib documentation link error
In-Reply-To: <1482571303.3.0.781602557285.issue29062@psf.upfronthosting.co.za>
Message-ID: <20170114120503.3334.40121.48B34CC2@psf.io>


Roundup Robot added the comment:

New changeset ea0c488b9bac by INADA Naoki in branch '3.6':
Issue #29062: Doc: Fix make suspicious
https://hg.python.org/cpython/rev/ea0c488b9bac

New changeset 5c48fbe12cb8 by INADA Naoki in branch 'default':
Issue #29062: Doc: Fix make suspicious
https://hg.python.org/cpython/rev/5c48fbe12cb8

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29062>
_______________________________________

From report at bugs.python.org  Sat Jan 14 07:06:30 2017
From: report at bugs.python.org (INADA Naoki)
Date: Sat, 14 Jan 2017 12:06:30 +0000
Subject: [issue29062] hashlib documentation link error
In-Reply-To: <1482571303.3.0.781602557285.issue29062@psf.upfronthosting.co.za>
Message-ID: <1484395590.83.0.435201163659.issue29062@psf.upfronthosting.co.za>


INADA Naoki added the comment:

Martin, thank you for pointing it out.
I hadn't know about suspicious check.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29062>
_______________________________________

From report at bugs.python.org  Sat Jan 14 07:43:37 2017
From: report at bugs.python.org (Masayuki Yamamoto)
Date: Sat, 14 Jan 2017 12:43:37 +0000
Subject: [issue25658] PyThread assumes pthread_key_t is an integer,
 which is against POSIX
In-Reply-To: <1447861816.88.0.632584646231.issue25658@psf.upfronthosting.co.za>
Message-ID: <1484397816.99.0.31629607314.issue25658@psf.upfronthosting.co.za>


Masayuki Yamamoto added the comment:

I commented at the Rietveld, since I think that patch needs a bit more modification.

* rename PyThread_ReInitTLS
* update comments and messages that have explained CPython TLS API

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue25658>
_______________________________________

From report at bugs.python.org  Sat Jan 14 07:57:45 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sat, 14 Jan 2017 12:57:45 +0000
Subject: [issue29125] Shell injection via TIX_LIBRARY when using tkinter.tix
In-Reply-To: <1483210830.52.0.920210756798.issue29125@psf.upfronthosting.co.za>
Message-ID: <1484398665.94.0.146711673799.issue29125@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

TclError in Terry's example is raised because Tcl script has unpaired braces. You should add "{" at the end of TIX_LIBRARY.

Here is working exploit:

$ TIX_LIBRARY="/dev/null}; exec python3 -m this >spoiled; set x {"  python3 -c "from tkinter.tix import Tk; Tk()"

It creates the file "spoiled" in current directory containing The Zen of Python.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29125>
_______________________________________

From report at bugs.python.org  Sat Jan 14 08:28:44 2017
From: report at bugs.python.org (Berker Peksag)
Date: Sat, 14 Jan 2017 13:28:44 +0000
Subject: =?utf-8?b?W2lzc3VlMjkyNzRdIENoYW5nZSDigJx0ZXN0cyBjYXNlc+KAnSDihpIg4oCc?=
 =?utf-8?q?test_cases=E2=80=9D?=
In-Reply-To: <1484379059.0.0.00296348115264.issue29274@psf.upfronthosting.co.za>
Message-ID: <1484400524.77.0.0440736262073.issue29274@psf.upfronthosting.co.za>


Changes by Berker Peksag <berker.peksag at gmail.com>:


----------
nosy: +michael.foord

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29274>
_______________________________________

From report at bugs.python.org  Sat Jan 14 08:28:51 2017
From: report at bugs.python.org (Eryk Sun)
Date: Sat, 14 Jan 2017 13:28:51 +0000
Subject: [issue29270] super call in ctypes sub-class fails in 3.6
In-Reply-To: <1484345422.14.0.967631121608.issue29270@psf.upfronthosting.co.za>
Message-ID: <1484400531.05.0.129485915966.issue29270@psf.upfronthosting.co.za>


Eryk Sun added the comment:

Resolving this would be straightforward if we could use a subclass for the swapped type, but ctypes simple types behave differently for subclasses. A simple subclass doesn't automatically call the getfunc to get a converted value when returned as a function result, struct field, or array index.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29270>
_______________________________________

From report at bugs.python.org  Sat Jan 14 08:45:04 2017
From: report at bugs.python.org (Chi Hsuan Yen)
Date: Sat, 14 Jan 2017 13:45:04 +0000
Subject: [issue29264] sparc/ffi.c:440 error: 'asm' undeclared
In-Reply-To: <1484328459.12.0.655564355741.issue29264@psf.upfronthosting.co.za>
Message-ID: <1484401504.12.0.410845578834.issue29264@psf.upfronthosting.co.za>


Chi Hsuan Yen added the comment:

Since Python 3.6, building ctypes with bundled libffi is deprecated. Please build libffi separately and configure CPython with --with-system-ffi.

This issue can be closed as third-party.

----------
nosy: +Chi Hsuan Yen

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29264>
_______________________________________

From report at bugs.python.org  Sat Jan 14 08:53:30 2017
From: report at bugs.python.org (INADA Naoki)
Date: Sat, 14 Jan 2017 13:53:30 +0000
Subject: [issue29062] hashlib documentation link error
In-Reply-To: <1482571303.3.0.781602557285.issue29062@psf.upfronthosting.co.za>
Message-ID: <1484402010.2.0.23517345463.issue29062@psf.upfronthosting.co.za>


Changes by INADA Naoki <songofacandy at gmail.com>:


----------
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29062>
_______________________________________

From report at bugs.python.org  Sat Jan 14 09:05:46 2017
From: report at bugs.python.org (Eryk Sun)
Date: Sat, 14 Jan 2017 14:05:46 +0000
Subject: [issue29240] Implementation of the PEP 540: Add a new UTF-8 mode
In-Reply-To: <1484133592.17.0.263024020483.issue29240@psf.upfronthosting.co.za>
Message-ID: <1484402746.93.0.636003345971.issue29240@psf.upfronthosting.co.za>


Eryk Sun added the comment:

> it should be replaced with sys.getfilesystemencodeerrors() 
> to support UTF-8 Strict mode.

I did that in the patch for issue 28188. The focus of the patch is to add bytes support on Windows for os.putenv and os.environb, but I also tried to maximize consistency (at least parallel structure) between the POSIX and Windows implementations.

----------
nosy: +eryksun

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29240>
_______________________________________

From report at bugs.python.org  Sat Jan 14 09:42:52 2017
From: report at bugs.python.org (Evan)
Date: Sat, 14 Jan 2017 14:42:52 +0000
Subject: [issue29262] Provide a way to check for *real* typing.Union instances
In-Reply-To: <1484322347.31.0.5780880295.issue29262@psf.upfronthosting.co.za>
Message-ID: <1484404972.13.0.073857214221.issue29262@psf.upfronthosting.co.za>


Evan added the comment:

I'm also interested in this. I've been using 'thing.__origin__ is typing.Union', but this doesn't work in some of the older versions of typing.

----------
nosy: +evan_

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29262>
_______________________________________

From report at bugs.python.org  Sat Jan 14 09:47:37 2017
From: report at bugs.python.org (INADA Naoki)
Date: Sat, 14 Jan 2017 14:47:37 +0000
Subject: [issue29259] Add tp_fastcall to PyTypeObject: support FASTCALL
 calling convention for all callable objects
In-Reply-To: <1484310274.99.0.832491900546.issue29259@psf.upfronthosting.co.za>
Message-ID: <1484405257.11.0.505201661258.issue29259@psf.upfronthosting.co.za>


INADA Naoki added the comment:

I've created pull request about it:
https://github.com/haypo/cpython/pull/1

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29259>
_______________________________________

From report at bugs.python.org  Sat Jan 14 09:52:12 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sat, 14 Jan 2017 14:52:12 +0000
Subject: [issue26296] colorsys rgb_to_hls algorithm error
In-Reply-To: <1454680812.05.0.172060571572.issue26296@psf.upfronthosting.co.za>
Message-ID: <1484405532.43.0.386975475849.issue26296@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

> The modulus fixes it for exact numbers.  It doesn't produce exactly the same result with floats, because of rounding.  Is that a problem?

I don't think that is a problem. The function uses a number of floating point operations producing inexact intermediate values. There is not an evidence that the proposed code produces more accurate value.

----------
nosy: +serhiy.storchaka
resolution:  -> not a bug
stage: needs patch -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue26296>
_______________________________________

From report at bugs.python.org  Sat Jan 14 09:54:07 2017
From: report at bugs.python.org (Craig Holmquist)
Date: Sat, 14 Jan 2017 14:54:07 +0000
Subject: [issue23407] os.walk always follows Windows junctions
In-Reply-To: <1423342763.81.0.179122512075.issue23407@psf.upfronthosting.co.za>
Message-ID: <1484405647.36.0.493436835151.issue23407@psf.upfronthosting.co.za>


Craig Holmquist added the comment:

Here's a new patch:  now, _Py_attribute_data_to_stat and Py_DeleteFileW will just use the IsReparseTagNameSurrogate macro to determine if the file is a link, so os.walk etc. will know not to follow them.  os.readlink, however, will only work with junctions and symbolic links; otherwise it will raise ValueError with "unsupported reparse tag".

This way, there's a basic level of support for all name-surrogate tags, but os.readlink only works with the ones whose internal structure is (semi-) documented.

----------
Added file: http://bugs.python.org/file46289/issue23407-3.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23407>
_______________________________________

From report at bugs.python.org  Sat Jan 14 10:13:24 2017
From: report at bugs.python.org (David Edelsohn)
Date: Sat, 14 Jan 2017 15:13:24 +0000
Subject: [issue18235] _sysconfigdata.py wrong on AIX installations
In-Reply-To: <1371429201.94.0.372851990361.issue18235@psf.upfronthosting.co.za>
Message-ID: <1484406804.39.0.354032842827.issue18235@psf.upfronthosting.co.za>


David Edelsohn added the comment:

Michael, you need to build from scratch. The values are set and tweaked in various phases of configure and then written to _sysconfigdata.py for installation.

The values in the file reflect the values used during the build, but many of them are not used in an installed version of Python.

Three important phases are:

1) Building modules in the tree during the build process.
2) In-tree testing of build module feature.
3) Building and installing modules with an installed version of Python.

The initial configuration scripts must match the location where the export files will be installed. And the _sysconfigdata.py definitions used to build external modules in an installed version of Python must refer to the proper location.

All of the pieces are interconnected and must be tested in a wholistic manner. A partial rebuild does not test the impact of the patch.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue18235>
_______________________________________

From report at bugs.python.org  Sat Jan 14 10:31:45 2017
From: report at bugs.python.org (Craig Holmquist)
Date: Sat, 14 Jan 2017 15:31:45 +0000
Subject: [issue29248] os.readlink fails on Windows
In-Reply-To: <1484208560.98.0.375529555783.issue29248@psf.upfronthosting.co.za>
Message-ID: <1484407905.68.0.953955822358.issue29248@psf.upfronthosting.co.za>


Changes by Craig Holmquist <craigholm at gmail.com>:


----------
keywords: +patch
Added file: http://bugs.python.org/file46290/issue29248.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29248>
_______________________________________

From report at bugs.python.org  Sat Jan 14 10:36:30 2017
From: report at bugs.python.org (Craig Holmquist)
Date: Sat, 14 Jan 2017 15:36:30 +0000
Subject: [issue23407] os.walk always follows Windows junctions
In-Reply-To: <1423342763.81.0.179122512075.issue23407@psf.upfronthosting.co.za>
Message-ID: <1484408190.36.0.822446587464.issue23407@psf.upfronthosting.co.za>


Craig Holmquist added the comment:

New patch with spaces instead of tabs

----------
Added file: http://bugs.python.org/file46291/issue23407-4.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23407>
_______________________________________

From report at bugs.python.org  Sat Jan 14 10:46:18 2017
From: report at bugs.python.org (Elizabeth Myers)
Date: Sat, 14 Jan 2017 15:46:18 +0000
Subject: [issue29275] time module still has Y2K issues note
Message-ID: <1484408778.39.0.526783015881.issue29275@psf.upfronthosting.co.za>


New submission from Elizabeth Myers:

It's 2017. I think it's time to remove the Y2K warning from this: https://docs.python.org/3/library/time.html :P. It's 17 years past the sell-by date for that notice.

----------
assignee: docs at python
components: Documentation
messages: 285489
nosy: Elizacat, docs at python
priority: normal
severity: normal
status: open
title: time module still has Y2K issues note
versions: Python 2.7, Python 3.3, Python 3.4, Python 3.5, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29275>
_______________________________________

From report at bugs.python.org  Sat Jan 14 10:50:46 2017
From: report at bugs.python.org (Elizabeth Myers)
Date: Sat, 14 Jan 2017 15:50:46 +0000
Subject: [issue29275] time module still has Y2K issues note
In-Reply-To: <1484408778.39.0.526783015881.issue29275@psf.upfronthosting.co.za>
Message-ID: <1484409046.98.0.587018750012.issue29275@psf.upfronthosting.co.za>


Changes by Elizabeth Myers <elizabeth at interlinked.me>:


----------
type:  -> enhancement

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29275>
_______________________________________

From report at bugs.python.org  Sat Jan 14 10:52:38 2017
From: report at bugs.python.org (Ari)
Date: Sat, 14 Jan 2017 15:52:38 +0000
Subject: [issue29276] HTMLParser in Python 2.7 doesn't recognize image tags
 wrapped up in link tags
Message-ID: <1484409158.63.0.451865597774.issue29276@psf.upfronthosting.co.za>


New submission from Ari:

The following code produces incorrect results under Python 2.7.13. One would expect it to print 2 lines, "Encountered a start tag: a" and "Encountered a start tag: img". Yet it prints only "Encountered a start tag: a".

from HTMLParser import HTMLParser
class MyHTMLParser(HTMLParser):
    def handle_starttag(self, tag, attrs):
        print 'Encountered a start tag: %s' % tag
parser = MyHTMLParser()
parser.feed('<a href="http://somesite.com/large_image.jpg"><img src="http://somesite.com/small_image.jpg" width="800px" /></a>')


Python 3.5.2 produces correct results on the same input and prints the expected "Encountered a start tag: a" and "Encountered a start tag: img".

from html.parser import HTMLParser
class MyHTMLParser(HTMLParser):
    def handle_starttag(self, tag, attrs):
        print("Encountered a start tag:", tag)
parser = MyHTMLParser()
parser.feed('<a href="http://somesite.com/large_image.jpg"><img src="http://somesite.com/small_image.jpg" width="800px" /></a>')

----------
components: Library (Lib)
messages: 285490
nosy: Ari
priority: normal
severity: normal
status: open
title: HTMLParser in Python 2.7 doesn't recognize image tags wrapped up in link tags
type: behavior
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29276>
_______________________________________

From report at bugs.python.org  Sat Jan 14 11:57:02 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Sat, 14 Jan 2017 16:57:02 +0000
Subject: [issue29276] HTMLParser in Python 2.7 doesn't recognize image tags
 wrapped up in link tags
In-Reply-To: <1484409158.63.0.451865597774.issue29276@psf.upfronthosting.co.za>
Message-ID: <1484413022.52.0.843107429227.issue29276@psf.upfronthosting.co.za>


Xiang Zhang added the comment:

I can get the expected behaviour with the lastest 2.7 build.

Python 2.7.13+ (2.7:0d4e0a736688, Jan 15 2017, 00:51:57) 
[GCC 5.2.1 20151010] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from HTMLParser import HTMLParser
>>> class MyHTMLParser(HTMLParser):
...     def handle_starttag(self, tag, attrs):
...         print 'Encountered a start tag: %s' % tag
... 
>>> parser = MyHTMLParser()
>>> parser.feed('<a href="http://somesite.com/large_image.jpg"><img src="http://somesite.com/small_image.jpg" width="800px" /></a>')
Encountered a start tag: a
Encountered a start tag: img

----------
nosy: +xiang.zhang

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29276>
_______________________________________

From report at bugs.python.org  Sat Jan 14 12:07:50 2017
From: report at bugs.python.org (miniflow)
Date: Sat, 14 Jan 2017 17:07:50 +0000
Subject: [issue29277] os.getcwd failing on LOFS share
Message-ID: <1484413670.89.0.750982203265.issue29277@psf.upfronthosting.co.za>


New submission from miniflow:

I am calling os.getcwd and am occasionally seeing failures inside my program.  This is not due to files missing as the files on the share are static.

Eror is:

[Errno 2] No such file or directory

Which is definitely not the case.

Can any sort of trace be done at the time of failure to detect if the issue is with python or the OS?

System is running Debian inside of an Illumos LX zone.

----------
components: IO, Library (Lib)
messages: 285492
nosy: David Terk
priority: normal
severity: normal
status: open
title: os.getcwd failing on LOFS share
type: crash
versions: Python 2.7, Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29277>
_______________________________________

From report at bugs.python.org  Sat Jan 14 12:21:32 2017
From: report at bugs.python.org (Xavier de Gaye)
Date: Sat, 14 Jan 2017 17:21:32 +0000
Subject: [issue26865] Meta-issue: support of the android platform
In-Reply-To: <1461685011.99.0.617135447086.issue26865@psf.upfronthosting.co.za>
Message-ID: <1484414492.14.0.491123004104.issue26865@psf.upfronthosting.co.za>


Xavier de Gaye added the comment:

Current status:
---------------

Available using the pyona build system [1]:
  * Cross compilation of Python for a given Android API level and architecture with android-ndk-r13b.
  * Cross compilation of third party extension modules (currently as a patch).
  * Interactive interpreter with curses and readline support, the Android adb (remote) shell is used to start the interpreter on the qemu emulator or a device connected with USB.
  * Remote debugging with gdb and support of the python-gdb module.
  * The adb shell is used to start a run of the test suite on the emulator or a device.

To be defined for the support of the Android platform:
  * Support starting with which Python release ?
     sys.getandroidapilevel() has been defined in Python 3.7 by issue 28740, so unless this enhancement is backported to 3.6 the first release to support Android could be 3.7.
  * What is the supported Android API level(s) [2] ?
     Level 21 is the first to provide a reliable wide character support.
     Level 24 is the most recent api level currently supported by the NDK and is the first where the adb shell is run as the 'shell' user instead of as 'root' and as a consequence, where the test suite must now cope with Android SELinux non permitted operations (hard link, mkfifo, mknod, bind on a unix socket).
  * On which architecture(s) ?
     The x86 platform is useful for testing and debugging as it runs fast on the Android qemu emulator.
     AFAIK the armv7 platform is still one of the most widespread Android platforms [3] [4].
  * Building with which NDK version ?
     The next android-ndk-r14 release is the first to provide "Unified headers", see issue #29040.
  * The buildbots run the test suite on the Android qemu emulator or on a device or both ?

Test suite results on the Android qemu emulator:
  test_builtin is excluded in all the tests - test_asyncio is excluded on armv7-android-api-21.
  x86 platform (duration: about 27 minutes, to be compared with 26 minutes when the test suite is run natively on the same laptop).
    api 21: success
    api 24: success
  armv7 platform (duration: about 410 minutes. Without the latest released libffi-3.2.1 that does not build on armv7).
    api 21: success
    api 24: success

Remaining issues:
1) Blocker issues:
  test_builtin:
    issue #13886: failure when test_readline is run before test_builtin.
      This problem is not specific to Android.
  test_asyncio:
    issue #26858: android: setting SO_REUSEPORT fails
      Failure only on armv7 at api 21.

2) Issues with a patch tested successfully on x86 and armv7 at Android api 21 and 24:
  Build:
    issue #28833: cross compilation of third-party extension modules.
  PermissionError raised at api 24:
    test_eintr test_genericpath test_pathlib test_posix test_shutil test_stat
      issue #28759: access to mkfifo, mknod and hard links is controled by SELinux MAC on Android
    test_os:
      issue #29180: skip tests that raise PermissionError in test_os (non-root user on Android)
    test_tarfile:
      issue #29181: skip tests that raise PermissionError in test_tarfile (non-root user on Android)
    test_socketserver:
      issue #29184: skip tests of test_socketserver when bind() raises PermissionError (non-root user on Android)
    test_distutils:
      issue #29185: test_distutils fails on Android api 24
    test_asyncio:
      issue #28684: [asyncio] bind() on a unix socket raises PermissionError on Android for a non-root user.
  Miscellaneous:
    test_asyncio:
      issue #28562: test_asyncio fails on Android upon calling getaddrinfo().
    test_readline:
      issue #28997: test_readline.test_nonascii fails on Android.
    test_curses:
      issue #29176: /tmp does not exist on Android and is used by curses.window.putwin()

3) Enhancement issues:
  issue #26855: android: add platform.android_ver().
  issue #27659: Prohibit implicit C function declarations.
  issue #29040: building Android with android-ndk-r14.

4) Languishing issues:
  issue #22724: byte-compile fails for cross-builds.
  issue #27606: Android cross-built for armv5te with clang and '-mthumb' crashes with SIGSEGV or SIGILL.
  issue #26852: add the '--enable-sourceless-distribution' option to configure.
  issue #27640: add the '--disable-test-suite' option to configure.

[1] https://bitbucket.org/xdegaye/pyona
[2] Andoid versions:
      Android Version      Released        API Level  Name
      Android 7.1          October 2016    25         Nougat
      Android 7.0          August 2016     24         Nougat
      Android 6.0          August 2015     23         Marshmallow
      Android 5.1          March 2015      22         Lollipop
      Android 5.0          November 2014   21         Lollipop
      Android 4.4W         June 2014       20         Kitkat Watch
      Android 4.4          October 2013    19         Kitkat
      Android 4.3          July 2013       18         Jelly Bean
      Android 4.2-4.2.2    November 2012   17         Jelly Bean
      Android 4.1-4.1.1    June 2012       16         Jelly Bean
      Android 4.0.3-4.0.4  December 2011   15         Ice Cream Sandwich
      Android 4.0-4.0.2    October 2011    14         Ice Cream Sandwich
      Android 3.2          June 2011       13         Honeycomb
      Android 3.1.x        May 2011        12         Honeycomb
      Android 3.0.x        February 2011   11         Honeycomb
      Android 2.3.3-2.3.4  February 2011   10         Gingerbread
      Android 2.3-2.3.2    November 2010   9          Gingerbread
      Android 2.2.x        June 2010       8          Froyo
      Android 2.1.x        January 2010    7          Eclair
      Android 2.0.1        December 2009   6          Eclair
      Android 2.0          November 2009   5          Eclair
      Android 1.6          September 2009  4          Donut
      Android 1.5          May 2009        3          Cupcake
      Android 1.1          February 2009   2          Base
      Android 1.0          October 2008    1          Base
[3] https://en.wikipedia.org/wiki/Comparison_of_smartphones
[4] https://en.wikipedia.org/wiki/ARM_Cortex-A

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue26865>
_______________________________________

From report at bugs.python.org  Sat Jan 14 12:58:10 2017
From: report at bugs.python.org (Brendan Donegan)
Date: Sat, 14 Jan 2017 17:58:10 +0000
Subject: [issue29276] HTMLParser in Python 2.7 doesn't recognize image tags
 wrapped up in link tags
In-Reply-To: <1484409158.63.0.451865597774.issue29276@psf.upfronthosting.co.za>
Message-ID: <1484416690.98.0.531649105089.issue29276@psf.upfronthosting.co.za>


Brendan Donegan added the comment:

I even get the correct behaviour in 2.7.12:
Python 2.7.12+ (default, Sep 17 2016, 12:08:02) 
[GCC 6.2.0 20160914] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from HTMLParser import HTMLParser
>>> class MyHTMLParser(HTMLParser):
...     def handle_starttag(self, tag, attrs):
...         print("Encountered a start tag: %s" % tag)
... 
>>> parser = MyHTMLParser()
>>> parser.feed('<a href="http://somesite.com/large_image.jpg"><img src="http://somesite.com/small_image.jpg" width="800px" /></a>')
Encountered a start tag: a
Encountered a start tag: img

Ari, can you provide more info about the exact version and platform you are using?

----------
nosy: +brendan-donegan

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29276>
_______________________________________

From report at bugs.python.org  Sat Jan 14 12:59:49 2017
From: report at bugs.python.org (Ari)
Date: Sat, 14 Jan 2017 17:59:49 +0000
Subject: [issue29276] HTMLParser in Python 2.7 doesn't recognize image tags
 wrapped up in link tags
In-Reply-To: <1484409158.63.0.451865597774.issue29276@psf.upfronthosting.co.za>
Message-ID: <1484416789.06.0.950389726728.issue29276@psf.upfronthosting.co.za>


Ari added the comment:

Sorry, it was a false alarm. I had a poorly constructed testcase. I was trying to grab all image links from html markup while simultaneously substituting image links to somesite.com with links to anothersite.com. I had handle_starttag and handle_startendtag defined in my class, but all the image processing resided in handle_starttag, and I didn't realize that handle_starttag is not automatically called when handle_startendtag is executed. So <img src="http://somesite.com/small_image.jpg" width="800px" /> went straight to handle_startendtag, and it did nothing to process the image link because handle_starttag was never called. When I removed handle_startendtag definition, I got the link I wanted (but the absense of handle_startendtag totally mangled the markup I was trying to produce).

----------
resolution:  -> not a bug
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29276>
_______________________________________

From report at bugs.python.org  Sat Jan 14 13:14:32 2017
From: report at bugs.python.org (SilentGhost)
Date: Sat, 14 Jan 2017 18:14:32 +0000
Subject: [issue29275] time module still has Y2K issues note
In-Reply-To: <1484408778.39.0.526783015881.issue29275@psf.upfronthosting.co.za>
Message-ID: <1484417672.36.0.0819679795931.issue29275@psf.upfronthosting.co.za>


SilentGhost added the comment:

The main message there is about the parsing of the two-digit year, so while it might worth it to remove the mention of Y2K, it would only be a minor change.

----------
nosy: +SilentGhost

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29275>
_______________________________________

From report at bugs.python.org  Sat Jan 14 14:42:36 2017
From: report at bugs.python.org (Eryk Sun)
Date: Sat, 14 Jan 2017 19:42:36 +0000
Subject: [issue29256] Windows select() errors out when given no fds to select
 on, which breaks SelectSelector
In-Reply-To: <1484264538.85.0.902649207203.issue29256@psf.upfronthosting.co.za>
Message-ID: <1484422956.61.0.039871250519.issue29256@psf.upfronthosting.co.za>


Eryk Sun added the comment:

Some WinSock functions are just dispatchers that call a provider function. The dispatch table is set up when WinSock (i.e. ws2_32.dll) calls the WSPStartup function [1] of a provider DLL (e.g. mswsock.dll). In the case of select(), it calls the socket provider's WSPSelect function [2]. It determines the provider from the first socket found in the order of readfds, writefds, and exceptfds. It fails if there isn't at least one socket from which it can determine the provider. Also, given this design, each socket in all three sets has to be from the same provider. 

In general, using a dummy socket isn't a good workaround for using select() to implement a sleep function. WSPSelect can't be interrupted (or rather, it can only be interrupted temporarily to execute an asynchronous procedure call that's queued to the thread via QueueUserAPC, after which it resumes waiting). If a script uses select() with a dummy socket to sleep on the main thread, the user won't be able to cancel the wait with Ctrl+C. It would be better in this case if select.select waited on the SIGINT event like the Windows implementation of time.sleep.

[1]: https://msdn.microsoft.com/en-us/library/ms742296
[2]: https://msdn.microsoft.com/en-us/library/ms742289

----------
components: +Windows
nosy: +eryksun, paul.moore, steve.dower, tim.golden, zach.ware

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29256>
_______________________________________

From report at bugs.python.org  Sat Jan 14 14:49:34 2017
From: report at bugs.python.org (INADA Naoki)
Date: Sat, 14 Jan 2017 19:49:34 +0000
Subject: [issue20180] Derby #11: Convert 50 sites to Argument Clinic across 9
 files
In-Reply-To: <1389140162.94.0.578256004005.issue20180@psf.upfronthosting.co.za>
Message-ID: <1484423374.1.0.844364638402.issue20180@psf.upfronthosting.co.za>


INADA Naoki added the comment:

Updated patch for unicodeobject.

@taleinat, could you confirm it?

----------
nosy: +inada.naoki
Added file: http://bugs.python.org/file46292/unicodeobject.c.v6.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue20180>
_______________________________________

From report at bugs.python.org  Sat Jan 14 15:11:02 2017
From: report at bugs.python.org (Eryk Sun)
Date: Sat, 14 Jan 2017 20:11:02 +0000
Subject: [issue23407] os.walk always follows Windows junctions
In-Reply-To: <1423342763.81.0.179122512075.issue23407@psf.upfronthosting.co.za>
Message-ID: <1484424662.43.0.0654021094887.issue23407@psf.upfronthosting.co.za>


Eryk Sun added the comment:

Craig, can you add a patch for issue 29248, including a test based on the "All Users" link?

----------
dependencies: +os.readlink fails on Windows

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23407>
_______________________________________

From report at bugs.python.org  Sat Jan 14 15:24:56 2017
From: report at bugs.python.org (INADA Naoki)
Date: Sat, 14 Jan 2017 20:24:56 +0000
Subject: [issue20180] Derby #11: Convert 50 sites to Argument Clinic across 9
 files
In-Reply-To: <1389140162.94.0.578256004005.issue20180@psf.upfronthosting.co.za>
Message-ID: <1484425496.32.0.053471871876.issue20180@psf.upfronthosting.co.za>


INADA Naoki added the comment:

transmogrify.h uses hack to share docstring.
I can't find straightforward way.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue20180>
_______________________________________

From report at bugs.python.org  Sat Jan 14 15:38:49 2017
From: report at bugs.python.org (Craig Holmquist)
Date: Sat, 14 Jan 2017 20:38:49 +0000
Subject: [issue29248] os.readlink fails on Windows
In-Reply-To: <1484208560.98.0.375529555783.issue29248@psf.upfronthosting.co.za>
Message-ID: <1484426329.57.0.707790628045.issue29248@psf.upfronthosting.co.za>


Craig Holmquist added the comment:

New patch with test.

I'm not sure if C:\Users\All Users and C:\ProgramData exist with those names on non-English installations of Windows.  I set the test to skip if they aren't found.

----------
nosy: +craigh
Added file: http://bugs.python.org/file46293/issue29248-2.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29248>
_______________________________________

From report at bugs.python.org  Sat Jan 14 18:41:38 2017
From: report at bugs.python.org (Michael Felt)
Date: Sat, 14 Jan 2017 23:41:38 +0000
Subject: [issue18235] _sysconfigdata.py wrong on AIX installations
In-Reply-To: <1371429201.94.0.372851990361.issue18235@psf.upfronthosting.co.za>
Message-ID: <1484437298.13.0.118188286751.issue18235@psf.upfronthosting.co.za>


Michael Felt added the comment:

Ok. I shall rebuild from scratch.

When I do, I start from a "clean" system - only the compiler and my mkinstallp helper script.

I am adding the latest zlib for what I would package but I shall forgoe that for this test.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue18235>
_______________________________________

From report at bugs.python.org  Sat Jan 14 22:12:29 2017
From: report at bugs.python.org (Eryk Sun)
Date: Sun, 15 Jan 2017 03:12:29 +0000
Subject: [issue22302] Windows os.path.isabs UNC path bug
In-Reply-To: <1409349502.55.0.177656049078.issue22302@psf.upfronthosting.co.za>
Message-ID: <1484449949.42.0.801314285194.issue22302@psf.upfronthosting.co.za>


Changes by Eryk Sun <eryksun at gmail.com>:


----------
stage:  -> needs patch
versions: +Python 3.6, Python 3.7 -Python 3.3, Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue22302>
_______________________________________

From report at bugs.python.org  Sat Jan 14 22:13:00 2017
From: report at bugs.python.org (Eryk Sun)
Date: Sun, 15 Jan 2017 03:13:00 +0000
Subject: [issue22302] Windows os.path.isabs UNC path bug
In-Reply-To: <1409349502.55.0.177656049078.issue22302@psf.upfronthosting.co.za>
Message-ID: <1484449980.36.0.380401632536.issue22302@psf.upfronthosting.co.za>


Changes by Eryk Sun <eryksun at gmail.com>:


----------
Removed message: http://bugs.python.org/msg226094

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue22302>
_______________________________________

From report at bugs.python.org  Sun Jan 15 02:44:06 2017
From: report at bugs.python.org (Utku Gultopu)
Date: Sun, 15 Jan 2017 07:44:06 +0000
Subject: [issue29278] Python 3.6 build fails with parallel make
Message-ID: <1484466246.11.0.873853911133.issue29278@psf.upfronthosting.co.za>


New submission from Utku Gultopu:

Version Info
============
Linux 4.4.0-59-generic #80-Ubuntu SMP x86_64 x86_64 x86_64 GNU/Linux

Issue
=====
When the multiple jobs option (`make -j`) is specified, build fails after compiling the `structmember.c` file. Subsequent compilation attempts show the following message:

    gcc: internal compiler error: Killed (program cc1)
    Please submit a full bug report,
    with preprocessed source if appropriate.
    See <file:///usr/share/doc/gcc-5/README.Bugs> for instructions.

----------
components: Build
messages: 285503
nosy: ugultopu
priority: normal
severity: normal
status: open
title: Python 3.6 build fails with parallel make
type: compile error
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29278>
_______________________________________

From report at bugs.python.org  Sun Jan 15 03:13:15 2017
From: report at bugs.python.org (INADA Naoki)
Date: Sun, 15 Jan 2017 08:13:15 +0000
Subject: [issue29278] Python 3.6 build fails with parallel make
In-Reply-To: <1484466246.11.0.873853911133.issue29278@psf.upfronthosting.co.za>
Message-ID: <1484467995.78.0.0454704770491.issue29278@psf.upfronthosting.co.za>


INADA Naoki added the comment:

Isn't it a OOM killer?
How much memory your machine has?
How about make -j4? (you can use number of cores instead of 4).

----------
nosy: +inada.naoki

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29278>
_______________________________________

From report at bugs.python.org  Sun Jan 15 03:17:47 2017
From: report at bugs.python.org (Arcadiy Ivanov)
Date: Sun, 15 Jan 2017 08:17:47 +0000
Subject: [issue29279] --with-pydebug optimizes too much
Message-ID: <1484468267.58.0.136858573865.issue29279@psf.upfronthosting.co.za>


New submission from Arcadiy Ivanov:

Around line 1480 of configure.ac:
	    if test "$Py_DEBUG" = 'true' ; then
		# Optimization messes up debuggers, so turn it off for
		# debug builds.
                if "$CC" -v --help 2>/dev/null |grep -- -Og > /dev/null; then
                    OPT="-g -Og -Wall $STRICT_PROTO"
                else
                    OPT="-g -O0 -Wall $STRICT_PROTO"
                fi
	    else
		OPT="-g $WRAP -O3 -Wall $STRICT_PROTO"
	    fi

The "-Og", unfortunately, optimizes too much resulting in local variables and arguments being optimized out:

(gdb) info locals
kwlist = {0x7ffff02c3318 "database", 0x7ffff02c3321 "timeout", 0x7ffff02c3329 "detect_types", 0x7ffff02c3336 "isolation_level", 0x7ffff02c3346 "check_same_thread", 0x7ffff02c3392 "factory", 0x7ffff02c3358 "cached_statements", 
  0x7ffff02c336a "uri", 0x0}
database = 0x7ffff04f7da0 ":memory:"
detect_types = 0
isolation_level = 0x1
factory = 0x7ffff04c7dc0 <pysqlite_ConnectionType>
check_same_thread = 1
cached_statements = 4888694
uri = 0
timeout = 5
(gdb) info args
self = <optimized out>
args = 0x7fffeff6be28
kwargs = 0x0

When "self" is optimized out it's too much optimization for debug mode.

----------
components: Build
messages: 285505
nosy: arcivanov
priority: normal
severity: normal
status: open
title: --with-pydebug optimizes too much
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29279>
_______________________________________

From report at bugs.python.org  Sun Jan 15 03:23:40 2017
From: report at bugs.python.org (Arcadiy Ivanov)
Date: Sun, 15 Jan 2017 08:23:40 +0000
Subject: [issue29279] --with-pydebug optimizes too much
In-Reply-To: <1484468267.58.0.136858573865.issue29279@psf.upfronthosting.co.za>
Message-ID: <1484468620.39.0.124214131381.issue29279@psf.upfronthosting.co.za>


Arcadiy Ivanov added the comment:

gcc (GCC) 6.3.1 20161221 (Red Hat 6.3.1-1)
Fedora 25

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29279>
_______________________________________

From report at bugs.python.org  Sun Jan 15 04:05:03 2017
From: report at bugs.python.org (Georg Sauthoff)
Date: Sun, 15 Jan 2017 09:05:03 +0000
Subject: [issue21258] Add __iter__ support for mock_open
In-Reply-To: <1397671005.18.0.0743040241878.issue21258@psf.upfronthosting.co.za>
Message-ID: <1484471103.77.0.331343101476.issue21258@psf.upfronthosting.co.za>


Changes by Georg Sauthoff <mail at georg.so>:


----------
nosy: +gms

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue21258>
_______________________________________

From report at bugs.python.org  Sun Jan 15 04:06:19 2017
From: report at bugs.python.org (SilentGhost)
Date: Sun, 15 Jan 2017 09:06:19 +0000
Subject: [issue29278] Python 3.6 build fails with parallel make
In-Reply-To: <1484466246.11.0.873853911133.issue29278@psf.upfronthosting.co.za>
Message-ID: <1484471179.01.0.0562754788868.issue29278@psf.upfronthosting.co.za>


SilentGhost added the comment:

Can't reproduce this on 3.7, can you supply additional information, such as what exact revision you're trying to build and what gcc version you're using?

----------
nosy: +SilentGhost

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29278>
_______________________________________

From report at bugs.python.org  Sun Jan 15 04:32:05 2017
From: report at bugs.python.org (Georg Sauthoff)
Date: Sun, 15 Jan 2017 09:32:05 +0000
Subject: [issue21258] Add __iter__ support for mock_open
In-Reply-To: <1397671005.18.0.0743040241878.issue21258@psf.upfronthosting.co.za>
Message-ID: <1484472725.38.0.550584285291.issue21258@psf.upfronthosting.co.za>


Georg Sauthoff added the comment:

For working around this issue on Python 3.5 it is sufficient to overwrite just the `return_value.__iter__` method of the object returned by `mock_open()` with an iterator that calls the mocked `readline()` method until it returns the empty string.

cf. e.g. https://github.com/gsauthof/utility/blob/6489c7215dac341be4e40e5348e64d69461766dd/user-installed.py#L176-L179

This also works in combination with `csv.reader()`, i.e. when calling it with the mocked file object.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue21258>
_______________________________________

From report at bugs.python.org  Sun Jan 15 05:08:05 2017
From: report at bugs.python.org (Roundup Robot)
Date: Sun, 15 Jan 2017 10:08:05 +0000
Subject: [issue29132] shlex.shlex with punctuation_chars and posix doesn't
 handle punctuation next to quotes
In-Reply-To: <1483361783.89.0.00336686739256.issue29132@psf.upfronthosting.co.za>
Message-ID: <20170115100802.13908.4414.57A6378A@psf.io>


Roundup Robot added the comment:

New changeset 7e40c940df14 by Vinay Sajip in branch '3.6':
Fixed #29132: Updated shlex to work better with punctuation chars in POSIX mode.
https://hg.python.org/cpython/rev/7e40c940df14

New changeset c163a76163f7 by Vinay Sajip in branch 'default':
Closes #29132: Merged fix from 3.6.
https://hg.python.org/cpython/rev/c163a76163f7

----------
nosy: +python-dev
resolution:  -> fixed
stage:  -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29132>
_______________________________________

From report at bugs.python.org  Sun Jan 15 09:31:24 2017
From: report at bugs.python.org (Eryk Sun)
Date: Sun, 15 Jan 2017 14:31:24 +0000
Subject: [issue22302] Windows os.path.isabs UNC path bug
In-Reply-To: <1409349502.55.0.177656049078.issue22302@psf.upfronthosting.co.za>
Message-ID: <1484490684.29.0.698593810258.issue22302@psf.upfronthosting.co.za>


Eryk Sun added the comment:

isabs also fails for device paths such as r"\\.\C:", which is an absolute path for opening the C: volume. UNC and device paths (i.e. \\server, \\?, and \\.) should always be considered absolute. Only logical drives (i.e. C:, D:, etc) support drive-relative paths.

Also, join() needs to be smarter in this case:

    >>> os.path.join(r'\\.\C:', 'spam')
    '\\\\.\\C:spam'

It doesn't insert a backslash because the 'drive' ends in a colon. It needs to always insert a backslash for UNC paths, which cannot be reliably identified by checking whether the last character of the drive is a colon.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue22302>
_______________________________________

From report at bugs.python.org  Sun Jan 15 10:36:46 2017
From: report at bugs.python.org (INADA Naoki)
Date: Sun, 15 Jan 2017 15:36:46 +0000
Subject: [issue26110] Speedup method calls 1.2x
In-Reply-To: <1452791347.05.0.923215900831.issue26110@psf.upfronthosting.co.za>
Message-ID: <1484494606.85.0.767196633475.issue26110@psf.upfronthosting.co.za>


INADA Naoki added the comment:

Yury, could you review this?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue26110>
_______________________________________

From report at bugs.python.org  Sun Jan 15 10:40:44 2017
From: report at bugs.python.org (INADA Naoki)
Date: Sun, 15 Jan 2017 15:40:44 +0000
Subject: [issue29279] --with-pydebug optimizes too much
In-Reply-To: <1484468267.58.0.136858573865.issue29279@psf.upfronthosting.co.za>
Message-ID: <1484494844.69.0.424318412864.issue29279@psf.upfronthosting.co.za>


INADA Naoki added the comment:

I feel this is a problem of balance between speed and debugger usability.
I prefer default to -Og because:

* It's recommended by gcc for this purpose.
* --pydebug option is not only for gdb.  I use debug build to run various programs with many assertion.

You can override CFLAGS when default option is not fit for you.

----------
nosy: +inada.naoki

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29279>
_______________________________________

From report at bugs.python.org  Sun Jan 15 11:53:39 2017
From: report at bugs.python.org (Kushal Das)
Date: Sun, 15 Jan 2017 16:53:39 +0000
Subject: =?utf-8?b?W2lzc3VlMjkyNzRdIENoYW5nZSDigJx0ZXN0cyBjYXNlc+KAnSDihpIg4oCc?=
 =?utf-8?q?test_cases=E2=80=9D?=
In-Reply-To: <1484379059.0.0.00296348115264.issue29274@psf.upfronthosting.co.za>
Message-ID: <1484499219.76.0.791632070141.issue29274@psf.upfronthosting.co.za>


Kushal Das added the comment:

The patch looks good to me.

----------
nosy: +kushal.das

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29274>
_______________________________________

From report at bugs.python.org  Sun Jan 15 12:25:14 2017
From: report at bugs.python.org (Tal Einat)
Date: Sun, 15 Jan 2017 17:25:14 +0000
Subject: [issue20180] Derby #11: Convert 50 sites to Argument Clinic across 9
 files
In-Reply-To: <1389140162.94.0.578256004005.issue20180@psf.upfronthosting.co.za>
Message-ID: <1484501114.78.0.407460122941.issue20180@psf.upfronthosting.co.za>


Tal Einat added the comment:

Unfortunately, I won't be able to review this due to lack of time.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue20180>
_______________________________________

From report at bugs.python.org  Sun Jan 15 12:25:57 2017
From: report at bugs.python.org (INADA Naoki)
Date: Sun, 15 Jan 2017 17:25:57 +0000
Subject: [issue29029] Faster positional arguments parsing in
 PyArg_ParseTupleAndKeywords
In-Reply-To: <1482269418.38.0.550958237488.issue29029@psf.upfronthosting.co.za>
Message-ID: <1484501157.0.0.887926778861.issue29029@psf.upfronthosting.co.za>


INADA Naoki added the comment:

LGTM

----------
nosy: +inada.naoki

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29029>
_______________________________________

From report at bugs.python.org  Sun Jan 15 13:03:47 2017
From: report at bugs.python.org (INADA Naoki)
Date: Sun, 15 Jan 2017 18:03:47 +0000
Subject: [issue20180] Derby #11: Convert 50 sites to Argument Clinic across 9
 files
In-Reply-To: <1389140162.94.0.578256004005.issue20180@psf.upfronthosting.co.za>
Message-ID: <1484503427.08.0.316515414886.issue20180@psf.upfronthosting.co.za>


INADA Naoki added the comment:

Serhiy Storchaka, would you review again?

Merging unicodeobject.c.v5.patch into default branch was almost straightforward.
But there was one place I should rewrite.  Here is the part of diff -u v5.patch v6.patch

-- is old docstring w/o AC
+- is current docstring w/o AC
-+ is AC in v5 patch
++ is AC in v6 patch

 -PyDoc_STRVAR(translate__doc__,
 -             "S.translate(table) -> str\n\
 -\n\
--Return a copy of the string S, where all characters have been mapped\n\
--through the given translation table, which must be a mapping of\n\
--Unicode ordinals to Unicode ordinals, strings, or None.\n\
--Unmapped characters are left untouched. Characters mapped to None\n\
--are deleted.");
+-Return a copy of the string S in which each character has been mapped\n\
+-through the given translation table. The table must implement\n\
+-lookup/indexing via __getitem__, for instance a dictionary or list,\n\
+-mapping Unicode ordinals to Unicode ordinals, strings, or None. If\n\
+-this operation raises LookupError, the character is left untouched.\n\
+-Characters mapped to None are deleted.");
 -
 -static PyObject*
 +/*[clinic input]
@@ -1194,14 +1271,14 @@
 +
 +Replace each character in the string using the given translation table.
 +
-+Characters not in the translation table are left untouched.
-+
-+Characters mapped to None are deleted.
++The table must implement lookup/indexing via __getitem__, for instance a
++dictionary or list.  If this operation raises LookupError, the character is
++left untouched.  Characters mapped to None are deleted.
 +[clinic start generated code]*/

----------
versions: +Python 3.7 -Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue20180>
_______________________________________

From report at bugs.python.org  Sun Jan 15 14:03:25 2017
From: report at bugs.python.org (Arthur Vanwalleghen)
Date: Sun, 15 Jan 2017 19:03:25 +0000
Subject: [issue29280] gdbm & ndbm support missing in Windows
Message-ID: <1484507005.86.0.826464747916.issue29280@psf.upfronthosting.co.za>


New submission from Arthur Vanwalleghen:

When I tried to work with GDBM and Ndbm files I saw that their supporting modules were not found.  This can be verified with Pydoc:

Pydoc\Python36-32\lib \dbm(package)\Package Contents\gnu|ndbm

ErrorDuringImport: problem in dbm.gnu - ModuleNotFoundError: No module named '_gdbm' 

ErrorDuringImport: problem in dbm.ndbm - ModuleNotFoundError: No module named '_dbm'

I first noticed this in 3.5 and waited to see if it was fixed in 3.6. I am running Win8.1.

I want to work with files created on Linux/Perl using GDBM or NDBM.  I cannot find any perl support for Python's dbm.dumb. I am looking for an out-of-the-box support for simple key/data files.

----------
components: Windows
messages: 285517
nosy: ArtVan, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: gdbm & ndbm support missing in Windows
type: behavior
versions: Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29280>
_______________________________________

From report at bugs.python.org  Sun Jan 15 14:23:52 2017
From: report at bugs.python.org (miniflow)
Date: Sun, 15 Jan 2017 19:23:52 +0000
Subject: [issue29277] os.getcwd failing on LOFS share
In-Reply-To: <1484413670.89.0.750982203265.issue29277@psf.upfronthosting.co.za>
Message-ID: <1484508232.77.0.393415218621.issue29277@psf.upfronthosting.co.za>


miniflow added the comment:

After performing an strace - it shows that the underlying call to getcwd is indeed returning ENOENT which is not correct.  Does this become a libc bug?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29277>
_______________________________________

From report at bugs.python.org  Sun Jan 15 15:12:17 2017
From: report at bugs.python.org (STINNER Victor)
Date: Sun, 15 Jan 2017 20:12:17 +0000
Subject: [issue29277] os.getcwd failing on LOFS share
In-Reply-To: <1484413670.89.0.750982203265.issue29277@psf.upfronthosting.co.za>
Message-ID: <1484511137.28.0.531444951788.issue29277@psf.upfronthosting.co.za>


STINNER Victor added the comment:

When getcwd() fails with ENOENT, os.getcwd() raises OSError(ENOENT). It's expected and not a bug in Python.

Please report the issue to LOFS.

----------
nosy: +haypo
resolution:  -> not a bug
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29277>
_______________________________________

From report at bugs.python.org  Sun Jan 15 17:07:26 2017
From: report at bugs.python.org (Ivan Levkivskyi)
Date: Sun, 15 Jan 2017 22:07:26 +0000
Subject: [issue29262] Provide a way to check for *real* typing.Union instances
In-Reply-To: <1484322347.31.0.5780880295.issue29262@psf.upfronthosting.co.za>
Message-ID: <1484518046.1.0.299878740158.issue29262@psf.upfronthosting.co.za>


Ivan Levkivskyi added the comment:

In principle, such a function could be added if it is not called ``isinstance``. For example, we could add a helper ``is_union(tp)`` (or maybe also ``is_generic(tp)`` etc). Such function(s) will be simple one-liners wrapping private API in a right way (e.g. using _gorg instead of __origin__ where needed etc).

Guido, what do you think?

----------
nosy: +gvanrossum, levkivskyi
type:  -> enhancement

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29262>
_______________________________________

From report at bugs.python.org  Sun Jan 15 17:17:02 2017
From: report at bugs.python.org (Guido van Rossum)
Date: Sun, 15 Jan 2017 22:17:02 +0000
Subject: [issue29262] Provide a way to check for *real* typing.Union instances
In-Reply-To: <1484322347.31.0.5780880295.issue29262@psf.upfronthosting.co.za>
Message-ID: <1484518622.74.0.62147159137.issue29262@psf.upfronthosting.co.za>


Guido van Rossum added the comment:

Hm, maybe isinstance(t, Union) should actually be allowed? (Though isinstance(x, Union[int, str]) should not be!)  After all we can write isinstance(t, Callable) as well, even though isinstance(x, Callable[[int], int]) is disallowed.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29262>
_______________________________________

From report at bugs.python.org  Sun Jan 15 17:30:41 2017
From: report at bugs.python.org (Ivan Levkivskyi)
Date: Sun, 15 Jan 2017 22:30:41 +0000
Subject: [issue29262] Provide a way to check for *real* typing.Union instances
In-Reply-To: <1484322347.31.0.5780880295.issue29262@psf.upfronthosting.co.za>
Message-ID: <1484519441.21.0.603774787697.issue29262@psf.upfronthosting.co.za>


Ivan Levkivskyi added the comment:

This will be a bit unusual since ``isinstance`` is typically called for instances (i.e. not types) as in ``isinstance(func, Callable)``. But on the other hand this is probably what one would expect when one sees ``isinstance(tp, Union)``. Thus I am fine with doing it this way.

If there are no objections/other ideas, then I will make a PR upstream in next few days.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29262>
_______________________________________

From report at bugs.python.org  Sun Jan 15 17:37:42 2017
From: report at bugs.python.org (Guido van Rossum)
Date: Sun, 15 Jan 2017 22:37:42 +0000
Subject: [issue29262] Provide a way to check for *real* typing.Union instances
In-Reply-To: <1484322347.31.0.5780880295.issue29262@psf.upfronthosting.co.za>
Message-ID: <1484519862.67.0.073332534768.issue29262@psf.upfronthosting.co.za>


Guido van Rossum added the comment:

Hm, let me back-pedal a bit. The situation with Callable is murky, as e.g. isinstance(typing.Tuple[int, int], typing.Callable) returns True.

Maybe we need to take a step back and look at the needs for code that wants to implement runtime type checking more in general? ISTM we have ways to access the parameters of a parameterized type (typically t.__parameters__) but we don't have a reasonable way yet to determine what kind of thing t is.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29262>
_______________________________________

From report at bugs.python.org  Sun Jan 15 17:48:51 2017
From: report at bugs.python.org (Ivan Levkivskyi)
Date: Sun, 15 Jan 2017 22:48:51 +0000
Subject: [issue29262] Provide a way to check for *real* typing.Union instances
In-Reply-To: <1484322347.31.0.5780880295.issue29262@psf.upfronthosting.co.za>
Message-ID: <1484520531.13.0.674627680751.issue29262@psf.upfronthosting.co.za>


Ivan Levkivskyi added the comment:

> Maybe we need to take a step back and look at the needs for code that wants to implement runtime type checking more in general?

I would say that the most convenient way for me would be a set of "inspect-style" simple helpers like ``is_union``, ``is_generic``, ``get_parameters`` (similar to inspect.ismethod, inspect.getargspec etc).

> ISTM we have ways to access the parameters of a parameterized type (typically t.__parameters__) but we don't have a reasonable way yet to determine what kind of thing t is.

There is one way that I see right now: using _gorg. For example, ``_gorg(Tuple[int, str]) is Tuple``, ``_gorg(Callable[..., str]) is Callable``, etc. This will also work for ``Union`` if we relax the assert that requires type to be instance of GenericMeta (now there is a common internal API used by almost everything in typing).

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29262>
_______________________________________

From report at bugs.python.org  Sun Jan 15 17:50:57 2017
From: report at bugs.python.org (Guido van Rossum)
Date: Sun, 15 Jan 2017 22:50:57 +0000
Subject: [issue29262] Provide a way to check for *real* typing.Union instances
In-Reply-To: <1484520531.13.0.674627680751.issue29262@psf.upfronthosting.co.za>
Message-ID: <CAP7+vJJTO-BkZep0axmovph95FPEDay85DkqH-2mFnxSFqdU=w@mail.gmail.com>


Guido van Rossum added the comment:

Maybe a proposal should be discussed as an addendum to PEP 484? Or would
Mark Shannon reject that?

On Sun, Jan 15, 2017 at 2:48 PM, Ivan Levkivskyi <report at bugs.python.org>
wrote:

>
> Ivan Levkivskyi added the comment:
>
> > Maybe we need to take a step back and look at the needs for code that
> wants to implement runtime type checking more in general?
>
> I would say that the most convenient way for me would be a set of
> "inspect-style" simple helpers like ``is_union``, ``is_generic``,
> ``get_parameters`` (similar to inspect.ismethod, inspect.getargspec etc).
>
> > ISTM we have ways to access the parameters of a parameterized type
> (typically t.__parameters__) but we don't have a reasonable way yet to
> determine what kind of thing t is.
>
> There is one way that I see right now: using _gorg. For example,
> ``_gorg(Tuple[int, str]) is Tuple``, ``_gorg(Callable[..., str]) is
> Callable``, etc. This will also work for ``Union`` if we relax the assert
> that requires type to be instance of GenericMeta (now there is a common
> internal API used by almost everything in typing).
>
> ----------
>
> _______________________________________
> Python tracker <report at bugs.python.org>
> <http://bugs.python.org/issue29262>
> _______________________________________
>

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29262>
_______________________________________

From report at bugs.python.org  Sun Jan 15 17:56:46 2017
From: report at bugs.python.org (Ivan Levkivskyi)
Date: Sun, 15 Jan 2017 22:56:46 +0000
Subject: [issue29262] Provide a way to check for *real* typing.Union instances
In-Reply-To: <1484322347.31.0.5780880295.issue29262@psf.upfronthosting.co.za>
Message-ID: <1484521006.76.0.610826727885.issue29262@psf.upfronthosting.co.za>


Ivan Levkivskyi added the comment:

> Maybe a proposal should be discussed as an addendum to PEP 484? Or would
Mark Shannon reject that?

On one hand, I would like to involve a wider audience to this discussion. On the other hand, an addition to the PEP could slow things down. Maybe a discussion on python-dev followed by implementation plus extensive docs would work?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29262>
_______________________________________

From report at bugs.python.org  Sun Jan 15 18:00:47 2017
From: report at bugs.python.org (Guido van Rossum)
Date: Sun, 15 Jan 2017 23:00:47 +0000
Subject: [issue29262] Provide a way to check for *real* typing.Union instances
In-Reply-To: <1484521006.76.0.610826727885.issue29262@psf.upfronthosting.co.za>
Message-ID: <CAP7+vJK8Q4_dzx0qTH2qr6kRoxsx_MHPtQWcZuy1Hf7oLL49wQ@mail.gmail.com>


Guido van Rossum added the comment:

Posting to python-dev would probably cause more friction than a PR for the
PEPs repo. But maybe the best way to do this is to use a third party module
with a proposed API? E.g. typing_inspect. We could then iterate quickly on
the design and implementation there, and we could commit to keeping it in
sync with changes to typing.py, so flying sheep's package could depend on
it.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29262>
_______________________________________

From report at bugs.python.org  Sun Jan 15 18:13:00 2017
From: report at bugs.python.org (Ivan Levkivskyi)
Date: Sun, 15 Jan 2017 23:13:00 +0000
Subject: [issue29262] Provide a way to check for *real* typing.Union instances
In-Reply-To: <1484322347.31.0.5780880295.issue29262@psf.upfronthosting.co.za>
Message-ID: <1484521980.83.0.939745279192.issue29262@psf.upfronthosting.co.za>


Ivan Levkivskyi added the comment:

I have a similar idea. We already have mypy_extensions for runtime counterparts of experimental features. However, the runtime inspections are not related to mypy, so that I am not sure. I am just a bit worried there will be to many things to keep in mind/in sync. What do you think?

By the way maybe later we could add ``typing.inspect`` similar to ``typing.re`` and ``typing.io``?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29262>
_______________________________________

From report at bugs.python.org  Sun Jan 15 18:20:17 2017
From: report at bugs.python.org (Guido van Rossum)
Date: Sun, 15 Jan 2017 23:20:17 +0000
Subject: [issue29262] Provide a way to check for *real* typing.Union instances
In-Reply-To: <1484521980.83.0.939745279192.issue29262@psf.upfronthosting.co.za>
Message-ID: <CAP7+vJLr7RxN97ipzzUCKF=MM34jUBo_jcGTP17mbpsegvPCMQ@mail.gmail.com>


Guido van Rossum added the comment:

I think it should be separate from mypy_extensions, since it's not even
related to mypy.

Regarding typing.re and typing.io, typing.inspect would be a typed version
of the inspect module, so that's not quite the same. (Though I consider
typing.re/io mistakes now -- people seem to mostly ignore it and use the
toplevel names.)

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29262>
_______________________________________

From report at bugs.python.org  Sun Jan 15 18:30:45 2017
From: report at bugs.python.org (Ivan Levkivskyi)
Date: Sun, 15 Jan 2017 23:30:45 +0000
Subject: [issue29262] Provide a way to check for *real* typing.Union instances
In-Reply-To: <1484322347.31.0.5780880295.issue29262@psf.upfronthosting.co.za>
Message-ID: <1484523045.35.0.891458076953.issue29262@psf.upfronthosting.co.za>


Ivan Levkivskyi added the comment:

OK, I agree.

How then it should be done logistically? Should I just make a separate repo on GitHub for this? Or will it be inside typing (like mypy_extesions is inside mypy) but published on PyPI separately?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29262>
_______________________________________

From report at bugs.python.org  Sun Jan 15 18:52:09 2017
From: report at bugs.python.org (Guido van Rossum)
Date: Sun, 15 Jan 2017 23:52:09 +0000
Subject: [issue29262] Provide a way to check for *real* typing.Union instances
In-Reply-To: <1484523045.35.0.891458076953.issue29262@psf.upfronthosting.co.za>
Message-ID: <CAP7+vJKc58xLFYxsz72OM3mmwJ5GQ=5X=Rt5-Cbw14w-G=O-UQ@mail.gmail.com>


Guido van Rossum added the comment:

Up to you, but the latter might make it clearer that the two are to be kept
in sync.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29262>
_______________________________________

From report at bugs.python.org  Sun Jan 15 18:59:25 2017
From: report at bugs.python.org (Ivan Levkivskyi)
Date: Sun, 15 Jan 2017 23:59:25 +0000
Subject: [issue29262] Provide a way to check for *real* typing.Union instances
In-Reply-To: <1484322347.31.0.5780880295.issue29262@psf.upfronthosting.co.za>
Message-ID: <1484524765.8.0.756262531661.issue29262@psf.upfronthosting.co.za>


Ivan Levkivskyi added the comment:

OK, I will make a PR to typing adding a folder (as it is done for mypy_extensions) with a basic set of runtime type inspection functions.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29262>
_______________________________________

From report at bugs.python.org  Sun Jan 15 21:07:48 2017
From: report at bugs.python.org (Martin Panter)
Date: Mon, 16 Jan 2017 02:07:48 +0000
Subject: [issue28997] test_readline.test_nonascii fails on Android
In-Reply-To: <1481965369.72.0.409169776064.issue28997@psf.upfronthosting.co.za>
Message-ID: <1484532468.66.0.943713194332.issue28997@psf.upfronthosting.co.za>


Martin Panter added the comment:

So the problem seems to be that Python assumes Readline?s encoding is UTF-8, but Readline actually uses ASCII (depending on locale variables). The code at the start of the test is supposed to catch when add_history() calls PyUnicode_EncodeLocale() and fails.

I don?t understand the details of UTF-8 vs locale on Android, but maybe we could adjust the encode() and decode() implementations in Modules/readline.c, to account for the Readline library?s idea of the locale encoding. Or maybe we could adjust the temporary setlocale() calls in Modules/readline.c.

If you are happy to declare the Readline library is broken on Android, I now think I would prefer to skip the test based on support.is_android, rather than the previous patches. Otherwise, we risk masking genuine test failures on other platforms. Something like:

@unittest.skipIf(is_android,
    "Gnu Readline disagrees about the locale encoding on Android")
def test_nonascii(self):
    try:
        readline.add_history("\xEB\xEF")
    ...

When you run ?LANG= bash?, it is only Bash and Readline that gets the C locale; the terminal is unchanged. I presume the terminal inputs ? as two UTF-8 bytes, but Readline with the C locale is not aware of UTF-8, and assumes the two bytes are two separate characters.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28997>
_______________________________________

From report at bugs.python.org  Sun Jan 15 21:08:22 2017
From: report at bugs.python.org (Larry Hastings)
Date: Mon, 16 Jan 2017 02:08:22 +0000
Subject: [issue29125] Shell injection via TIX_LIBRARY when using tkinter.tix
In-Reply-To: <1483210830.52.0.920210756798.issue29125@psf.upfronthosting.co.za>
Message-ID: <1484532502.25.0.452831373171.issue29125@psf.upfronthosting.co.za>


Larry Hastings added the comment:

I'll make you a deal.  If you check this in in the next 3 hours, I'll cherry-pick it for 3.5.3.  Otherwise I don't want to hold up the release.  To be honest I'm not sure why it's marked as "release blocker" if it's "low severity".

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29125>
_______________________________________

From report at bugs.python.org  Sun Jan 15 22:03:55 2017
From: report at bugs.python.org (INADA Naoki)
Date: Mon, 16 Jan 2017 03:03:55 +0000
Subject: [issue26110] Speedup method calls 1.2x
In-Reply-To: <1452791347.05.0.923215900831.issue26110@psf.upfronthosting.co.za>
Message-ID: <1484535835.6.0.201382666058.issue26110@psf.upfronthosting.co.za>


Changes by INADA Naoki <songofacandy at gmail.com>:


Added file: http://bugs.python.org/file46294/call-method-doc5.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue26110>
_______________________________________

From report at bugs.python.org  Sun Jan 15 22:35:34 2017
From: report at bugs.python.org (Thane Brimhall)
Date: Mon, 16 Jan 2017 03:35:34 +0000
Subject: [issue29235] Allow profile/cProfile to be used as context managers
In-Reply-To: <1484097220.43.0.363931456874.issue29235@psf.upfronthosting.co.za>
Message-ID: <1484537734.58.0.873108033272.issue29235@psf.upfronthosting.co.za>


Thane Brimhall added the comment:

So this is my first time contributing to Python, but here's a (trivial) patch for this issue. Let me know what else is required to make this happen. I suspect unit tests and documentation updates?

----------
keywords: +patch
Added file: http://bugs.python.org/file46295/profileContextManager.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29235>
_______________________________________

From report at bugs.python.org  Sun Jan 15 22:40:42 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Mon, 16 Jan 2017 03:40:42 +0000
Subject: [issue29235] Allow profile/cProfile to be used as context managers
In-Reply-To: <1484097220.43.0.363931456874.issue29235@psf.upfronthosting.co.za>
Message-ID: <1484538042.23.0.986434462707.issue29235@psf.upfronthosting.co.za>


Xiang Zhang added the comment:

> I suspect unit tests and documentation updates?

Yes. Code alone is not enough. It's better to have tests and documentation. Also, it's appreciated to sign the CLA: https://www.python.org/psf/contrib/contrib-form/. :-)

----------
nosy: +xiang.zhang
stage: needs patch -> test needed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29235>
_______________________________________

From report at bugs.python.org  Sun Jan 15 22:45:28 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Mon, 16 Jan 2017 03:45:28 +0000
Subject: [issue29279] --with-pydebug optimizes too much
In-Reply-To: <1484468267.58.0.136858573865.issue29279@psf.upfronthosting.co.za>
Message-ID: <1484538328.41.0.544975255294.issue29279@psf.upfronthosting.co.za>


Raymond Hettinger added the comment:

> You can override CFLAGS when default option is not fit for you.

I concur with INADA and agree that the existing defaults should be left alone because they've worked well for many years and they provide all tolerably fast result for running the entire test suite (which is important for many developers).

----------
nosy: +rhettinger
resolution:  -> rejected
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29279>
_______________________________________

From report at bugs.python.org  Sun Jan 15 22:53:46 2017
From: report at bugs.python.org (Thane Brimhall)
Date: Mon, 16 Jan 2017 03:53:46 +0000
Subject: [issue29235] Allow profile/cProfile to be used as context managers
In-Reply-To: <1484097220.43.0.363931456874.issue29235@psf.upfronthosting.co.za>
Message-ID: <1484538826.68.0.538232020359.issue29235@psf.upfronthosting.co.za>


Thane Brimhall added the comment:

I've signed the agreement. I will do the necessary research to figure out how to do unit tests and documentation updates.

I should also mention that while maintaining API-compatibility with `profile` was a stated goal, it turns out that the pure-python profiler actually does not have the enable() and disable() methods that the context manager needs.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29235>
_______________________________________

From report at bugs.python.org  Mon Jan 16 01:13:25 2017
From: report at bugs.python.org (Martin Panter)
Date: Mon, 16 Jan 2017 06:13:25 +0000
Subject: [issue13886] readline-related test_builtin failure
In-Reply-To: <1327659819.71.0.475160045664.issue13886@psf.upfronthosting.co.za>
Message-ID: <1484547205.32.0.987381760853.issue13886@psf.upfronthosting.co.za>


Martin Panter added the comment:

V3 of my patch skips the Readline tests in cases involving non-ASCII bytes when the locale seen by Readline would be ASCII. Readline may translate the non-ASCII bytes to escape sequences, and treat them as special Meta (Alt) key combinations. This behaviour depends on Readline configuration (?set convert-meta off? in /etc/inputrc in my case).

It also includes a potential workaround for Android, depending on the resolution of Issue 28997.

----------
Added file: http://bugs.python.org/file46296/input-readline.v3.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue13886>
_______________________________________

From report at bugs.python.org  Mon Jan 16 02:41:20 2017
From: report at bugs.python.org (Larry Hastings)
Date: Mon, 16 Jan 2017 07:41:20 +0000
Subject: [issue29125] Shell injection via TIX_LIBRARY when using tkinter.tix
In-Reply-To: <1483210830.52.0.920210756798.issue29125@psf.upfronthosting.co.za>
Message-ID: <1484552480.02.0.857995996078.issue29125@psf.upfronthosting.co.za>


Larry Hastings added the comment:

If it "has a small attack surface" and affects "a very small number of applications", I don't think it's a release blocker.  Demoting to "high" priority, which will permit me to release 3.5.3.

----------
priority: release blocker -> high

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29125>
_______________________________________

From report at bugs.python.org  Mon Jan 16 03:04:35 2017
From: report at bugs.python.org (Larry Hastings)
Date: Mon, 16 Jan 2017 08:04:35 +0000
Subject: [issue29057] Compiler failure on Mac OS X - sys/random.h
In-Reply-To: <1482514649.72.0.460227366509.issue29057@psf.upfronthosting.co.za>
Message-ID: <1484553875.08.0.104335009167.issue29057@psf.upfronthosting.co.za>


Larry Hastings added the comment:

Releasing 3.5.3 even though technically this is an open release blocker.  IIUC the fix is checked in, and fixed the issue for OS X.  We don't know whether or not it is also fixed on OpenBSD, because we don't know anybody running OpenBSD, and nobody contacted us regarding OpenBSD during the 3.5.3 RC period.  Anyway I'm declaring victory and moving on.  Maybe we can mark this bug closed?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29057>
_______________________________________

From report at bugs.python.org  Mon Jan 16 03:07:37 2017
From: report at bugs.python.org (Roundup Robot)
Date: Mon, 16 Jan 2017 08:07:37 +0000
Subject: [issue10513] sqlite3.InterfaceError after commit
In-Reply-To: <1290515734.43.0.643837241692.issue10513@psf.upfronthosting.co.za>
Message-ID: <20170116080734.56039.52304.3652CAF7@psf.io>


Roundup Robot added the comment:

New changeset 74a68d86569c by Benjamin Peterson in branch '2.7':
revert dd13098a5dc2 (#29006, #10513)
https://hg.python.org/cpython/rev/74a68d86569c

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue10513>
_______________________________________

From report at bugs.python.org  Mon Jan 16 03:07:37 2017
From: report at bugs.python.org (Roundup Robot)
Date: Mon, 16 Jan 2017 08:07:37 +0000
Subject: [issue29006] 2.7.13 _sqlite more prone to "database table is locked"
In-Reply-To: <1482095939.25.0.891516594682.issue29006@psf.upfronthosting.co.za>
Message-ID: <20170116080734.56039.70602.46D37750@psf.io>


Roundup Robot added the comment:

New changeset 74a68d86569c by Benjamin Peterson in branch '2.7':
revert dd13098a5dc2 (#29006, #10513)
https://hg.python.org/cpython/rev/74a68d86569c

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29006>
_______________________________________

From report at bugs.python.org  Mon Jan 16 03:23:41 2017
From: report at bugs.python.org (Roundup Robot)
Date: Mon, 16 Jan 2017 08:23:41 +0000
Subject: [issue26110] Speedup method calls 1.2x
In-Reply-To: <1452791347.05.0.923215900831.issue26110@psf.upfronthosting.co.za>
Message-ID: <20170116082338.29542.30053.A4ECCE03@psf.io>


Roundup Robot added the comment:

New changeset a6241b2073c6 by INADA Naoki in branch 'default':
Issue #26110: Add document for LOAD_METHOD and CALL_METHOD opcode.
https://hg.python.org/cpython/rev/a6241b2073c6

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue26110>
_______________________________________

From report at bugs.python.org  Mon Jan 16 03:23:59 2017
From: report at bugs.python.org (INADA Naoki)
Date: Mon, 16 Jan 2017 08:23:59 +0000
Subject: [issue26110] Speedup method calls 1.2x
In-Reply-To: <1452791347.05.0.923215900831.issue26110@psf.upfronthosting.co.za>
Message-ID: <1484555039.34.0.686580435604.issue26110@psf.upfronthosting.co.za>


Changes by INADA Naoki <songofacandy at gmail.com>:


----------
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue26110>
_______________________________________

From report at bugs.python.org  Mon Jan 16 04:19:59 2017
From: report at bugs.python.org (Alexey Popravka)
Date: Mon, 16 Jan 2017 09:19:59 +0000
Subject: [issue29281] json.loads documentation missing "versionchanged"
 statement
Message-ID: <1484558399.93.0.792457597374.issue29281@psf.upfronthosting.co.za>


New submission from Alexey Popravka:

json.loads function was changed in Python 3.6 to accept
bytes and bytearrays, however documentation is missing
`versionchanged` block describing this changes.

----------
assignee: docs at python
components: Documentation
messages: 285545
nosy: Alexey Popravka, docs at python
priority: normal
severity: normal
status: open
title: json.loads documentation missing "versionchanged" statement
versions: Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29281>
_______________________________________

From report at bugs.python.org  Mon Jan 16 04:28:40 2017
From: report at bugs.python.org (Michael Foord)
Date: Mon, 16 Jan 2017 09:28:40 +0000
Subject: =?utf-8?b?W2lzc3VlMjkyNzRdIENoYW5nZSDigJx0ZXN0cyBjYXNlc+KAnSDihpIg4oCc?=
 =?utf-8?q?test_cases=E2=80=9D?=
In-Reply-To: <1484379059.0.0.00296348115264.issue29274@psf.upfronthosting.co.za>
Message-ID: <1484558920.27.0.462990243528.issue29274@psf.upfronthosting.co.za>


Michael Foord added the comment:

Yep, looks like an improvement to me!

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29274>
_______________________________________

From report at bugs.python.org  Mon Jan 16 04:43:56 2017
From: report at bugs.python.org (INADA Naoki)
Date: Mon, 16 Jan 2017 09:43:56 +0000
Subject: [issue20180] Derby #11: Convert 50 sites to Argument Clinic across 9
 files
In-Reply-To: <1389140162.94.0.578256004005.issue20180@psf.upfronthosting.co.za>
Message-ID: <1484559836.63.0.840159500551.issue20180@psf.upfronthosting.co.za>


Changes by INADA Naoki <songofacandy at gmail.com>:


Added file: http://bugs.python.org/file46297/unicodeobject.c.v7.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue20180>
_______________________________________

From report at bugs.python.org  Mon Jan 16 05:01:37 2017
From: report at bugs.python.org (Juraj Sukop)
Date: Mon, 16 Jan 2017 10:01:37 +0000
Subject: [issue29282] Fused multiply-add: proposal to add math.fma()
Message-ID: <1484560897.19.0.0955404368118.issue29282@psf.upfronthosting.co.za>


New submission from Juraj Sukop:

Fused multiply-add (henceforth FMA) is an operation which calculates the product of two numbers and then the sum of the product and a third number with just one floating-point rounding. More concretely:

    r = x*y + z

The value of `r` is the same as if the RHS was calculated with infinite precision and the rounded to a 32-bit single-precision or 64-bit double-precision floating-point number [1].

Even though one FMA CPU instruction might be calculated faster than the two separate instructions for multiply and add, its main advantage comes from the increased precision of numerical computations that involve the accumulation of products. Examples which benefit from using FMA are: dot product [2], compensated arithmetic [3], polynomial evaluation [4], matrix multiplication, Newton's method and many more [5].

C99 includes [6] `fma` function to `math.h` and emulates the calculation if the FMA instruction is not present on the host CPU [7]. PEP 7 states that "Python versions greater than or equal to 3.6 use C89 with several select C99 features" and that "Future C99 features may be added to this list in the future depending on compiler support" [8].

This proposal is then about adding new `fma` function with the following signature to `math` module:

    math.fma(x, y, z)
    
'''Return a float representing the result of the operation `x*y + z` with single rounding error, as defined by the platform C library. The result is the same as if the operation was carried with infinite precision and rounded to a floating-point number.'''

Attached is a simple module for Python 3 demonstrating the fused multiply-add operation. On my machine, `example.py` prints:

    40037.524591982365   horner_double
    40037.48821639768    horner_fma
    40037.49486325783    horner_compensated
    40037.49486325783    horner_decimal

[1] https://en.wikipedia.org/wiki/Multiply%E2%80%93accumulate_operation
[2] S. Graillat, P. Langlois, N. Louvet. Accurate dot products with FMA. 2006
[3] S. Graillat, Accurate Floating Point Product and Exponentiation. 2007.
[4] S. Graillat, P. Langlois, N. Louvet. Improving the compensated Horner scheme with a Fused Multiply and Add. 2006
[5] J.-M. Muller, N. Brisebarre, F. de Dinechin, C.-P. Jeannerod, V. Lef?vre, G. Melquiond, N. Revol, D. Stehl?, S. Torres. Handbook of Floating-Point Arithmetic. 2010. Chapter 5
[6] ISO/IEC 9899:TC3, "7.12.13.1 The fma functions", Committee Draft - Septermber 7, 2007
[7] https://git.musl-libc.org/cgit/musl/tree/src/math/fma.c
[8] https://www.python.org/dev/peps/pep-0007/

----------
components: Library (Lib)
messages: 285547
nosy: juraj.sukop
priority: normal
severity: normal
status: open
title: Fused multiply-add: proposal to add math.fma()
type: enhancement
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29282>
_______________________________________

From report at bugs.python.org  Mon Jan 16 05:02:24 2017
From: report at bugs.python.org (Juraj Sukop)
Date: Mon, 16 Jan 2017 10:02:24 +0000
Subject: [issue29282] Fused multiply-add: proposal to add math.fma()
In-Reply-To: <1484560897.19.0.0955404368118.issue29282@psf.upfronthosting.co.za>
Message-ID: <1484560944.86.0.140911110993.issue29282@psf.upfronthosting.co.za>


Changes by Juraj Sukop <juraj.sukop at gmail.com>:


Added file: http://bugs.python.org/file46298/xmathmodule.c

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29282>
_______________________________________

From report at bugs.python.org  Mon Jan 16 05:02:37 2017
From: report at bugs.python.org (Juraj Sukop)
Date: Mon, 16 Jan 2017 10:02:37 +0000
Subject: [issue29282] Fused multiply-add: proposal to add math.fma()
In-Reply-To: <1484560897.19.0.0955404368118.issue29282@psf.upfronthosting.co.za>
Message-ID: <1484560957.96.0.0073425758236.issue29282@psf.upfronthosting.co.za>


Changes by Juraj Sukop <juraj.sukop at gmail.com>:


Added file: http://bugs.python.org/file46299/example.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29282>
_______________________________________

From report at bugs.python.org  Mon Jan 16 05:02:45 2017
From: report at bugs.python.org (Juraj Sukop)
Date: Mon, 16 Jan 2017 10:02:45 +0000
Subject: [issue29282] Fused multiply-add: proposal to add math.fma()
In-Reply-To: <1484560897.19.0.0955404368118.issue29282@psf.upfronthosting.co.za>
Message-ID: <1484560965.54.0.732696200248.issue29282@psf.upfronthosting.co.za>


Changes by Juraj Sukop <juraj.sukop at gmail.com>:


Added file: http://bugs.python.org/file46300/setup.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29282>
_______________________________________

From report at bugs.python.org  Mon Jan 16 05:03:00 2017
From: report at bugs.python.org (STINNER Victor)
Date: Mon, 16 Jan 2017 10:03:00 +0000
Subject: [issue29282] Fused multiply-add: proposal to add math.fma()
In-Reply-To: <1484560897.19.0.0955404368118.issue29282@psf.upfronthosting.co.za>
Message-ID: <1484560980.38.0.191891652566.issue29282@psf.upfronthosting.co.za>


Changes by STINNER Victor <victor.stinner at gmail.com>:


----------
nosy: +mark.dickinson

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29282>
_______________________________________

From report at bugs.python.org  Mon Jan 16 05:03:26 2017
From: report at bugs.python.org (STINNER Victor)
Date: Mon, 16 Jan 2017 10:03:26 +0000
Subject: [issue29282] Fused multiply-add: proposal to add math.fma()
In-Reply-To: <1484560897.19.0.0955404368118.issue29282@psf.upfronthosting.co.za>
Message-ID: <1484561006.24.0.357438037268.issue29282@psf.upfronthosting.co.za>


Changes by STINNER Victor <victor.stinner at gmail.com>:


----------
nosy: +haypo

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29282>
_______________________________________

From report at bugs.python.org  Mon Jan 16 05:03:57 2017
From: report at bugs.python.org (flying sheep)
Date: Mon, 16 Jan 2017 10:03:57 +0000
Subject: [issue29262] Provide a way to check for *real* typing.Union instances
In-Reply-To: <1484322347.31.0.5780880295.issue29262@psf.upfronthosting.co.za>
Message-ID: <1484561037.69.0.944534639977.issue29262@psf.upfronthosting.co.za>


flying sheep added the comment:

Cool! This set of basic initial check will consist of all the is_* functions that were mentioned right?

FWIW I also think that this is the way to go, as it?s not obvious if the semantics should be ?conforms to this type annotation? or ?is a type annotation of that kind? or other variants.

In case this isn?t already too much future think: What should be the way forward from there? E.g. when working with Union[A, B], you will probably want to get ?(A, B)?.

So will that mean more introspection functions (like union_types(Union[str,int]),
or public APIs for typings (e.g. a_union.__iter__() or a_union.types)?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29262>
_______________________________________

From report at bugs.python.org  Mon Jan 16 05:04:40 2017
From: report at bugs.python.org (STINNER Victor)
Date: Mon, 16 Jan 2017 10:04:40 +0000
Subject: [issue29282] Fused multiply-add: proposal to add math.fma()
In-Reply-To: <1484560897.19.0.0955404368118.issue29282@psf.upfronthosting.co.za>
Message-ID: <1484561080.51.0.134610480783.issue29282@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Thread on python-ideas:
https://mail.python.org/pipermail/python-ideas/2017-January/044300.html

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29282>
_______________________________________

From report at bugs.python.org  Mon Jan 16 05:09:47 2017
From: report at bugs.python.org (Antoine Pitrou)
Date: Mon, 16 Jan 2017 10:09:47 +0000
Subject: [issue29282] Fused multiply-add: proposal to add math.fma()
In-Reply-To: <1484560897.19.0.0955404368118.issue29282@psf.upfronthosting.co.za>
Message-ID: <1484561387.74.0.409616203129.issue29282@psf.upfronthosting.co.za>


Antoine Pitrou added the comment:

What's the point of adding this in the math module rather than a more specialized library like Numpy?

----------
nosy: +pitrou

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29282>
_______________________________________

From report at bugs.python.org  Mon Jan 16 05:40:50 2017
From: report at bugs.python.org (Juraj Sukop)
Date: Mon, 16 Jan 2017 10:40:50 +0000
Subject: [issue29282] Fused multiply-add: proposal to add math.fma()
In-Reply-To: <1484560897.19.0.0955404368118.issue29282@psf.upfronthosting.co.za>
Message-ID: <1484563250.55.0.660726558565.issue29282@psf.upfronthosting.co.za>


Juraj Sukop added the comment:

I would say because it has wide applicability, especially considering the amount of code it adds. It is similar in spirit to `copysign` or `fsum` which are already there and C99 includes it anyway. For simpler things like dot product or polynomial evaluation importing Numpy might be too much of an dependency.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29282>
_______________________________________

From report at bugs.python.org  Mon Jan 16 05:46:49 2017
From: report at bugs.python.org (Antoine Pitrou)
Date: Mon, 16 Jan 2017 10:46:49 +0000
Subject: [issue29282] Fused multiply-add: proposal to add math.fma()
In-Reply-To: <1484560897.19.0.0955404368118.issue29282@psf.upfronthosting.co.za>
Message-ID: <1484563609.87.0.607429907067.issue29282@psf.upfronthosting.co.za>


Antoine Pitrou added the comment:

I don't know. If I want to compute a dot product, the first thing I'll do is import Numpy and then use the `@` operator on Numpy arrays.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29282>
_______________________________________

From report at bugs.python.org  Mon Jan 16 06:05:48 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Mon, 16 Jan 2017 11:05:48 +0000
Subject: [issue29282] Fused multiply-add: proposal to add math.fma()
In-Reply-To: <1484560897.19.0.0955404368118.issue29282@psf.upfronthosting.co.za>
Message-ID: <1484564748.98.0.0737919727642.issue29282@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

The performance argument unlikely is applicable in this case. I suppose that an overhead of function call in Python make two operators faster than one function call.

Alternatives to fma() for exact computations are integer arithmetic (if all values can be scaled to integers), fractions and decimal numbers.

But since fma() is a part of C (C99), C++ (C++11) and POSIX (POSIX.1-2001) standards for long time, I don't have objections against including it in the math module.

----------
nosy: +serhiy.storchaka

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29282>
_______________________________________

From report at bugs.python.org  Mon Jan 16 06:12:56 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Mon, 16 Jan 2017 11:12:56 +0000
Subject: [issue20180] Derby #11: Convert 50 sites to Argument Clinic across 9
 files
In-Reply-To: <1389140162.94.0.578256004005.issue20180@psf.upfronthosting.co.za>
Message-ID: <1484565176.07.0.120748761552.issue20180@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

LGTM again.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue20180>
_______________________________________

From report at bugs.python.org  Mon Jan 16 06:14:45 2017
From: report at bugs.python.org (jan matejek)
Date: Mon, 16 Jan 2017 11:14:45 +0000
Subject: [issue29283] duplicate README in site-packages
Message-ID: <1484565285.81.0.453565653975.issue29283@psf.upfronthosting.co.za>


New submission from jan matejek:

Lib/site-packages directory now contains README.txt in addition to README. Both files are identical. One of them should probably go away?

----------
components: Installation
messages: 285555
nosy: matejcik
priority: normal
severity: normal
status: open
title: duplicate README in site-packages
type: enhancement
versions: Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29283>
_______________________________________

From report at bugs.python.org  Mon Jan 16 06:20:22 2017
From: report at bugs.python.org (jan matejek)
Date: Mon, 16 Jan 2017 11:20:22 +0000
Subject: [issue1294959] Problems with /usr/lib64 builds.
Message-ID: <1484565622.45.0.536853711645.issue1294959@psf.upfronthosting.co.za>


jan matejek added the comment:

Attached is a patch that I'd like to propose for inclusion.

It introduces a new configure option "--with-custom-platlibdir=<name>", which defaults to `basename $libdir`. This is converted to makefile variable "platlibdir", which is used in getpath.c to generate value of the lib_python variable.

sysconfig and distutils.sysconfig retrieve the variable from makefile (i learned about existence of _sysconfigdata so that is where the variable now lives) to correctly substitute in posix_prefix (and unix_prefix for distutils install) scheme.

Sysconfig is then used in pydoc and trace module, instead of locally calculating paths. Perhaps distutils.sysconfig should also use more of sysconfig instead of duplicating the functionality?

As it stands, the python stdlib is installed into /usr/$platlibdir, because I changed SCRIPTDIR to $libdir instead of $prefix/lib. Maybe this should be also separately configurable?
In any case, third-party modules get installed either into /usr/lib or /usr/$platlibdir depending on whether they are arch-independent or not. Hence the final modification in site.py that adds /usr/lib to search path on systems where $platlibdir != "lib".

This caused test_sysconfig to fail because it checks that posix_user and posix_prefix schemes generate "similar" paths. I have left the posix_user scheme alone, so all its paths are based on "lib", where the posix_prefix scheme uses $platlibdir. Maybe posix_user scheme should be modified as well?
Anyway, for now, I have accounted for the changes in the test.

What now?
(please don't make me write a PEP please don't make me write a PEP please don't make......)

----------
versions: +Python 3.7
Added file: http://bugs.python.org/file46301/python-3.6.0-multilib-new.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue1294959>
_______________________________________

From report at bugs.python.org  Mon Jan 16 06:36:54 2017
From: report at bugs.python.org (Mark Dickinson)
Date: Mon, 16 Jan 2017 11:36:54 +0000
Subject: [issue29282] Fused multiply-add: proposal to add math.fma()
In-Reply-To: <1484560897.19.0.0955404368118.issue29282@psf.upfronthosting.co.za>
Message-ID: <1484566614.29.0.263907414409.issue29282@psf.upfronthosting.co.za>


Mark Dickinson added the comment:

> The performance argument unlikely is applicable in this case.

Agreed. This is mainly about accuracy, not speed: the FMA operation is a fundamental building block for floating-point arithmetic, is useful in some numerical algorithms, and essential in others (especially when doing things like double-double arithmetic). It would be valuable to have when prototyping numerical algorithms in pure Python.

Given that it's supported in C99 and on current Windows, I'm +1 on including it in the math module.

Note that implementing this it not quite as straightforward as simply wrapping the libm version, since we'll also want the correct exceptional behaviour, for consistency with the rest of the math module: i.e., we should be raising ValueError where the fma operation would signal the invalid FPE, and OverflowError where the fma operation would signal the overflow FPE.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29282>
_______________________________________

From report at bugs.python.org  Mon Jan 16 06:41:39 2017
From: report at bugs.python.org (Roundup Robot)
Date: Mon, 16 Jan 2017 11:41:39 +0000
Subject: [issue20180] Derby #11: Convert 50 sites to Argument Clinic across 9
 files
In-Reply-To: <1389140162.94.0.578256004005.issue20180@psf.upfronthosting.co.za>
Message-ID: <20170116114128.56526.64049.8D388836@psf.io>


Roundup Robot added the comment:

New changeset 27dc9a1c061e by INADA Naoki in branch 'default':
Issue #20180: convert unicode methods to AC.
https://hg.python.org/cpython/rev/27dc9a1c061e

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue20180>
_______________________________________

From report at bugs.python.org  Mon Jan 16 06:58:34 2017
From: report at bugs.python.org (INADA Naoki)
Date: Mon, 16 Jan 2017 11:58:34 +0000
Subject: [issue20180] Derby #11: Convert 50 sites to Argument Clinic across 9
 files
In-Reply-To: <1389140162.94.0.578256004005.issue20180@psf.upfronthosting.co.za>
Message-ID: <1484567914.02.0.296179511447.issue20180@psf.upfronthosting.co.za>


INADA Naoki added the comment:

Thank you, Martin and Serhiy.

As http://bugs.python.org/issue20180#msg247987 , remained module to converted
is transmogrify.h.

But docstrings of methods in transmogrify.h were moved to bytes_methods.c in issue26765.

May I close this issue for now?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue20180>
_______________________________________

From report at bugs.python.org  Mon Jan 16 07:30:08 2017
From: report at bugs.python.org (Denny Weinberg)
Date: Mon, 16 Jan 2017 12:30:08 +0000
Subject: [issue27400] Datetime NoneType after calling Py_Finalize and
 Py_Initialize
In-Reply-To: <1467035358.55.0.678180807375.issue27400@psf.upfronthosting.co.za>
Message-ID: <1484569808.79.0.317062878429.issue27400@psf.upfronthosting.co.za>


Denny Weinberg added the comment:

Any news here? 3.6.0 is also affected by this bug.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue27400>
_______________________________________

From report at bugs.python.org  Mon Jan 16 07:42:31 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Mon, 16 Jan 2017 12:42:31 +0000
Subject: [issue20180] Derby #11: Convert 50 sites to Argument Clinic across 9
 files
In-Reply-To: <1389140162.94.0.578256004005.issue20180@psf.upfronthosting.co.za>
Message-ID: <1484570551.4.0.447379290346.issue20180@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

Seems you haven't updated generated AC files.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue20180>
_______________________________________

From report at bugs.python.org  Mon Jan 16 07:49:25 2017
From: report at bugs.python.org (Roundup Robot)
Date: Mon, 16 Jan 2017 12:49:25 +0000
Subject: [issue20180] Derby #11: Convert 50 sites to Argument Clinic across 9
 files
In-Reply-To: <1389140162.94.0.578256004005.issue20180@psf.upfronthosting.co.za>
Message-ID: <20170116124921.13986.2656.F1745F98@psf.io>


Roundup Robot added the comment:

New changeset 01b06ca45f64 by INADA Naoki in branch 'default':
Issue #20180: forgot to update AC output.
https://hg.python.org/cpython/rev/01b06ca45f64

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue20180>
_______________________________________

From report at bugs.python.org  Mon Jan 16 07:49:52 2017
From: report at bugs.python.org (INADA Naoki)
Date: Mon, 16 Jan 2017 12:49:52 +0000
Subject: [issue20180] Derby #11: Convert 50 sites to Argument Clinic across 9
 files
In-Reply-To: <1389140162.94.0.578256004005.issue20180@psf.upfronthosting.co.za>
Message-ID: <1484570992.41.0.911918033389.issue20180@psf.upfronthosting.co.za>


INADA Naoki added the comment:

Uhhh!  I'm sorry.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue20180>
_______________________________________

From report at bugs.python.org  Mon Jan 16 08:11:01 2017
From: report at bugs.python.org (Mark Dickinson)
Date: Mon, 16 Jan 2017 13:11:01 +0000
Subject: [issue29282] Fused multiply-add: proposal to add math.fma()
In-Reply-To: <1484560897.19.0.0955404368118.issue29282@psf.upfronthosting.co.za>
Message-ID: <1484572261.3.0.37755491484.issue29282@psf.upfronthosting.co.za>


Mark Dickinson added the comment:

An implementation note: IEEE 754-2008 leaves it to the implementation to decide whether FMA operations like:

    0 * inf + nan

and

    inf * 0 + nan

(where nan represents a quiet NaN and the inf and 0 can have arbitrary signs) signal the invalid operation FPE or not. (Ref: 7.2(c) in IEEE 754-2008.) 

I'd suggest that in this case we follow what Intel does in its x64 chips with FMA3 support.(according to ). If I'm reading the table in section 2.3 of the Intel Advanced Vector Extensions Programming Reference correctly, Intel does *not* signal the invalid operation FPE in this case. That is, we're following the usual rule of quiet NaN in => quiet NaN out with no exception. This does unfortunately conflict with the IBM decimal specification and Python's decimal module, where these operations *do* set the "invalid" flag (see the spec, and test fmax0809 in the decimal test set).

----------
nosy: +skrah

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29282>
_______________________________________

From report at bugs.python.org  Mon Jan 16 08:14:42 2017
From: report at bugs.python.org (Antoine Pitrou)
Date: Mon, 16 Jan 2017 13:14:42 +0000
Subject: [issue29282] Fused multiply-add: proposal to add math.fma()
In-Reply-To: <1484560897.19.0.0955404368118.issue29282@psf.upfronthosting.co.za>
Message-ID: <1484572482.32.0.805891272222.issue29282@psf.upfronthosting.co.za>


Antoine Pitrou added the comment:

Isn't the behaviour of quiet NaNs kindof implementation-dependent already?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29282>
_______________________________________

From report at bugs.python.org  Mon Jan 16 08:26:09 2017
From: report at bugs.python.org (Mark Dickinson)
Date: Mon, 16 Jan 2017 13:26:09 +0000
Subject: [issue29282] Fused multiply-add: proposal to add math.fma()
In-Reply-To: <1484560897.19.0.0955404368118.issue29282@psf.upfronthosting.co.za>
Message-ID: <1484573169.45.0.240908043673.issue29282@psf.upfronthosting.co.za>


Mark Dickinson added the comment:

> Isn't the behaviour of quiet NaNs kindof implementation-dependent already?

Not as far as IEEE 754-2008 is concerned, and not as far as Python's math module is concerned, either: handling of special cases is, as far as I know, both consistent across platforms and compliant with IEEE 754. That's not true across Python as a whole, but it should be true for the math module.

If you find an exception to the above statement, please do open a bug report!

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29282>
_______________________________________

From report at bugs.python.org  Mon Jan 16 09:37:55 2017
From: report at bugs.python.org (Chi Hsuan Yen)
Date: Mon, 16 Jan 2017 14:37:55 +0000
Subject: [issue27659] Prohibit implicit C function declarations
In-Reply-To: <1469948678.43.0.816389049938.issue27659@psf.upfronthosting.co.za>
Message-ID: <1484577475.17.0.914409728336.issue27659@psf.upfronthosting.co.za>


Chi Hsuan Yen added the comment:

Thanks for the comment and sorry for the mistake. Here's another updated patch.

In PEP7:

> Use 4-space indents and no tabs at all.

Does that apply to configuration files, too?

----------
Added file: http://bugs.python.org/file46302/prohibit-implicit-function-declarations.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue27659>
_______________________________________

From report at bugs.python.org  Mon Jan 16 10:53:50 2017
From: report at bugs.python.org (STINNER Victor)
Date: Mon, 16 Jan 2017 15:53:50 +0000
Subject: [issue29259] Add tp_fastcall to PyTypeObject: support FASTCALL
 calling convention for all callable objects
In-Reply-To: <1484310274.99.0.832491900546.issue29259@psf.upfronthosting.co.za>
Message-ID: <1484582030.41.0.271943634328.issue29259@psf.upfronthosting.co.za>


STINNER Victor added the comment:

> Strange, I'm unable to reproduce the result on a different computer: (...)

Ah, the benchmark result change after a reboot. I reran the same benchmark (LTO+PGO on speed-python) after a reboot:
---
haypo at speed-python$ python3 -m perf compare_to ~/test/2017-01-13_19-18-default-3486425ed82b.json ~/patchdir/patch.json  -G 
Slower (29):
- spectral_norm: 275 ms +- 4 ms -> 301 ms +- 16 ms: 1.09x slower (+9%)
- fannkuch: 1.10 sec +- 0.01 sec -> 1.16 sec +- 0.18 sec: 1.06x slower (+6%)
- scimark_monte_carlo: 282 ms +- 3 ms -> 298 ms +- 6 ms: 1.06x slower (+6%)
- sqlite_synth: 9.39 us +- 0.19 us -> 9.82 us +- 0.15 us: 1.05x slower (+5%)
- nqueens: 261 ms +- 2 ms -> 271 ms +- 2 ms: 1.04x slower (+4%)
- meteor_contest: 209 ms +- 1 ms -> 217 ms +- 2 ms: 1.04x slower (+4%)
- scimark_fft: 684 ms +- 22 ms -> 709 ms +- 9 ms: 1.04x slower (+4%)
- pickle_pure_python: 1.35 ms +- 0.01 ms -> 1.39 ms +- 0.02 ms: 1.03x slower (+3%)
- xml_etree_iterparse: 227 ms +- 5 ms -> 233 ms +- 6 ms: 1.03x slower (+3%)
- sympy_integrate: 49.3 ms +- 0.3 ms -> 50.5 ms +- 0.3 ms: 1.02x slower (+2%)
- genshi_text: 90.3 ms +- 1.0 ms -> 92.4 ms +- 1.1 ms: 1.02x slower (+2%)
- regex_dna: 283 ms +- 1 ms -> 289 ms +- 1 ms: 1.02x slower (+2%)
- telco: 22.3 ms +- 0.5 ms -> 22.6 ms +- 0.3 ms: 1.02x slower (+2%)
- chaos: 295 ms +- 3 ms -> 300 ms +- 3 ms: 1.02x slower (+2%)
- go: 598 ms +- 5 ms -> 607 ms +- 7 ms: 1.01x slower (+1%)
- json_dumps: 30.5 ms +- 0.4 ms -> 30.9 ms +- 0.5 ms: 1.01x slower (+1%)
- richards: 183 ms +- 4 ms -> 186 ms +- 5 ms: 1.01x slower (+1%)
- deltablue: 17.1 ms +- 0.5 ms -> 17.4 ms +- 0.3 ms: 1.01x slower (+1%)
- logging_format: 42.1 us +- 0.6 us -> 42.5 us +- 0.6 us: 1.01x slower (+1%)
- sympy_str: 511 ms +- 4 ms -> 515 ms +- 5 ms: 1.01x slower (+1%)
- 2to3: 760 ms +- 4 ms -> 766 ms +- 4 ms: 1.01x slower (+1%)
- regex_compile: 462 ms +- 4 ms -> 465 ms +- 5 ms: 1.01x slower (+1%)
- django_template: 447 ms +- 5 ms -> 450 ms +- 4 ms: 1.01x slower (+1%)
- logging_simple: 35.6 us +- 0.5 us -> 35.8 us +- 0.6 us: 1.01x slower (+1%)
- hexiom: 23.0 ms +- 0.2 ms -> 23.2 ms +- 0.1 ms: 1.01x slower (+1%)
- python_startup: 16.0 ms +- 0.1 ms -> 16.1 ms +- 0.1 ms: 1.00x slower (+0%)
- regex_effbot: 5.08 ms +- 0.03 ms -> 5.10 ms +- 0.09 ms: 1.00x slower (+0%)
- call_method: 13.8 ms +- 0.1 ms -> 13.9 ms +- 0.2 ms: 1.00x slower (+0%)
- python_startup_no_site: 9.36 ms +- 0.04 ms -> 9.37 ms +- 0.07 ms: 1.00x slower (+0%)

Faster (19):
- scimark_lu: 518 ms +- 18 ms -> 458 ms +- 25 ms: 1.13x faster (-11%)
- xml_etree_process: 242 ms +- 4 ms -> 234 ms +- 4 ms: 1.03x faster (-3%)
- unpickle: 36.0 us +- 0.5 us -> 35.0 us +- 0.5 us: 1.03x faster (-3%)
- chameleon: 30.6 ms +- 0.2 ms -> 29.8 ms +- 0.3 ms: 1.03x faster (-3%)
- nbody: 240 ms +- 2 ms -> 234 ms +- 2 ms: 1.02x faster (-2%)
- call_method_slots: 13.9 ms +- 0.2 ms -> 13.6 ms +- 0.4 ms: 1.02x faster (-2%)
- unpickle_pure_python: 924 us +- 8 us -> 904 us +- 9 us: 1.02x faster (-2%)
- xml_etree_generate: 289 ms +- 4 ms -> 283 ms +- 4 ms: 1.02x faster (-2%)
- mako: 43.2 ms +- 0.4 ms -> 42.3 ms +- 0.4 ms: 1.02x faster (-2%)
- pickle_dict: 65.5 us +- 4.1 us -> 64.4 us +- 0.9 us: 1.02x faster (-2%)
- call_simple: 12.8 ms +- 0.4 ms -> 12.6 ms +- 0.4 ms: 1.01x faster (-1%)
- xml_etree_parse: 294 ms +- 13 ms -> 291 ms +- 9 ms: 1.01x faster (-1%)
- pickle_list: 9.38 us +- 0.35 us -> 9.32 us +- 0.14 us: 1.01x faster (-1%)
- unpack_sequence: 118 ns +- 3 ns -> 118 ns +- 1 ns: 1.00x faster (-0%)
- sympy_expand: 1.14 sec +- 0.01 sec -> 1.13 sec +- 0.01 sec: 1.00x faster (-0%)
- sqlalchemy_declarative: 338 ms +- 3 ms -> 337 ms +- 3 ms: 1.00x faster (-0%)
- crypto_pyaes: 236 ms +- 2 ms -> 235 ms +- 1 ms: 1.00x faster (-0%)
- pidigits: 321 ms +- 1 ms -> 320 ms +- 1 ms: 1.00x faster (-0%)
- regex_v8: 45.6 ms +- 0.7 ms -> 45.6 ms +- 0.3 ms: 1.00x faster (-0%)

Benchmark hidden because not significant (16): call_method_unknown, dulwich_log, float, genshi_xml, html5lib, json_loads, logging_silent, pathlib, pickle, raytrace, scimark_sor, scimark_sparse_mat_mult, sqlalchemy_imperative, sympy_sum, tornado_http, unpickle_list
---

Lighter verbose with a threshold of 5%:
---
haypo at speed-python$ python3 -m perf compare_to ~/test/2017-01-13_19-18-default-3486425ed82b.json ~/patchdir/patch.json  -G --min-speed=5
Slower (3):
- spectral_norm: 275 ms +- 4 ms -> 301 ms +- 16 ms: 1.09x slower (+9%)
- fannkuch: 1.10 sec +- 0.01 sec -> 1.16 sec +- 0.18 sec: 1.06x slower (+6%)
- scimark_monte_carlo: 282 ms +- 3 ms -> 298 ms +- 6 ms: 1.06x slower (+6%)

Faster (1):
- scimark_lu: 518 ms +- 18 ms -> 458 ms +- 25 ms: 1.13x faster (-11%)

Benchmark hidden because not significant (60): 2to3, call_method, ...
---

spectral_norm and scimark_lu are two unstable benchmarks. Sadly, it seems like the current patch is not really significant on benchmarks. But again, it doesn't optimize __call__() slot yet.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29259>
_______________________________________

From report at bugs.python.org  Mon Jan 16 11:03:05 2017
From: report at bugs.python.org (Michael Foord)
Date: Mon, 16 Jan 2017 16:03:05 +0000
Subject: [issue28911] Clarify the behaviour of assert_called_once_with
In-Reply-To: <1481233700.68.0.637550951007.issue28911@psf.upfronthosting.co.za>
Message-ID: <1484582585.14.0.435969902795.issue28911@psf.upfronthosting.co.za>


Michael Foord added the comment:

I like the documentation improvement, thank you.

(The purpose of the method is to combine an assert about the arguments the method was called with and an assertion that it was only called once. To change the semantics would be both undesirable and backwards incompatible.)

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28911>
_______________________________________

From report at bugs.python.org  Mon Jan 16 11:12:53 2017
From: report at bugs.python.org (INADA Naoki)
Date: Mon, 16 Jan 2017 16:12:53 +0000
Subject: [issue29259] Add tp_fastcall to PyTypeObject: support FASTCALL
 calling convention for all callable objects
In-Reply-To: <1484310274.99.0.832491900546.issue29259@psf.upfronthosting.co.za>
Message-ID: <1484583173.28.0.068288046878.issue29259@psf.upfronthosting.co.za>


INADA Naoki added the comment:

Additionally, there are not enough METH_FASTCALL CFunctions.

I'm interested in performance difference between METH_VARARGS and METH_FASTCALL.  If METH_FASTCALL is significant faster than VARARGS,
we can convert some common methods from VARARGS to FASTCALL, by hand or
by AC.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29259>
_______________________________________

From report at bugs.python.org  Mon Jan 16 11:32:18 2017
From: report at bugs.python.org (Roundup Robot)
Date: Mon, 16 Jan 2017 16:32:18 +0000
Subject: [issue29259] Add tp_fastcall to PyTypeObject: support FASTCALL
 calling convention for all callable objects
In-Reply-To: <1484310274.99.0.832491900546.issue29259@psf.upfronthosting.co.za>
Message-ID: <20170116163215.112105.23070.CCA758D6@psf.io>


Roundup Robot added the comment:

New changeset 2ccdf2b3819d by Victor Stinner in branch 'default':
Optimize _PyCFunction_FastCallKeywords()
https://hg.python.org/cpython/rev/2ccdf2b3819d

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29259>
_______________________________________

From report at bugs.python.org  Mon Jan 16 11:42:44 2017
From: report at bugs.python.org (John Taylor)
Date: Mon, 16 Jan 2017 16:42:44 +0000
Subject: [issue29284] Include thread_name_prefix in the
 concurrent.futures.ThreadPoolExecutor example 17.4.2.1
Message-ID: <1484584964.59.0.239435362381.issue29284@psf.upfronthosting.co.za>


New submission from John Taylor:

Please include how to use the thread_name_prefix method argument (new to Python 3.6) in the example: 17.4.2.1. ThreadPoolExecutor Example

----------
assignee: docs at python
components: Documentation
messages: 285572
nosy: docs at python, jftuga
priority: normal
severity: normal
status: open
title: Include thread_name_prefix  in the concurrent.futures.ThreadPoolExecutor example 17.4.2.1
type: enhancement
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29284>
_______________________________________

From report at bugs.python.org  Mon Jan 16 12:06:40 2017
From: report at bugs.python.org (John Taylor)
Date: Mon, 16 Jan 2017 17:06:40 +0000
Subject: [issue29285] Unicode errors occur inside of multi-line comments
Message-ID: <1484586400.61.0.654989234385.issue29285@psf.upfronthosting.co.za>


New submission from John Taylor:

I am using Python 3.5.2 on OS X 10.11.6.  The same error occurs with 3.5.2 on Windows 10.

When I run the attached code, I get this error:

SyntaxError: (unicode error) 'unicodeescape' codec can't decode bytes in position 23-24: truncated \uXXXX escape

I think unicode processing should be ignored inside of multiline comments.
Inside of my comments, I really want to use this text verbatim.

----------
components: Interpreter Core
files: example.py
messages: 285573
nosy: jftuga
priority: normal
severity: normal
status: open
title: Unicode errors occur inside of multi-line comments
type: behavior
versions: Python 3.5
Added file: http://bugs.python.org/file46303/example.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29285>
_______________________________________

From report at bugs.python.org  Mon Jan 16 12:30:51 2017
From: report at bugs.python.org (STINNER Victor)
Date: Mon, 16 Jan 2017 17:30:51 +0000
Subject: [issue29286] Use METH_FASTCALL in str methods
Message-ID: <1484587851.78.0.529922329593.issue29286@psf.upfronthosting.co.za>


New submission from STINNER Victor:

Changes 27dc9a1c061e and 01b06ca45f64 converted the str (Unicode) methods to Argument Clinic, cool! But str methods taking more than one argument use positional-only arguments.

Currently, Argument Clinic doesn't use METH_FASTCALL for these methods, but METH_VARARGS.

There are two options:

* Allow passing arguments as keywoards: str.replace(old='a', new='b')
* Enhance Argument Clinic to use also METH_FASTCALL for functions using positional-only functions

The goal is to speedup method calls. Example with str.replace():

$ ./python-patch -m perf timeit '"a".replace("x", "y")' --duplicate=100 --compare-to ./python-ref
python-ref: ..................... 132 ns +- 1 ns
python-patch: ..................... 102 ns +- 2 ns

Median +- std dev: [python-ref] 132 ns +- 1 ns -> [python-patch] 102 ns +- 2 ns: 1.30x faster (-23%)

----------
messages: 285574
nosy: haypo, inada.naoki, larry, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Use METH_FASTCALL in str methods
type: performance
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29286>
_______________________________________

From report at bugs.python.org  Mon Jan 16 12:31:49 2017
From: report at bugs.python.org (STINNER Victor)
Date: Mon, 16 Jan 2017 17:31:49 +0000
Subject: [issue29286] Use METH_FASTCALL in str methods
In-Reply-To: <1484587851.78.0.529922329593.issue29286@psf.upfronthosting.co.za>
Message-ID: <1484587909.17.0.8522521055.issue29286@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Background: see also the old issue #17170 closed as rejected: "string method lookup is too slow".

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29286>
_______________________________________

From report at bugs.python.org  Mon Jan 16 12:32:52 2017
From: report at bugs.python.org (STINNER Victor)
Date: Mon, 16 Jan 2017 17:32:52 +0000
Subject: [issue29286] Use METH_FASTCALL in str methods
In-Reply-To: <1484587851.78.0.529922329593.issue29286@psf.upfronthosting.co.za>
Message-ID: <1484587972.86.0.40247822461.issue29286@psf.upfronthosting.co.za>


STINNER Victor added the comment:

The issue #29263 "Implement LOAD_METHOD/CALL_METHOD for C functions" should also optimize C methods even more.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29286>
_______________________________________

From report at bugs.python.org  Mon Jan 16 12:33:57 2017
From: report at bugs.python.org (Zachary Ware)
Date: Mon, 16 Jan 2017 17:33:57 +0000
Subject: [issue29285] Unicode errors occur inside of multi-line comments
In-Reply-To: <1484586400.61.0.654989234385.issue29285@psf.upfronthosting.co.za>
Message-ID: <1484588037.36.0.674865405819.issue29285@psf.upfronthosting.co.za>


Zachary Ware added the comment:

Triple-quoted strings are strings, not multi-line comments, though people do abuse them for that purpose sometimes.  Changing this behavior would be a huge change for next to no benefit, so I'm closing the issue.

If you insist on using a triple-quoted string as a comment (which I repeat, it is not; comments are ignored by the compiler, while triple-quoted strings are treated just as any other string literal by the compiler), you can work around this by using a 'raw' string literal:

r"""
dir \\dept.example.com\user
"""

----------
nosy: +zach.ware
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29285>
_______________________________________

From report at bugs.python.org  Mon Jan 16 12:54:46 2017
From: report at bugs.python.org (Yury Selivanov)
Date: Mon, 16 Jan 2017 17:54:46 +0000
Subject: [issue29286] Use METH_FASTCALL in str methods
In-Reply-To: <1484587851.78.0.529922329593.issue29286@psf.upfronthosting.co.za>
Message-ID: <1484589286.52.0.24902541007.issue29286@psf.upfronthosting.co.za>


Yury Selivanov added the comment:

> * Allow passing arguments as keywoards: str.replace(old='a', new='b')

I think Guido explicitly stated that he doesn't like the idea to always allow keyword arguments for all methods. I.e. `str.find('aaa')` just reads better than `str.find(needle='aaa')`. Essentially, the idea is that for most of the builtins that accept one or two arguments, positional-only parameters are better.

> * Enhance Argument Clinic to use also METH_FASTCALL for functions using positional-only functions

So this is the option to go with.

----------
nosy: +yselivanov

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29286>
_______________________________________

From report at bugs.python.org  Mon Jan 16 13:26:08 2017
From: report at bugs.python.org (Daniel Fackrell)
Date: Mon, 16 Jan 2017 18:26:08 +0000
Subject: [issue29230] Segfault in sqlite3
In-Reply-To: <1484077511.04.0.514172163049.issue29230@psf.upfronthosting.co.za>
Message-ID: <1484591168.62.0.754726392402.issue29230@psf.upfronthosting.co.za>


Daniel Fackrell added the comment:

I was using versions of python from pyenv, but they may have been using the system version of sqlite3. I recreated the virtualenv using python 2.7.13 installed with Brew, and it doesn't crash.

sqlite3.sqlite_version went from 3.8.10.2 to 3.16.2, which seems like a rather large jump.

Thank you for your assistance on this! I'm still trying to find a good way to get pyenv to recognize and use the newer library, but I can handle that outside of this ticket unless you have some quick tips already handy for that.

----------
status: pending -> open

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29230>
_______________________________________

From report at bugs.python.org  Mon Jan 16 13:38:15 2017
From: report at bugs.python.org (Ned Deily)
Date: Mon, 16 Jan 2017 18:38:15 +0000
Subject: [issue29230] Segfault in sqlite3
In-Reply-To: <1484077511.04.0.514172163049.issue29230@psf.upfronthosting.co.za>
Message-ID: <1484591895.56.0.88800002756.issue29230@psf.upfronthosting.co.za>


Ned Deily added the comment:

> I'm still trying to find a good way to get pyenv to recognize and use the newer library

Thanks for confirming that the problem was due to the version of sqlite3 in use.  If you haven't already, suggest you open an issue on the pyenv project tracker or ask elsewhere.  Good luck!

https://github.com/yyuu/pyenv

----------
stage:  -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29230>
_______________________________________

From report at bugs.python.org  Mon Jan 16 13:52:37 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Mon, 16 Jan 2017 18:52:37 +0000
Subject: [issue29286] Use METH_FASTCALL in str methods
In-Reply-To: <1484587851.78.0.529922329593.issue29286@psf.upfronthosting.co.za>
Message-ID: <1484592757.17.0.248392311646.issue29286@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

What is python-patch?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29286>
_______________________________________

From report at bugs.python.org  Mon Jan 16 14:04:14 2017
From: report at bugs.python.org (Mark Dickinson)
Date: Mon, 16 Jan 2017 19:04:14 +0000
Subject: [issue29282] Fused multiply-add: proposal to add math.fma()
In-Reply-To: <1484560897.19.0.0955404368118.issue29282@psf.upfronthosting.co.za>
Message-ID: <1484593454.17.0.385832984198.issue29282@psf.upfronthosting.co.za>


Mark Dickinson added the comment:

Here's a pure Python reference implementation, with tests.

----------
Added file: http://bugs.python.org/file46304/fma_reference.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29282>
_______________________________________

From report at bugs.python.org  Mon Jan 16 14:42:34 2017
From: report at bugs.python.org (Martin Panter)
Date: Mon, 16 Jan 2017 19:42:34 +0000
Subject: [issue27659] Prohibit implicit C function declarations
In-Reply-To: <1469948678.43.0.816389049938.issue27659@psf.upfronthosting.co.za>
Message-ID: <1484595754.14.0.0669606847795.issue27659@psf.upfronthosting.co.za>


Martin Panter added the comment:

I would say it is more important to fit in with the surrounding style than mindlessly follow PEP 7. IMO the indentation in the configure script is a mess, but if we fix it up, it should probably be done separately to adding this extra flag.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue27659>
_______________________________________

From report at bugs.python.org  Mon Jan 16 15:00:59 2017
From: report at bugs.python.org (STINNER Victor)
Date: Mon, 16 Jan 2017 20:00:59 +0000
Subject: [issue29286] Use METH_FASTCALL in str methods
In-Reply-To: <1484587851.78.0.529922329593.issue29286@psf.upfronthosting.co.za>
Message-ID: <1484596859.69.0.220779389785.issue29286@psf.upfronthosting.co.za>


STINNER Victor added the comment:

> What is python-patch?

It's Python patched with attached unicode_allow_kw.patch: allow to pass parameters as keywords for Unicode methods.

----------
keywords: +patch
Added file: http://bugs.python.org/file46305/unicode_allow_kw.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29286>
_______________________________________

From report at bugs.python.org  Mon Jan 16 17:20:08 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Mon, 16 Jan 2017 22:20:08 +0000
Subject: [issue29287] IDLE needs syntax highlighting for f-strings
Message-ID: <1484605208.08.0.296182628472.issue29287@psf.upfronthosting.co.za>


New submission from Raymond Hettinger:

Follow the lead from Vim, MagicPython, and PyCharm.  Needs separate colorization to make the expression distinct from the rest of the string.  Needs close-brace matching.  Would be desirable to have autocompletion as well.  

https://twitter.com/raymondh/status/819085176764366848

----------
assignee: terry.reedy
components: IDLE
messages: 285585
nosy: rhettinger, terry.reedy
priority: normal
severity: normal
stage: needs patch
status: open
title: IDLE needs syntax highlighting for f-strings
type: enhancement
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29287>
_______________________________________

From report at bugs.python.org  Mon Jan 16 17:30:45 2017
From: report at bugs.python.org (Ilya Kulakov)
Date: Mon, 16 Jan 2017 22:30:45 +0000
Subject: [issue29288] Lookup Error while importing idna from a worker thread
Message-ID: <1484605845.11.0.283710954459.issue29288@psf.upfronthosting.co.za>


New submission from Ilya Kulakov:

See this post: https://github.com/kennethreitz/requests/issues/3578

The current workaround for requests is to have a no-op import somewhere in the code.

However, that doesn't really work for us: our python and stdlib are bundled by pyqtdeploy as in Qt resources system and we frequently see this error. On certain machines. Workaround that seems to work more reliably is a no-op look up via ''.encode('ascii').decode('idna').

----------
components: Library (Lib)
messages: 285586
nosy: Ilya.Kulakov
priority: normal
severity: normal
status: open
title: Lookup Error while importing idna from a worker thread
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29288>
_______________________________________

From report at bugs.python.org  Mon Jan 16 18:24:01 2017
From: report at bugs.python.org (INADA Naoki)
Date: Mon, 16 Jan 2017 23:24:01 +0000
Subject: [issue29286] Use METH_FASTCALL in str methods
In-Reply-To: <1484587851.78.0.529922329593.issue29286@psf.upfronthosting.co.za>
Message-ID: <1484609041.44.0.0510744510658.issue29286@psf.upfronthosting.co.za>


INADA Naoki added the comment:

Parsing positional arguments for METH_KEYWORDS and METH_FASTCALL can be faster by 
http://bugs.python.org/issue29029

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29286>
_______________________________________

From report at bugs.python.org  Mon Jan 16 18:24:44 2017
From: report at bugs.python.org (STINNER Victor)
Date: Mon, 16 Jan 2017 23:24:44 +0000
Subject: [issue29289] Convert OrderedDict methods to Argument Clinic
Message-ID: <1484609084.15.0.406928886689.issue29289@psf.upfronthosting.co.za>


New submission from STINNER Victor:

Attached patch converts methods using METH_VARARGS|METH_KEYWORDS calling convention to METH_FASTCALL using Argument Clinic. This calling convention is faster, and Argument Clinic provides better docstring.

See also issue #29263 "Implement LOAD_METHOD/CALL_METHOD for C functions" and issue #29286 "Use METH_FASTCALL in str methods".

----------
components: Interpreter Core
files: odict_clinic.patch
keywords: patch
messages: 285588
nosy: eric.snow, haypo, inada.naoki
priority: normal
severity: normal
status: open
title: Convert OrderedDict methods to Argument Clinic
type: performance
versions: Python 3.7
Added file: http://bugs.python.org/file46306/odict_clinic.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29289>
_______________________________________

From report at bugs.python.org  Mon Jan 16 18:26:08 2017
From: report at bugs.python.org (STINNER Victor)
Date: Mon, 16 Jan 2017 23:26:08 +0000
Subject: [issue29289] Convert OrderedDict methods to Argument Clinic
In-Reply-To: <1484609084.15.0.406928886689.issue29289@psf.upfronthosting.co.za>
Message-ID: <1484609168.07.0.82712471017.issue29289@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Converted methods:

* fromkeys() (@classmethod)
* setdefault()
* pop()
* popitem()
* move_to_end()

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29289>
_______________________________________

From report at bugs.python.org  Mon Jan 16 18:39:01 2017
From: report at bugs.python.org (Roundup Robot)
Date: Mon, 16 Jan 2017 23:39:01 +0000
Subject: [issue29029] Faster positional arguments parsing in
 PyArg_ParseTupleAndKeywords
In-Reply-To: <1482269418.38.0.550958237488.issue29029@psf.upfronthosting.co.za>
Message-ID: <20170116233857.56526.62660.C6DE3D9D@psf.io>


Roundup Robot added the comment:

New changeset a36e60c5fee0 by Victor Stinner in branch 'default':
Rename keywords to kwargs in getargs.c
https://hg.python.org/cpython/rev/a36e60c5fee0

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29029>
_______________________________________

From report at bugs.python.org  Mon Jan 16 18:39:56 2017
From: report at bugs.python.org (STINNER Victor)
Date: Mon, 16 Jan 2017 23:39:56 +0000
Subject: [issue29029] Faster positional arguments parsing in
 PyArg_ParseTupleAndKeywords
In-Reply-To: <1482269418.38.0.550958237488.issue29029@psf.upfronthosting.co.za>
Message-ID: <1484609996.0.0.840631467566.issue29029@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Oh, I missed this issue. Since I made minor cleanup recently, I took the liberty of rebasing your patch.

I also pushed your "keywords => kwargs" change, just to make the patch simpler to review.

----------
nosy: +haypo
Added file: http://bugs.python.org/file46307/PyArg_ParseTupleAndKeywords-faster-positional-args-parse-2.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29029>
_______________________________________

From report at bugs.python.org  Mon Jan 16 18:49:18 2017
From: report at bugs.python.org (STINNER Victor)
Date: Mon, 16 Jan 2017 23:49:18 +0000
Subject: [issue29029] Faster positional arguments parsing in
 PyArg_ParseTupleAndKeywords
In-Reply-To: <1482269418.38.0.550958237488.issue29029@psf.upfronthosting.co.za>
Message-ID: <1484610558.54.0.865315980061.issue29029@psf.upfronthosting.co.za>


STINNER Victor added the comment:

> Unpatched:  Median +- std dev: 2.01 us +- 0.09 us
> Patched:    Median +- std dev: 1.23 us +- 0.03 us

Oh wow, impressive speedup! As usual, good job Serhiy ;-)

PyArg_ParseTupleAndKeywords-faster-positional-args-parse-2.patch: LGTM! Can you please push it?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29029>
_______________________________________

From report at bugs.python.org  Mon Jan 16 19:01:34 2017
From: report at bugs.python.org (Robert Collins)
Date: Tue, 17 Jan 2017 00:01:34 +0000
Subject: [issue9216] FIPS support for hashlib
In-Reply-To: <1278721335.16.0.522410247151.issue9216@psf.upfronthosting.co.za>
Message-ID: <1484611294.43.0.117287266512.issue9216@psf.upfronthosting.co.za>


Robert Collins added the comment:

A few thoughts;

usedforsecurity=xxx seems awkward: I wouldn't want, as a user of hashlib, to have to put that in literally every use I make of it.

If I understand the situation correctly, the goal is for both linters, and at runtime, identification of the intended purpose of a call to md5 - e.g. whether there are security implications in its use (as far as FIPS is concerned).

Perhaps having two separate implementations of the interfaces, one general purpose and one FIPS would be decent.

e.g. from hashlib.fips import sha1 
etc
etc
and hashlib.fips simply wouldn't contain md5.

Then the md5 thats in hashlib is by definition not FIPS ready and any code using it should be fixed.

----------
nosy: +rbcollins

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue9216>
_______________________________________

From report at bugs.python.org  Mon Jan 16 19:09:26 2017
From: report at bugs.python.org (INADA Naoki)
Date: Tue, 17 Jan 2017 00:09:26 +0000
Subject: [issue29286] Use METH_FASTCALL in str methods
In-Reply-To: <1484587851.78.0.529922329593.issue29286@psf.upfronthosting.co.za>
Message-ID: <1484611766.73.0.615776129838.issue29286@psf.upfronthosting.co.za>


INADA Naoki added the comment:

This patch makes AC produces more FASTCALL instead of VARARGS.

When looking AC output, one downside is it produces many consts like:
+    static const char * const _keywords[] = {"", "", "", "", "", NULL};

----------
Added file: http://bugs.python.org/file46308/ac_more_fastcalls.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29286>
_______________________________________

From report at bugs.python.org  Mon Jan 16 20:10:05 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 17 Jan 2017 01:10:05 +0000
Subject: [issue29286] Use METH_FASTCALL in str methods
In-Reply-To: <1484587851.78.0.529922329593.issue29286@psf.upfronthosting.co.za>
Message-ID: <1484615405.29.0.220284980222.issue29286@psf.upfronthosting.co.za>


STINNER Victor added the comment:

ac_more_fastcalls.patch: Hum, I guess that your change on _PyStack_UnpackDict() is related to a bug in the function prototype. The function is unable to report failure if args is NULL. It changed the API in the change 38ab8572fde2.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29286>
_______________________________________

From report at bugs.python.org  Mon Jan 16 20:17:09 2017
From: report at bugs.python.org (Roundup Robot)
Date: Tue, 17 Jan 2017 01:17:09 +0000
Subject: [issue29286] Use METH_FASTCALL in str methods
In-Reply-To: <1484587851.78.0.529922329593.issue29286@psf.upfronthosting.co.za>
Message-ID: <20170117010733.13926.48407.864D7F20@psf.io>


Roundup Robot added the comment:

New changeset d07fd6e6d449 by Victor Stinner in branch 'default':
Rename _PyArg_ParseStack to _PyArg_ParseStackAndKeywords
https://hg.python.org/cpython/rev/d07fd6e6d449

New changeset 01c57ef1b651 by Victor Stinner in branch 'default':
Add _PyArg_ParseStack() helper function
https://hg.python.org/cpython/rev/01c57ef1b651

New changeset 8bfec37ea86a by Victor Stinner in branch 'default':
Add _PyArg_NoStackKeywords() helper function
https://hg.python.org/cpython/rev/8bfec37ea86a

New changeset 38ab8572fde2 by Victor Stinner in branch 'default':
_PyStack_UnpackDict() now returns -1 on error
https://hg.python.org/cpython/rev/38ab8572fde2

New changeset de90f3d06bc9 by Victor Stinner in branch 'default':
Argument Clinic: Use METH_FASTCALL for positionals
https://hg.python.org/cpython/rev/de90f3d06bc9

New changeset dea8922751a1 by Victor Stinner in branch 'default':
Run Argument Clinic: METH_VARARGS=>METH_FASTCALL
https://hg.python.org/cpython/rev/dea8922751a1

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29286>
_______________________________________

From report at bugs.python.org  Mon Jan 16 20:37:53 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 17 Jan 2017 01:37:53 +0000
Subject: [issue29286] Use METH_FASTCALL in str methods
In-Reply-To: <1484587851.78.0.529922329593.issue29286@psf.upfronthosting.co.za>
Message-ID: <1484617073.77.0.226350412989.issue29286@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Naoki> This patch makes AC produces more FASTCALL instead of VARARGS.

Oh, funny, I wrote the same patch :-) (almost)


> When looking AC output, one downside is it produces many consts like:
> +    static const char * const _keywords[] = {"", "", "", "", "", NULL};

Yeah, I got the same result. I tried to hack a _PyArg_ParseStack() function which doesn't take keyword argments (no kwnames), but I lost my mind in parser_init().

So I decided to simply rebase my old patches from my random CPython fastcall forks to add a simple _PyArg_ParseStack(). My implementation doesn't use the super-fast _PyArg_Parser object, but at least it allows to use METH_FASTCALL. Compared to what we had previously (METH_VARARGS), it is an enhancement :-)

I prefer to move step by step, since each commit is already big enough. It's also easier for me to maintain my forks if I push more changes upstream.

So there is still room for improvement in _PyArg_ParseStack(), but I suggest to defer further optimizations to a new issue.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29286>
_______________________________________

From report at bugs.python.org  Mon Jan 16 20:38:17 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Tue, 17 Jan 2017 01:38:17 +0000
Subject: [issue29289] Convert OrderedDict methods to Argument Clinic
In-Reply-To: <1484609084.15.0.406928886689.issue29289@psf.upfronthosting.co.za>
Message-ID: <1484617097.8.0.878443596506.issue29289@psf.upfronthosting.co.za>


Raymond Hettinger added the comment:

This patch segfaults for me (Mac OS 10.12.2 with the default clang-800.0.42.1).

For now, pop() should be omitted.  The argument clinic doesn't cope well with optional arguments without a default value.  The generated help "pop(self, /, key, default=None)" doesn't really make since because the default is not None, it is a sentinel value.  The generated help incorrectly makes it look like OrderedDict.get() which actually does have a default=None.

----------
nosy: +rhettinger

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29289>
_______________________________________

From report at bugs.python.org  Mon Jan 16 20:48:49 2017
From: report at bugs.python.org (Roundup Robot)
Date: Tue, 17 Jan 2017 01:48:49 +0000
Subject: [issue29286] Use METH_FASTCALL in str methods
In-Reply-To: <1484587851.78.0.529922329593.issue29286@psf.upfronthosting.co.za>
Message-ID: <20170117014845.56179.3322.B7FC44AD@psf.io>


Roundup Robot added the comment:

New changeset 3bf78c286daf by Victor Stinner in branch 'default':
Add _PyArg_UnpackStack() function helper
https://hg.python.org/cpython/rev/3bf78c286daf

New changeset 905e902bd47e by Victor Stinner in branch 'default':
Argument Clinic: Use METH_FASTCALL for boring positionals
https://hg.python.org/cpython/rev/905e902bd47e

New changeset 52acda52b353 by Victor Stinner in branch 'default':
Run Argument Clinic: METH_VARARGS=>METH_FASTCALL
https://hg.python.org/cpython/rev/52acda52b353

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29286>
_______________________________________

From report at bugs.python.org  Mon Jan 16 21:25:18 2017
From: report at bugs.python.org (Roundup Robot)
Date: Tue, 17 Jan 2017 02:25:18 +0000
Subject: [issue29286] Use METH_FASTCALL in str methods
In-Reply-To: <1484587851.78.0.529922329593.issue29286@psf.upfronthosting.co.za>
Message-ID: <20170117010733.13926.48407.864D7F20@psf.io>


Roundup Robot added the comment:

New changeset d07fd6e6d449 by Victor Stinner in branch 'default':
Rename _PyArg_ParseStack to _PyArg_ParseStackAndKeywords
https://hg.python.org/cpython/rev/d07fd6e6d449

New changeset 01c57ef1b651 by Victor Stinner in branch 'default':
Add _PyArg_ParseStack() helper function
https://hg.python.org/cpython/rev/01c57ef1b651

New changeset 8bfec37ea86a by Victor Stinner in branch 'default':
Add _PyArg_NoStackKeywords() helper function
https://hg.python.org/cpython/rev/8bfec37ea86a

New changeset 38ab8572fde2 by Victor Stinner in branch 'default':
_PyStack_UnpackDict() now returns -1 on error
https://hg.python.org/cpython/rev/38ab8572fde2

New changeset de90f3d06bc9 by Victor Stinner in branch 'default':
Argument Clinic: Use METH_FASTCALL for positionals
https://hg.python.org/cpython/rev/de90f3d06bc9

New changeset dea8922751a1 by Victor Stinner in branch 'default':
Run Argument Clinic: METH_VARARGS=>METH_FASTCALL
https://hg.python.org/cpython/rev/dea8922751a1

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29286>
_______________________________________

From report at bugs.python.org  Mon Jan 16 21:26:34 2017
From: report at bugs.python.org (INADA Naoki)
Date: Tue, 17 Jan 2017 02:26:34 +0000
Subject: [issue29289] Convert OrderedDict methods to Argument Clinic
In-Reply-To: <1484609084.15.0.406928886689.issue29289@psf.upfronthosting.co.za>
Message-ID: <1484619994.25.0.576188783384.issue29289@psf.upfronthosting.co.za>


INADA Naoki added the comment:

LGTM, except pop().

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29289>
_______________________________________

From report at bugs.python.org  Mon Jan 16 21:37:50 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 17 Jan 2017 02:37:50 +0000
Subject: [issue29259] Add tp_fastcall to PyTypeObject: support FASTCALL
 calling convention for all callable objects
In-Reply-To: <1484310274.99.0.832491900546.issue29259@psf.upfronthosting.co.za>
Message-ID: <1484620670.61.0.66627383851.issue29259@psf.upfronthosting.co.za>


STINNER Victor added the comment:

> Additionally, there are not enough METH_FASTCALL CFunctions.

I pushed many changes tonight which convert many C functions to METH_FASTCALL.


> ..., by hand or by AC.

IMHO performance is a good motivation to convert code to Argument Clinic ;-) But sometimes, AC can be painful, so by hand is also acceptable ;-)

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29259>
_______________________________________

From report at bugs.python.org  Mon Jan 16 21:42:06 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 17 Jan 2017 02:42:06 +0000
Subject: [issue29259] Add tp_fastcall to PyTypeObject: support FASTCALL
 calling convention for all callable objects
In-Reply-To: <1484310274.99.0.832491900546.issue29259@psf.upfronthosting.co.za>
Message-ID: <1484620926.14.0.0200840538719.issue29259@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Patch version 2:

* Add slot_tp_fastcall(): __call__() wrapper using the FASTCALL calling convention
* Add Py_TPFLAGS_HAVE_FASTCALL flag
* Remove completely the "cached args" optimization from property_descr_get()
* Rebase the patch serie on top of the default branch to benefit of recent changes related to FASTCALL

I'm not sure that the Py_TPFLAGS_HAVE_FASTCALL flag is used correctly. Maybe this new flag disabled optimizations in some cases, I didn't read carefully this part of the code.

----------
Added file: http://bugs.python.org/file46309/tp_fastcall-2.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29259>
_______________________________________

From report at bugs.python.org  Mon Jan 16 21:47:54 2017
From: report at bugs.python.org (Roundup Robot)
Date: Tue, 17 Jan 2017 02:47:54 +0000
Subject: [issue29289] Convert OrderedDict methods to Argument Clinic
In-Reply-To: <1484609084.15.0.406928886689.issue29289@psf.upfronthosting.co.za>
Message-ID: <20170117024750.7098.28884.54EF05C3@psf.io>


Roundup Robot added the comment:

New changeset 9f7d16266928 by Victor Stinner in branch 'default':
Convert some OrderedDict methods to Argument Clinic
https://hg.python.org/cpython/rev/9f7d16266928

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29289>
_______________________________________

From report at bugs.python.org  Mon Jan 16 21:48:40 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 17 Jan 2017 02:48:40 +0000
Subject: [issue29289] Convert OrderedDict methods to Argument Clinic
In-Reply-To: <1484609084.15.0.406928886689.issue29289@psf.upfronthosting.co.za>
Message-ID: <1484621320.66.0.652156076879.issue29289@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Raymond: " The argument clinic doesn't cope well with optional arguments without a default value."

Right, it's a bug in AC. I will try to fix it later. In the meanwhile, I pushed the first uncontroversal part.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29289>
_______________________________________

From report at bugs.python.org  Mon Jan 16 21:58:49 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 17 Jan 2017 02:58:49 +0000
Subject: [issue29289] Convert OrderedDict methods to Argument Clinic
In-Reply-To: <1484609084.15.0.406928886689.issue29289@psf.upfronthosting.co.za>
Message-ID: <1484621929.38.0.453673196889.issue29289@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Oh, I missed Naoki's comment on the review :-/ I got the notification of this review after I pushed the change. I will fix the docstring later.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29289>
_______________________________________

From report at bugs.python.org  Mon Jan 16 22:08:41 2017
From: report at bugs.python.org (Steven D'Aprano)
Date: Tue, 17 Jan 2017 03:08:41 +0000
Subject: [issue29290] argparse breaks long lines on NO-BREAK SPACE
Message-ID: <1484622521.42.0.0580919403747.issue29290@psf.upfronthosting.co.za>


New submission from Steven D'Aprano:

argparse help incorrectly breaks long lines on U+u00A0 NO-BREAK SPACE.

The attached script has been run on Python 3.5.3rc1 in a terminal window 80 columns wide, and it produces output::

    usage: argparse_nobreak.py [-h] [--no-condensedxxxx]

    optional arguments:
      -h, --help          show this help message and exit
      --no-condensedxxxx  Disable default font-style: condensed. Also disables "M+
                          1M" condensed monospace.


I expected the last line should have broken just before the "M+ 1M", rather than in the middle.

See also #20491.

----------
components: Library (Lib)
files: argparse_nobreak.py
messages: 285607
nosy: steven.daprano
priority: normal
severity: normal
status: open
title: argparse breaks long lines on NO-BREAK SPACE
type: behavior
versions: Python 3.5
Added file: http://bugs.python.org/file46310/argparse_nobreak.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29290>
_______________________________________

From report at bugs.python.org  Mon Jan 16 22:33:47 2017
From: report at bugs.python.org (Steven D'Aprano)
Date: Tue, 17 Jan 2017 03:33:47 +0000
Subject: [issue29290] argparse breaks long lines on NO-BREAK SPACE
In-Reply-To: <1484622521.42.0.0580919403747.issue29290@psf.upfronthosting.co.za>
Message-ID: <1484624027.86.0.065980222271.issue29290@psf.upfronthosting.co.za>


Steven D'Aprano added the comment:

Here's a slightly simpler demo, without the (fortunately harmless) typo.

----------
Added file: http://bugs.python.org/file46311/argparse_nobreak.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29290>
_______________________________________

From report at bugs.python.org  Mon Jan 16 22:34:22 2017
From: report at bugs.python.org (Steven D'Aprano)
Date: Tue, 17 Jan 2017 03:34:22 +0000
Subject: [issue29290] argparse breaks long lines on NO-BREAK SPACE
In-Reply-To: <1484622521.42.0.0580919403747.issue29290@psf.upfronthosting.co.za>
Message-ID: <1484624062.01.0.897654602341.issue29290@psf.upfronthosting.co.za>


Changes by Steven D'Aprano <steve+python at pearwood.info>:


Removed file: http://bugs.python.org/file46310/argparse_nobreak.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29290>
_______________________________________

From report at bugs.python.org  Mon Jan 16 22:39:55 2017
From: report at bugs.python.org (Martin Panter)
Date: Tue, 17 Jan 2017 03:39:55 +0000
Subject: [issue29290] argparse breaks long lines on NO-BREAK SPACE
In-Reply-To: <1484622521.42.0.0580919403747.issue29290@psf.upfronthosting.co.za>
Message-ID: <1484624395.91.0.897433670692.issue29290@psf.upfronthosting.co.za>


Martin Panter added the comment:

Maybe a duplicate of Issue 16623

----------
nosy: +martin.panter

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29290>
_______________________________________

From report at bugs.python.org  Mon Jan 16 23:09:31 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Tue, 17 Jan 2017 04:09:31 +0000
Subject: [issue29290] argparse breaks long lines on NO-BREAK SPACE
In-Reply-To: <1484622521.42.0.0580919403747.issue29290@psf.upfronthosting.co.za>
Message-ID: <1484626171.79.0.420509117446.issue29290@psf.upfronthosting.co.za>


Xiang Zhang added the comment:

textwrap has been fixed in #20491 but this problem still exists. The reason seems to be that argparse replaces the non-break spaces with spaces:

before self.whitespace_matcher.sub
'Disable default font-style: condensed.  Also disables "M+\\xa01M" condensed monospace.'
after self.whitespace_matcher.sub
'Disable default font-style: condensed. Also disables "M+ 1M" condensed monospace.'

----------
nosy: +xiang.zhang

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29290>
_______________________________________

From report at bugs.python.org  Tue Jan 17 01:06:54 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Tue, 17 Jan 2017 06:06:54 +0000
Subject: [issue29290] argparse breaks long lines on NO-BREAK SPACE
In-Reply-To: <1484622521.42.0.0580919403747.issue29290@psf.upfronthosting.co.za>
Message-ID: <1484633214.14.0.702903895696.issue29290@psf.upfronthosting.co.za>


Xiang Zhang added the comment:

I think this is a regression when coming to 3.x. In 2.7, r'\s+' is by default in ASCII mode and won't match unicode non-breaking spaces. In 3.x it's by default unicode mode so non-breaking spaces are replaced by spaces. I think we can just use [ \t\n\r\f\v]+.

Since here are more active core devs I am going to close #16623 and move this forward here.

----------
keywords: +patch
nosy: +paul.j3, roysmith, serhiy.storchaka
versions: +Python 3.6, Python 3.7
Added file: http://bugs.python.org/file46312/argparse-help-non-breaking-spaces.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29290>
_______________________________________

From report at bugs.python.org  Tue Jan 17 01:09:25 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Tue, 17 Jan 2017 06:09:25 +0000
Subject: [issue16623] argparse help formatter does not honor non-breaking space
In-Reply-To: <1354745001.59.0.580615965604.issue16623@psf.upfronthosting.co.za>
Message-ID: <1484633365.46.0.575755598435.issue16623@psf.upfronthosting.co.za>


Xiang Zhang added the comment:

#29290 reports the same problem as here.

----------
nosy: +xiang.zhang
resolution:  -> fixed
stage:  -> resolved
status: open -> closed
superseder:  -> argparse breaks long lines on NO-BREAK SPACE

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue16623>
_______________________________________

From report at bugs.python.org  Tue Jan 17 01:13:07 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Tue, 17 Jan 2017 06:13:07 +0000
Subject: [issue16623] argparse help formatter does not honor non-breaking space
In-Reply-To: <1354745001.59.0.580615965604.issue16623@psf.upfronthosting.co.za>
Message-ID: <1484633587.46.0.939461380845.issue16623@psf.upfronthosting.co.za>


Xiang Zhang added the comment:

Ahh, sorry. I made a mistake. It's not the same as 29290. This is about 2.7. Although #20491 is closed but 2.7 seems not patched. Nosy Serhiy.

----------
nosy: +serhiy.storchaka
resolution: fixed -> 
stage: resolved -> needs patch
status: closed -> open
superseder: argparse breaks long lines on NO-BREAK SPACE -> 

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue16623>
_______________________________________

From report at bugs.python.org  Tue Jan 17 01:24:01 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Tue, 17 Jan 2017 06:24:01 +0000
Subject: [issue29286] Use METH_FASTCALL in str methods
In-Reply-To: <1484587851.78.0.529922329593.issue29286@psf.upfronthosting.co.za>
Message-ID: <1484634241.78.0.40343573538.issue29286@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

Oh, I have wrote almost the same patch before going to sleep yesteday! ;)  But the building crashed (likely due to a bug in _PyStack_UnpackDict()) and it was too late to resolve this.

I would prefer to rename "l" to "nargs" in PyArg_UnpackStack_impl and don't use upper case and namespace prefix in static functions.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29286>
_______________________________________

From report at bugs.python.org  Tue Jan 17 01:38:37 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Tue, 17 Jan 2017 06:38:37 +0000
Subject: [issue28638] Creating namedtuple is too slow to be used in common
 stdlib (e.g. functools)
In-Reply-To: <1478578044.23.0.961493371476.issue28638@psf.upfronthosting.co.za>
Message-ID: <1484635117.89.0.226958130921.issue28638@psf.upfronthosting.co.za>


Raymond Hettinger added the comment:

Sorry INADA but I think this is all a foolish and inconsequential optimization that complicates the code in a way that isn't worth it (saving only a 1/2 millisecond in a single import. Also, we don't want the argument clinic code to start invading the pure python code which is used by other Python implementations.

----------
resolution:  -> rejected
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28638>
_______________________________________

From report at bugs.python.org  Tue Jan 17 01:44:23 2017
From: report at bugs.python.org (Roundup Robot)
Date: Tue, 17 Jan 2017 06:44:23 +0000
Subject: [issue29011] No entry Deque in typing.py
In-Reply-To: <1482128373.3.0.983889719325.issue29011@psf.upfronthosting.co.za>
Message-ID: <20170117064420.7323.8113.FD784041@psf.io>


Roundup Robot added the comment:

New changeset dfefbf6f6c73 by Raymond Hettinger in branch '3.5':
Issue #29011:  Fix an important omission by adding Deque to the typing module.
https://hg.python.org/cpython/rev/dfefbf6f6c73

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29011>
_______________________________________

From report at bugs.python.org  Tue Jan 17 01:44:44 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Tue, 17 Jan 2017 06:44:44 +0000
Subject: [issue29011] No entry Deque in typing.py
In-Reply-To: <1482128373.3.0.983889719325.issue29011@psf.upfronthosting.co.za>
Message-ID: <1484635484.92.0.143028590421.issue29011@psf.upfronthosting.co.za>


Changes by Raymond Hettinger <raymond.hettinger at gmail.com>:


----------
resolution:  -> fixed
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29011>
_______________________________________

From report at bugs.python.org  Tue Jan 17 01:48:49 2017
From: report at bugs.python.org (Mark Lawrence)
Date: Tue, 17 Jan 2017 06:48:49 +0000
Subject: [issue16623] argparse help formatter does not honor non-breaking space
In-Reply-To: <1354745001.59.0.580615965604.issue16623@psf.upfronthosting.co.za>
Message-ID: <1484635729.82.0.945888970702.issue16623@psf.upfronthosting.co.za>


Changes by Mark Lawrence <breamoreboy at gmail.com>:


----------
nosy:  -BreamoreBoy

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue16623>
_______________________________________

From report at bugs.python.org  Tue Jan 17 02:24:59 2017
From: report at bugs.python.org (INADA Naoki)
Date: Tue, 17 Jan 2017 07:24:59 +0000
Subject: [issue29029] Faster positional arguments parsing in
 PyArg_ParseTupleAndKeywords
In-Reply-To: <1482269418.38.0.550958237488.issue29029@psf.upfronthosting.co.za>
Message-ID: <1484637899.09.0.443547051774.issue29029@psf.upfronthosting.co.za>


INADA Naoki added the comment:

LGTM again.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29029>
_______________________________________

From report at bugs.python.org  Tue Jan 17 03:02:46 2017
From: report at bugs.python.org (Ma Lin)
Date: Tue, 17 Jan 2017 08:02:46 +0000
Subject: [issue28518] execute("begin immediate") throwing OperationalError
In-Reply-To: <1477307118.31.0.70785963389.issue28518@psf.upfronthosting.co.za>
Message-ID: <1484640166.07.0.347981365542.issue28518@psf.upfronthosting.co.za>


Ma Lin added the comment:

some comments on Aviv Palivoda's patch.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28518>
_______________________________________

From report at bugs.python.org  Tue Jan 17 03:04:31 2017
From: report at bugs.python.org (ipolcak)
Date: Tue, 17 Jan 2017 08:04:31 +0000
Subject: [issue29291] Misleading text in the documentation of re library for
 non-greedy match
Message-ID: <1484640271.49.0.239743289753.issue29291@psf.upfronthosting.co.za>


New submission from ipolcak:

The text about non-greedy match in the documentation for re library is misleading.

The docs for py2.7 (https://docs.python.org/2.7/library/re.html) and 3.6 (https://docs.python.org/3.6/library/re.html) says: "The '*', '+', and '?' qualifiers are all greedy; they match as much text as possible. Sometimes this behaviour isn?t desired; if the RE <.*> is matched against <a> b <c>, it will match the entire string, and not just <a>. Adding ? after the qualifier makes it perform the match in non-greedy or minimal fashion; as few characters as possible will be matched. Using the RE <.*?> will match only <a>."

The docs for py3.4 (https://docs.python.org/3.4/library/re.html) offers a little bit different example:
"The '*', '+', and '?' qualifiers are all greedy; they match as much text as possible. Sometimes this behaviour isn?t desired; if the RE <.*> is matched against '<H1>title</H1>', it will match the entire string, and not just '<H1>'. Adding '?' after the qualifier makes it perform the match in non-greedy or minimal fashion; as few characters as possible will be matched. Using .*? in the previous expression will match only '<H1>'."

However, in reality if the non-greedy match is not successful, it might fallback to the greedy match, see:

>>> import re
>>> a = re.compile(r"<.*?><span>")
>>> a.match("<a> b <c><span>")
<_sre.SRE_Match object; span=(0, 15), match='<a> b <c><span>'>
>>> a.search("<a> b <c><span>")
<_sre.SRE_Match object; span=(0, 15), match='<a> b <c><span>'>

So the '<.*?>' part of the regex matches '<a> b <c>' in this example. I propose to add to the documentation the following text:

"However, note that even the non-greedy version can match additional text, for example consider the RE '(<.*>)<d>' to be matched against '<a> b <c><d>'. The match is successful and the unnamed group contains '<a> b <c>'."

----------
assignee: docs at python
components: Documentation
messages: 285619
nosy: docs at python, ipolcak
priority: normal
severity: normal
status: open
title: Misleading text in the documentation of re library for non-greedy match
type: behavior
versions: Python 2.7, Python 3.4, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29291>
_______________________________________

From report at bugs.python.org  Tue Jan 17 03:06:20 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Tue, 17 Jan 2017 08:06:20 +0000
Subject: [issue29292] Missing a parameter in PyEval_EvalCodeEx doc
Message-ID: <1484640380.78.0.065326592654.issue29292@psf.upfronthosting.co.za>


New submission from Xiang Zhang:

The signature of PyEval_EvalCodeEx now gets a "PyObject *kwdefs" parameter but the doc doesn't mention it.

----------
assignee: docs at python
components: Documentation
messages: 285620
nosy: docs at python, xiang.zhang
priority: normal
severity: normal
status: open
title: Missing a parameter in PyEval_EvalCodeEx doc
versions: Python 3.5, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29292>
_______________________________________

From report at bugs.python.org  Tue Jan 17 03:07:56 2017
From: report at bugs.python.org (Roundup Robot)
Date: Tue, 17 Jan 2017 08:07:56 +0000
Subject: [issue29029] Faster positional arguments parsing in
 PyArg_ParseTupleAndKeywords
In-Reply-To: <1482269418.38.0.550958237488.issue29029@psf.upfronthosting.co.za>
Message-ID: <20170117080753.13947.55091.7B46CF94@psf.io>


Roundup Robot added the comment:

New changeset a8563ef0eb8a by Serhiy Storchaka in branch 'default':
Issue #29029: Speed up processing positional arguments in
https://hg.python.org/cpython/rev/a8563ef0eb8a

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29029>
_______________________________________

From report at bugs.python.org  Tue Jan 17 03:15:00 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Tue, 17 Jan 2017 08:15:00 +0000
Subject: [issue29029] Faster positional arguments parsing in
 PyArg_ParseTupleAndKeywords
In-Reply-To: <1482269418.38.0.550958237488.issue29029@psf.upfronthosting.co.za>
Message-ID: <1484640900.68.0.117973961592.issue29029@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

Thank you for your reviews Inada and Victor. Thank you for rebasing the patch Victor.

I were going first to try an alternative patch, maybe less efficient, but allowing to share more code. But since the code is changed so fast, it is better to commit the ready patch.

----------
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29029>
_______________________________________

From report at bugs.python.org  Tue Jan 17 03:27:40 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Tue, 17 Jan 2017 08:27:40 +0000
Subject: [issue29292] Missing a parameter in PyEval_EvalCodeEx doc
In-Reply-To: <1484640380.78.0.065326592654.issue29292@psf.upfronthosting.co.za>
Message-ID: <1484641660.07.0.678004572296.issue29292@psf.upfronthosting.co.za>


Changes by Xiang Zhang <angwerzx at 126.com>:


----------
keywords: +easy
stage:  -> needs patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29292>
_______________________________________

From report at bugs.python.org  Tue Jan 17 03:43:56 2017
From: report at bugs.python.org (Victor de la Fuente)
Date: Tue, 17 Jan 2017 08:43:56 +0000
Subject: [issue29293] Missing parameter "n" on
 multiprocessing.Condition.notify()
Message-ID: <1484642636.98.0.126375980974.issue29293@psf.upfronthosting.co.za>


New submission from Victor de la Fuente:

Versions:
Darwin MacBook-Pro-de-Victor.local 16.3.0 Darwin Kernel Version 16.3.0: Thu Nov 17 20:23:58 PST 2016; root:xnu-3789.31.2~1/RELEASE_X86_64 x86_64
Python 3.6.0 (v3.6.0:41df79263a11, Dec 22 2016, 17:23:13) 
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin

import multiprocessing as mp

condition = mp.Condition()

with condition:
    condition.notify(2) #Docs show notify(n=1)
#TypeError: notify() takes 1 positional argument but 2 were given

#Replacing the call with condition.notify(2):
#TypeError: notify() got an unexpected keyword argument 'n'


# Docs show signature: def notify(self, n=1)
# But found this debugging:
 
# multiprocessing/synchronize.py
# line 211: class Condition(object):
# .....
# line 271:     def notify(self): #<-- ?There is no n paramater?
# .....

----------
components: Library (Lib)
messages: 285623
nosy: Victor de la Fuente
priority: normal
severity: normal
status: open
title: Missing parameter "n" on multiprocessing.Condition.notify()
type: compile error
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29293>
_______________________________________

From report at bugs.python.org  Tue Jan 17 03:44:38 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Tue, 17 Jan 2017 08:44:38 +0000
Subject: [issue29291] Misleading text in the documentation of re library for
 non-greedy match
In-Reply-To: <1484640271.49.0.239743289753.issue29291@psf.upfronthosting.co.za>
Message-ID: <1484642678.25.0.58851498287.issue29291@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

The documentation doesn't look incorrect to me. The non-greedy match doesn't fallback to the greedy match, it always matches as few characters as *possible* will be matched. For example a.match("<a> b <c><span><d> e <f><span>") matches "<a> b <c><span>", not "<a> b <c><span><d> e <f><span>".

----------
nosy: +serhiy.storchaka

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29291>
_______________________________________

From report at bugs.python.org  Tue Jan 17 03:45:03 2017
From: report at bugs.python.org (Victor de la Fuente)
Date: Tue, 17 Jan 2017 08:45:03 +0000
Subject: [issue29293] Missing parameter "n" on
 multiprocessing.Condition.notify()
In-Reply-To: <1484642636.98.0.126375980974.issue29293@psf.upfronthosting.co.za>
Message-ID: <1484642703.2.0.672784032532.issue29293@psf.upfronthosting.co.za>


Victor de la Fuente added the comment:

Sorry for the typo, I meant:

#Replacing the call with condition.notify(n=2):
#TypeError: notify() got an unexpected keyword argument 'n'

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29293>
_______________________________________

From report at bugs.python.org  Tue Jan 17 03:48:45 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Tue, 17 Jan 2017 08:48:45 +0000
Subject: [issue29293] Missing parameter "n" on
 multiprocessing.Condition.notify()
In-Reply-To: <1484642636.98.0.126375980974.issue29293@psf.upfronthosting.co.za>
Message-ID: <1484642925.73.0.0156288667002.issue29293@psf.upfronthosting.co.za>


Changes by Raymond Hettinger <raymond.hettinger at gmail.com>:


----------
assignee:  -> davin
nosy: +davin

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29293>
_______________________________________

From report at bugs.python.org  Tue Jan 17 03:52:10 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Tue, 17 Jan 2017 08:52:10 +0000
Subject: [issue29291] Misleading text in the documentation of re library for
 non-greedy match
In-Reply-To: <1484640271.49.0.239743289753.issue29291@psf.upfronthosting.co.za>
Message-ID: <1484643130.44.0.926027573719.issue29291@psf.upfronthosting.co.za>


Xiang Zhang added the comment:

This doesn't look like a bug in the doc to me.

----------
nosy: +xiang.zhang

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29291>
_______________________________________

From report at bugs.python.org  Tue Jan 17 04:11:37 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Tue, 17 Jan 2017 09:11:37 +0000
Subject: [issue29291] Misleading text in the documentation of re library for
 non-greedy match
In-Reply-To: <1484640271.49.0.239743289753.issue29291@psf.upfronthosting.co.za>
Message-ID: <1484644297.16.0.312385892757.issue29291@psf.upfronthosting.co.za>


Raymond Hettinger added the comment:

I concur with Serhiy and Xiang that the docs are correct as-is.  Also IMO, the proposed rewording will cause more confusion than it cures.  Since this issue hasn't previously come up in the rather long life span of the re module, it suggests that most folks are getting the understanding they need from the existing docs.

I think it is the job for regex tutorials (rather than the main docs) to show when to use "<.*?><span>" versus "<[^>]*><span>".

----------
nosy: +rhettinger
resolution:  -> not a bug
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29291>
_______________________________________

From report at bugs.python.org  Tue Jan 17 04:12:38 2017
From: report at bugs.python.org (Thomas Heller)
Date: Tue, 17 Jan 2017 09:12:38 +0000
Subject: [issue29294] ctypes.windll.LoadLibrary refuses unicode argument
Message-ID: <1484644358.67.0.715247179434.issue29294@psf.upfronthosting.co.za>


New submission from Thomas Heller:

ctypes.windll.LoadLibrary refuses unicode argument; this is a regression in Python 2.7.13:

Python 2.7.13 (v2.7.13:a06454b1afa1, Dec 17 2016, 20:42:59) [MSC v.1500 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from ctypes import windll
>>> windll.LoadLibrary(u"kernel32.dll")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Python27\lib\ctypes\__init__.py", line 440, in LoadLibrary
    return self._dlltype(name)
  File "C:\Python27\lib\ctypes\__init__.py", line 362, in __init__
    self._handle = _dlopen(self._name, mode)
TypeError: LoadLibrary() argument 1 must be string, not unicode
>>>

It did work in Python 2.7.12 and before:

Python 2.7.12 (v2.7.12:d33e0cf91556, Jun 27 2016, 15:19:22) [MSC v.1500 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from ctypes import windll
>>> windll.LoadLibrary(u"kernel32")
<WinDLL 'kernel32', handle 77360000 at 3606f10>
>>>

----------
components: ctypes
messages: 285628
nosy: theller
priority: normal
severity: normal
status: open
title: ctypes.windll.LoadLibrary refuses unicode argument
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29294>
_______________________________________

From report at bugs.python.org  Tue Jan 17 04:17:04 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Tue, 17 Jan 2017 09:17:04 +0000
Subject: [issue29294] ctypes.windll.LoadLibrary refuses unicode argument
In-Reply-To: <1484644358.67.0.715247179434.issue29294@psf.upfronthosting.co.za>
Message-ID: <1484644624.46.0.106190679693.issue29294@psf.upfronthosting.co.za>


Xiang Zhang added the comment:

#29082 fixed it?

----------
nosy: +xiang.zhang

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29294>
_______________________________________

From report at bugs.python.org  Tue Jan 17 04:22:20 2017
From: report at bugs.python.org (Thomas Heller)
Date: Tue, 17 Jan 2017 09:22:20 +0000
Subject: [issue29294] ctypes.windll.LoadLibrary refuses unicode argument
In-Reply-To: <1484644624.46.0.106190679693.issue29294@psf.upfronthosting.co.za>
Message-ID: <8bb84156-5480-fa27-2c7a-ee5e25e0a159@ctypes.org>


Thomas Heller added the comment:

Yes, seems that it is fixed in the repository.  So I have to use 2.7.12
or wait for 2.7.14.  Thanks!

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29294>
_______________________________________

From report at bugs.python.org  Tue Jan 17 04:44:31 2017
From: report at bugs.python.org (Thomas Heller)
Date: Tue, 17 Jan 2017 09:44:31 +0000
Subject: [issue29294] ctypes.windll.LoadLibrary refuses unicode argument
In-Reply-To: <1484644358.67.0.715247179434.issue29294@psf.upfronthosting.co.za>
Message-ID: <1484646271.45.0.12884577329.issue29294@psf.upfronthosting.co.za>


Changes by Thomas Heller <theller at ctypes.org>:


----------
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29294>
_______________________________________

From report at bugs.python.org  Tue Jan 17 05:47:42 2017
From: report at bugs.python.org (INADA Naoki)
Date: Tue, 17 Jan 2017 10:47:42 +0000
Subject: [issue29295] dict: Optimize PyDict_GetItemString()
Message-ID: <1484650062.84.0.813029301463.issue29295@psf.upfronthosting.co.za>


New submission from INADA Naoki:

PyDict_GetItemString() is heavily used, especially from keyword argument parsing.
Current implementation creates temporary string for key object.
This patch avoid the temporary key string when passed C string is ASCII.

This benchmark is based on a8563ef0eb8a, so PyDict_GetItemString() calls for
parsing positional arguments is reduced already.


$ ../python -m perf compare_to -G --min-speed 2 default.json patched.json
Slower (1):
- scimark_lu: 430 ms +- 21 ms -> 446 ms +- 23 ms: 1.04x slower (+4%)

Faster (11):
- telco: 24.2 ms +- 0.4 ms -> 21.8 ms +- 0.7 ms: 1.11x faster (-10%)
- xml_etree_parse: 315 ms +- 17 ms -> 302 ms +- 14 ms: 1.04x faster (-4%)
- logging_simple: 31.6 us +- 0.3 us -> 30.4 us +- 0.3 us: 1.04x faster (-4%)
- mako: 41.6 ms +- 0.7 ms -> 40.3 ms +- 0.4 ms: 1.03x faster (-3%)
- logging_format: 36.5 us +- 0.3 us -> 35.5 us +- 0.4 us: 1.03x faster (-3%)
- float: 297 ms +- 4 ms -> 289 ms +- 4 ms: 1.03x faster (-3%)
- scimark_monte_carlo: 276 ms +- 10 ms -> 269 ms +- 7 ms: 1.02x faster (-2%)
- regex_effbot: 5.31 ms +- 0.37 ms -> 5.19 ms +- 0.06 ms: 1.02x faster (-2%)
- pickle_pure_python: 1.32 ms +- 0.02 ms -> 1.29 ms +- 0.02 ms: 1.02x faster (-2%)
- scimark_sor: 525 ms +- 9 ms -> 514 ms +- 8 ms: 1.02x faster (-2%)
- richards: 180 ms +- 3 ms -> 176 ms +- 2 ms: 1.02x faster (-2%)

Benchmark hidden because not significant (52): ...


Performance difference of telco is bit surprising.
Profiler shows the difference is from `print(t, file=outfil)` (here: https://github.com/python/performance/blob/master/performance/benchmarks/bm_telco.py#L79 )

Until most common builtin functions are converted to FASTCALL, this patch has significant
performance gain.

----------
components: Interpreter Core
files: dict_getitemascii.patch
keywords: patch
messages: 285631
nosy: inada.naoki
priority: normal
severity: normal
status: open
title: dict: Optimize PyDict_GetItemString()
type: performance
versions: Python 3.7
Added file: http://bugs.python.org/file46313/dict_getitemascii.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29295>
_______________________________________

From report at bugs.python.org  Tue Jan 17 06:34:31 2017
From: report at bugs.python.org (INADA Naoki)
Date: Tue, 17 Jan 2017 11:34:31 +0000
Subject: [issue29296] convert print() to METH_FASTCALL
Message-ID: <1484652871.69.0.785445814249.issue29296@psf.upfronthosting.co.za>


New submission from INADA Naoki:

Median +- std dev: [default] 24.2 ms +- 0.4 ms -> [patched] 19.2 ms +- 0.4 ms: 1.26x faster (-21%)

----------
files: print-fastcall.patch
keywords: patch
messages: 285632
nosy: haypo, inada.naoki
priority: normal
severity: normal
status: open
title: convert print() to METH_FASTCALL
type: performance
versions: Python 3.7
Added file: http://bugs.python.org/file46314/print-fastcall.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29296>
_______________________________________

From report at bugs.python.org  Tue Jan 17 06:54:56 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Tue, 17 Jan 2017 11:54:56 +0000
Subject: [issue29295] dict: Optimize PyDict_GetItemString()
In-Reply-To: <1484650062.84.0.813029301463.issue29295@psf.upfronthosting.co.za>
Message-ID: <1484654096.62.0.464246999195.issue29295@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

It looks to me that PyDict_GetItemString(), PyObject_GetAttrString(), etc are mainly for backward compatibility and for using in performance non-critical code. Performance critical code caches string objects.

The only code that heavily used PyDict_GetItemString() was parsing keyword arguments in PyArg_ParseTupleAndKeywords(). But this API was replaced with more efficient _PyArg_ParseTupleAndKeywordsFast() and _PyArg_ParseStackAndKeywords() for internal use. I think something similar will be exposed as public API when it become enough mature. Issue29029 made PyArg_ParseTupleAndKeywords() much less using PyDict_GetItemString().

PyDict_GetItemString() can be used with non-ASCII C strings. They are decoded with UTF-8. The patch works incorrectly in this case.

I afraid that adding more and more specialized code in Objects/dictobject.c can slow down other functions in this file. And this makes the maintenance harder.

----------
nosy: +haypo, rhettinger, serhiy.storchaka

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29295>
_______________________________________

From report at bugs.python.org  Tue Jan 17 07:03:05 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Tue, 17 Jan 2017 12:03:05 +0000
Subject: [issue29296] convert print() to METH_FASTCALL
In-Reply-To: <1484652871.69.0.785445814249.issue29296@psf.upfronthosting.co.za>
Message-ID: <1484654585.16.0.718662869152.issue29296@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

The performance of print() is not critical. It usually involves slow formatting and IO.

_PyArg_ParseStackAndKeywords() is a part of unstable, fast-evolved API. I would beware of using it in common code.

----------
nosy: +serhiy.storchaka

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29296>
_______________________________________

From report at bugs.python.org  Tue Jan 17 07:07:43 2017
From: report at bugs.python.org (INADA Naoki)
Date: Tue, 17 Jan 2017 12:07:43 +0000
Subject: [issue29295] dict: Optimize PyDict_GetItemString()
In-Reply-To: <1484650062.84.0.813029301463.issue29295@psf.upfronthosting.co.za>
Message-ID: <1484654863.68.0.0991037338429.issue29295@psf.upfronthosting.co.za>


INADA Naoki added the comment:

This patch checks passed C string is ascii or not.

But I don't want make dict complex too.  telco is more faster with issue29296.
Most common builtin functions are not METH_KEYWORDS when it merged.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29295>
_______________________________________

From report at bugs.python.org  Tue Jan 17 07:11:06 2017
From: report at bugs.python.org (INADA Naoki)
Date: Tue, 17 Jan 2017 12:11:06 +0000
Subject: [issue29295] dict: Optimize PyDict_GetItemString()
In-Reply-To: <1484650062.84.0.813029301463.issue29295@psf.upfronthosting.co.za>
Message-ID: <1484655066.59.0.0321383929022.issue29295@psf.upfronthosting.co.za>


INADA Naoki added the comment:

Close this issue for now, until profiler shows me PyDict_XxxxxString.

----------
resolution:  -> rejected
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29295>
_______________________________________

From report at bugs.python.org  Tue Jan 17 07:35:20 2017
From: report at bugs.python.org (INADA Naoki)
Date: Tue, 17 Jan 2017 12:35:20 +0000
Subject: [issue29296] convert print() to METH_FASTCALL
In-Reply-To: <1484652871.69.0.785445814249.issue29296@psf.upfronthosting.co.za>
Message-ID: <1484656520.67.0.844722777047.issue29296@psf.upfronthosting.co.za>


INADA Naoki added the comment:

I see.  AC should support this.

----------
dependencies: +Argument Clinic should understand *args and **kwargs parameters
resolution:  -> later
status: open -> pending

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29296>
_______________________________________

From report at bugs.python.org  Tue Jan 17 07:35:39 2017
From: report at bugs.python.org (INADA Naoki)
Date: Tue, 17 Jan 2017 12:35:39 +0000
Subject: [issue20291] Argument Clinic should understand *args and **kwargs
 parameters
In-Reply-To: <1390010125.91.0.58212727704.issue20291@psf.upfronthosting.co.za>
Message-ID: <1484656539.24.0.347543041577.issue20291@psf.upfronthosting.co.za>


Changes by INADA Naoki <songofacandy at gmail.com>:


----------
nosy: +inada.naoki

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue20291>
_______________________________________

From report at bugs.python.org  Tue Jan 17 08:14:35 2017
From: report at bugs.python.org (Eryk Sun)
Date: Tue, 17 Jan 2017 13:14:35 +0000
Subject: [issue29294] ctypes.windll.LoadLibrary refuses unicode argument
In-Reply-To: <1484644358.67.0.715247179434.issue29294@psf.upfronthosting.co.za>
Message-ID: <1484658875.35.0.783313927915.issue29294@psf.upfronthosting.co.za>


Changes by Eryk Sun <eryksun at gmail.com>:


----------
resolution:  -> duplicate
stage:  -> resolved
superseder:  -> In 2.7.13, _ctypes.LoadLibrary no longer accepts Unicode objects

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29294>
_______________________________________

From report at bugs.python.org  Tue Jan 17 08:23:37 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 17 Jan 2017 13:23:37 +0000
Subject: [issue29296] convert print() to METH_FASTCALL
In-Reply-To: <1484652871.69.0.785445814249.issue29296@psf.upfronthosting.co.za>
Message-ID: <1484659417.38.0.755701555539.issue29296@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Hi,

Serhiy Storchaka: "The performance of print() is not critical. It usually involves slow formatting and IO."

Oh, please, I want a print() using METH_FASTCALL! Not for the performance, but just to help me in my development!

To identify code not using FASTCALL yet, I'm putting a breakpoint on _PyStack_AsTuple() and _PyStack_AsDict(). It's common to stop on print() when testing the performance benchmark suite.

FYI currnently, I'm adding the following Python code to python-gdb.py:
---
class MyBreakpoint (gdb.Breakpoint):
    def stop(self):
        # frame of the caller
        frame = gdb.selected_frame()
        frame = frame.older()

        name = frame.name()
        if name == '_PyCFunction_FastCallKeywords':
            pass
        else:
            # callable->ob_type
            obj = frame.read_var('callable')
            obj = obj.referenced_value()['ob_type']

            obj2 = gdb.lookup_global_symbol('PyType_Type').value()

            if obj == obj2.address:
                return False

            print("don't skip", obj, obj2.address)

        gdb.execute("up")
        return True


MyBreakpoint("_PyStack_AsTuple")
MyBreakpoint("_PyStack_AsDict")
---

These macros make gdb 40x slower :-/

The code is used to skip type_call(): this function is known to not use FASTCALL yet. I plan to add tp_fastnew and tp_fastinit to optimize type_call(), but it's already hard enough to implement tp_fastcall, so I will do it later.

> _PyArg_ParseStackAndKeywords() is a part of unstable, fast-evolved API. I would beware of using it in common code.

Ah. I have a different opinion on that. Since _PyArg_ParseStackAndKeywords() and print() are part of the Python core, we have free to use private fast-moving APIs.

By the way, _PyArg_ParseStackAndKeywords() already uses the new efficient _PyArg_Parser object. Do you plan other enhancements?


INADA Naoki: "I see.  AC should support this."

Right, but IMHO it can be done later.

I tried to contribute to AC but the code is super complex. I understood that AC is not complex, but Python is complex :-D AC is full of corner cases. It's a very long list of special cases :-)

----------
status: pending -> open

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29296>
_______________________________________

From report at bugs.python.org  Tue Jan 17 08:27:12 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 17 Jan 2017 13:27:12 +0000
Subject: [issue29295] dict: Optimize PyDict_GetItemString()
In-Reply-To: <1484650062.84.0.813029301463.issue29295@psf.upfronthosting.co.za>
Message-ID: <1484659632.55.0.680955243655.issue29295@psf.upfronthosting.co.za>


STINNER Victor added the comment:

> Close this issue for now, until profiler shows me PyDict_XxxxxString.

I like Serhiy's rationale. We should try to avoid PyDict_GetItemString() wheneve possible. If PyDict_GetItemString() becomes a clear bottleneck, we can discuss again optimizing it. But in the meanwhile, I prefer to use anothe rule: an optimization should not modify the behaviour of a function. That's why I didn't optimize OrderedDict.pop() yet (it changed the docstring, AC should be enhanced for this case).

So yeah, let's close this one.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29295>
_______________________________________

From report at bugs.python.org  Tue Jan 17 08:54:40 2017
From: report at bugs.python.org (Doug Hellmann)
Date: Tue, 17 Jan 2017 13:54:40 +0000
Subject: [issue9216] FIPS support for hashlib
In-Reply-To: <1278721335.16.0.522410247151.issue9216@psf.upfronthosting.co.za>
Message-ID: <1484661280.57.0.552542087507.issue9216@psf.upfronthosting.co.za>


Doug Hellmann added the comment:

@rbcollins, I don't think providing a hashlib.fips module without md5() solves the problem. The idea is to have a way to call md5() in non-secure situations, and to signal to the FIPS system that the call is OK. A separate module would work if it included an md5() function that always did that signaling. But creating a separate module just to wrap one function like that seems like overkill, doesn't it?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue9216>
_______________________________________

From report at bugs.python.org  Tue Jan 17 09:01:18 2017
From: report at bugs.python.org (R. David Murray)
Date: Tue, 17 Jan 2017 14:01:18 +0000
Subject: [issue29280] gdbm & ndbm support missing in Windows
In-Reply-To: <1484507005.86.0.826464747916.issue29280@psf.upfronthosting.co.za>
Message-ID: <1484661678.46.0.849111230511.issue29280@psf.upfronthosting.co.za>


R. David Murray added the comment:

This has always been the case (that gdbm and ndbm are not supported on windows).  See also issue 3769 for background and dbm and windows, and issue 3783 for something to work on if you want to contribute to making the dbm story on windows better.

Googling found a gdbm port to windows, so if you are trying to use source files from linux, that might help you.  We're open to suggestions on improving the situation here, but someone has to figure out what would make sense and champion it.  Bundling the gdbm windows port is probably not an option, but making it possible to install something via PIP that will work is, if there's anything in core preventing that currently.

----------
nosy: +r.david.murray
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29280>
_______________________________________

From report at bugs.python.org  Tue Jan 17 09:05:35 2017
From: report at bugs.python.org (Peter Otten)
Date: Tue, 17 Jan 2017 14:05:35 +0000
Subject: [issue29287] IDLE needs syntax highlighting for f-strings
In-Reply-To: <1484605208.08.0.296182628472.issue29287@psf.upfronthosting.co.za>
Message-ID: <1484661935.74.0.685419601765.issue29287@psf.upfronthosting.co.za>


Changes by Peter Otten <__peter__ at web.de>:


----------
nosy: +peter.otten

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29287>
_______________________________________

From report at bugs.python.org  Tue Jan 17 09:06:01 2017
From: report at bugs.python.org (Peter Otten)
Date: Tue, 17 Jan 2017 14:06:01 +0000
Subject: [issue29290] argparse breaks long lines on NO-BREAK SPACE
In-Reply-To: <1484622521.42.0.0580919403747.issue29290@psf.upfronthosting.co.za>
Message-ID: <1484661961.21.0.487605333515.issue29290@psf.upfronthosting.co.za>


Changes by Peter Otten <__peter__ at web.de>:


----------
nosy: +peter.otten

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29290>
_______________________________________

From report at bugs.python.org  Tue Jan 17 09:08:56 2017
From: report at bugs.python.org (Peter Otten)
Date: Tue, 17 Jan 2017 14:08:56 +0000
Subject: [issue16623] argparse help formatter does not honor non-breaking space
In-Reply-To: <1354745001.59.0.580615965604.issue16623@psf.upfronthosting.co.za>
Message-ID: <1484662136.11.0.171361502465.issue16623@psf.upfronthosting.co.za>


Changes by Peter Otten <__peter__ at web.de>:


----------
nosy: +peter.otten

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue16623>
_______________________________________

From report at bugs.python.org  Tue Jan 17 09:21:01 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Tue, 17 Jan 2017 14:21:01 +0000
Subject: [issue20291] Argument Clinic should understand *args and **kwargs
 parameters
In-Reply-To: <1390010125.91.0.58212727704.issue20291@psf.upfronthosting.co.za>
Message-ID: <1484662861.56.0.106096927209.issue20291@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

I'll try to implement the support of var-positional parameters.

----------
assignee: larry -> serhiy.storchaka
versions: +Python 3.7 -Python 3.4, Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue20291>
_______________________________________

From report at bugs.python.org  Tue Jan 17 09:27:26 2017
From: report at bugs.python.org (Yolanda)
Date: Tue, 17 Jan 2017 14:27:26 +0000
Subject: [issue9216] FIPS support for hashlib
In-Reply-To: <1278721335.16.0.522410247151.issue9216@psf.upfronthosting.co.za>
Message-ID: <1484663246.46.0.356458378036.issue9216@psf.upfronthosting.co.za>


Yolanda added the comment:

I agree with Doug. From my understanding, the intention of the patch is to allow the usage of md5 for non-security purposes, without being blocked by FIPS.
Right now, md5() calls running on a FIPS enabled kernel, are blocked without discrimination of the usage, that shall be ok for hashing purposes, and more performant than other methods. This patch provides the ability to continue using md5 just flagging it properly.

----------
nosy: +yolanda.robla

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue9216>
_______________________________________

From report at bugs.python.org  Tue Jan 17 09:43:36 2017
From: report at bugs.python.org (Mo Ali)
Date: Tue, 17 Jan 2017 14:43:36 +0000
Subject: [issue29297] python3 open() does not check argument type before
 attempting to read() or write()
Message-ID: <1484664216.19.0.300485907535.issue29297@psf.upfronthosting.co.za>


New submission from Mo Ali:

Python3 open(), read(), or write()doesn't check argument type before action causing a hang.  Would like to catch exceptions but not without an exception to return.  See below.

Python3.6:
Python 3.6.0 (default, Dec 24 2016, 08:01:42)
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.42.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> test = False
>>> with open(str(test)) as f:
...     fail = f.read()
...


python2.7:
??? python2                                                                                                                                                                                     0 < 09:35:31
Python 2.7.13 (default, Dec 18 2016, 07:03:39)
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.42.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> test = False
>>> with open(test) as f:
...     fail = f.read()
...
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: coercing to Unicode: need string or buffer, bool found

----------
messages: 285644
nosy: Mo Ali
priority: normal
severity: normal
status: open
title: python3 open() does not check argument type before attempting to read() or write()
type: crash
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29297>
_______________________________________

From report at bugs.python.org  Tue Jan 17 09:46:13 2017
From: report at bugs.python.org (R. David Murray)
Date: Tue, 17 Jan 2017 14:46:13 +0000
Subject: [issue29285] Unicode errors occur inside of multi-line comments
In-Reply-To: <1484586400.61.0.654989234385.issue29285@psf.upfronthosting.co.za>
Message-ID: <1484664373.53.0.97661191537.issue29285@psf.upfronthosting.co.za>


R. David Murray added the comment:

@zach: actually, triple quoted strings *are* suggested as a way to do multiline comments, and are often used for that.  In particular, note that they do not appear in the byte code files if they are not assigned to anything or appear in the docstring position.  This is something I had to prove to myself at one point when I had the same reaction you just did to their use as comments :)  

That said, it is true that they are, first and foremost, strings, so the quoting behavior is not going to change.

----------
nosy: +r.david.murray

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29285>
_______________________________________

From report at bugs.python.org  Tue Jan 17 09:52:49 2017
From: report at bugs.python.org (zachrahan)
Date: Tue, 17 Jan 2017 14:52:49 +0000
Subject: [issue29298] argparse fails with required subparsers, un-named dest, 
 and empty argv
Message-ID: <1484664769.91.0.625725054847.issue29298@psf.upfronthosting.co.za>


New submission from zachrahan:

In python 3.6 (and several versions back), using argparse with required subparsers will cause an unhelpful TypeError if the 'dest' parameter is not explicitly specified, and no arguments are provided.

Test case:
import argparse
parser = argparse.ArgumentParser()
subparsers = parser.add_subparsers()
subparsers.required = True
args = parser.parse_args([])

Observed result:
TypeError: sequence item 0: expected str instance, NoneType found

If the line above is changed to:
subparsers = parser.add_subparsers(dest='function')

Then the following is printed to stderr:
usage: python [-h] {} ...
python: error: the following arguments are required: function

This issue goes back at least several years:
http://stackoverflow.com/questions/23349349/argparse-with-required-subparser/23354355

Though it seems odd to not specify a dest in the add_subparsers line, the pattern is not completely useless. The below works fine without setting a 'dest' in add_subparsers, except when argv is empty:
sub1 = subparsers.add_parser('print')
sub1.set_defaults(function=print)

However, an empty argv produces the unexpected TypeError above. I'm not sure if argparse should provide a more useful exception in this case, or if there is a clean way to do the right thing without a dest specified.

----------
components: Library (Lib)
messages: 285646
nosy: zachrahan
priority: normal
severity: normal
status: open
title: argparse fails with required subparsers, un-named dest, and empty argv
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29298>
_______________________________________

From report at bugs.python.org  Tue Jan 17 10:03:03 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Tue, 17 Jan 2017 15:03:03 +0000
Subject: [issue20291] Argument Clinic should understand *args and **kwargs
 parameters
In-Reply-To: <1390010125.91.0.58212727704.issue20291@psf.upfronthosting.co.za>
Message-ID: <1484665383.47.0.580618630565.issue20291@psf.upfronthosting.co.za>


Raymond Hettinger added the comment:

In case it is helpful, here's my list of examples where the AC and existing signature objects are insufficiently expressive:


type(obj)
type(name, bases, mapping)
    two different signatures depending on type   

range(stop)
range(start, stop)
range(start, stop, step)

dict.pop(key[, default])
   default of None has different meaning than missing default
   which raises KeyError when the key is missing

itertools.permutations(iterable[, r])
   where the absence of *r* implies r=len(iterable)

bisect.bisect_right(a, x[, lo[, hi]]) -> index
   where the absence of *hi* implies hi=len(a)

min(iterable, *[, default=obj, key=func]) -> value
min(arg1, arg2, *args, *[, key=func]) -> value
   has two signatures depending on the number of
   positional arguments and a keyword argument
   only used in the first signature.  It's implementation
   is also shared with max().


dict() -> new empty dictionary
dict(mapping) -> new dictionary initialized from a mapping object's
   (key, value) pairs
dict(iterable) -> new dictionary initialized as if via:
   d = {}
   for k, v in iterable:
       d[k] = v
dict(**kwargs) -> new dictionary initialized with the name=value pairs
   in the keyword argument list.  For example:  dict(one=1, two=2)

def sumseq(seq, a=0, b=None):
    # Pure python code with nullable int
    if b is None:
        b = len(seq)
    return sum(seq[a:b])

----------
nosy: +rhettinger

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue20291>
_______________________________________

From report at bugs.python.org  Tue Jan 17 10:06:54 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Tue, 17 Jan 2017 15:06:54 +0000
Subject: [issue29297] python3 open() does not check argument type before
 attempting to read() or write()
In-Reply-To: <1484664216.19.0.300485907535.issue29297@psf.upfronthosting.co.za>
Message-ID: <1484665614.38.0.864306793552.issue29297@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

The first example raises an error:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
FileNotFoundError: [Errno 2] No such file or directory: 'False'

Unless you have a file named "False" in the current directory of course.

What behavior you expected?

----------
nosy: +serhiy.storchaka

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29297>
_______________________________________

From report at bugs.python.org  Tue Jan 17 10:10:11 2017
From: report at bugs.python.org (Mo Ali)
Date: Tue, 17 Jan 2017 15:10:11 +0000
Subject: [issue29297] python3 open() does not check argument type before
 attempting to read() or write()
In-Reply-To: <1484664216.19.0.300485907535.issue29297@psf.upfronthosting.co.za>
Message-ID: <1484665811.34.0.271605857763.issue29297@psf.upfronthosting.co.za>


Mo Ali added the comment:

Serhiy,

I expected a type error or a filenotfound like you received, however mine doesn't return the same. It just hangs.  I've attached a picture.  Also, I meant this to be for 3.6 not 3.5.

>>> test = False
>>> with open(test) as f:
...     fail = f.read()
...

----------
versions: +Python 3.6 -Python 3.5
Added file: http://bugs.python.org/file46315/Screen Shot 2017-01-17 at 10.09.35 AM.png

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29297>
_______________________________________

From report at bugs.python.org  Tue Jan 17 10:17:14 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 17 Jan 2017 15:17:14 +0000
Subject: [issue29299] Argument Clinic: Fix signature of optional
 positional-only arguments
Message-ID: <1484666234.33.0.914406658402.issue29299@psf.upfronthosting.co.za>


New submission from STINNER Victor:

When a function has only positional arguments and at least one argument is optional, the expected signature is:

  func(mandatory_arg1, mandatory_arg2[, optional_arg3[, optinal_arg4]])

For example, the signature of format() is inconsistent with its documentation.

Signature:
---
$ python3 -c 'help(format)'|cat
Help on built-in function format in module builtins:

format(value, format_spec='', /)
    Return value.__format__(format_spec)
    
    format_spec defaults to the empty string
---

Documentation:
---
.. function:: format(value[, format_spec])
---

Attached patch is a first attempt to fix the issue. The problem is that my heuristic to check if an argument is "optional" doesn't seem to work as expected in all cases. I chose to check if the C default is NULL.

The problem is that some functions defines a C default to NULL whereas the Python default is set to a different value and is correct.

Example with _io.FileIO.truncate:

    /*[clinic input]
    _io.FileIO.truncate
        size as posobj: object = NULL
        /

whereas the documentation says that the default is None:

   .. method:: truncate(size=None)

It's easy to fix the default, but in this case my change doesn't fix the signature anymore since the C default is still NULL:

    /*[clinic input]
    _io.FileIO.truncate
        size as posobj: object(c_default="NULL") = None
        /

We need a different heuristic than C default is NULL, or we should fix functions where the heuristic fails.

----------
components: Argument Clinic
files: ac_optional_positional.patch
keywords: patch
messages: 285650
nosy: haypo, larry, rhettinger, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Argument Clinic: Fix signature of optional positional-only arguments
versions: Python 3.7
Added file: http://bugs.python.org/file46316/ac_optional_positional.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29299>
_______________________________________

From report at bugs.python.org  Tue Jan 17 10:18:15 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 17 Jan 2017 15:18:15 +0000
Subject: [issue29299] Argument Clinic: Fix signature of optional
 positional-only arguments
In-Reply-To: <1484666234.33.0.914406658402.issue29299@psf.upfronthosting.co.za>
Message-ID: <1484666295.7.0.515790743359.issue29299@psf.upfronthosting.co.za>


STINNER Victor added the comment:

List of functions modified when Argument Clinic is run to regenerate .c.h files:

builtin_format
builtin_getattr
builtin_input
builtin_sum
cmath_log
_codecs_ascii_decode
_codecs_ascii_encode
_codecs_charmap_decode
_codecs_charmap_encode
_codecs_code_page_encode
_codecs_escape_decode
_codecs_escape_encode
_codecs_latin_1_decode
_codecs_latin_1_encode
_codecs_mbcs_encode
_codecs_oem_encode
_codecs_raw_unicode_escape_decode
_codecs_raw_unicode_escape_encode
_codecs_readbuffer_encode
_codecs_unicode_escape_decode
_codecs_unicode_escape_encode
_codecs_unicode_internal_decode
_codecs_unicode_internal_encode
_codecs_utf_16_be_encode
_codecs_utf_16_le_encode
_codecs_utf_32_be_encode
_codecs_utf_32_le_encode
_codecs_utf_7_encode
_codecs_utf_8_encode
_dbm_dbm_get
_dbm_dbm_setdefault
fcntl_fcntl
_imp_create_dynamic
_io_FileIO_truncate
os_putenv
os_unsetenv
pyexpat_xmlparser_ExternalEntityParserCreate
_sre_SRE_Match_end
_sre_SRE_Match_span
_sre_SRE_Match_start
_tkinter_create
unicodedata_UCD_decimal
unicodedata_UCD_digit
unicodedata_UCD_name
unicodedata_UCD_numeric
unicode_lstrip
unicode_maketrans
unicode_rstrip

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29299>
_______________________________________

From report at bugs.python.org  Tue Jan 17 10:18:58 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 17 Jan 2017 15:18:58 +0000
Subject: [issue29289] Convert OrderedDict methods to Argument Clinic
In-Reply-To: <1484609084.15.0.406928886689.issue29289@psf.upfronthosting.co.za>
Message-ID: <1484666338.56.0.333650271818.issue29289@psf.upfronthosting.co.za>


STINNER Victor added the comment:

For OrderedDict.pop(), I created the issue #29299: "Argument Clinic: Fix signature of optional positional-only arguments".

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29289>
_______________________________________

From report at bugs.python.org  Tue Jan 17 10:20:15 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Tue, 17 Jan 2017 15:20:15 +0000
Subject: [issue29297] python3 open() does not check argument type before
 attempting to read() or write()
In-Reply-To: <1484664216.19.0.300485907535.issue29297@psf.upfronthosting.co.za>
Message-ID: <1484666415.53.0.758229659838.issue29297@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

Builtin open() in Python 3 (and io.open() in Python 2.7) accept unicode strings, byte strings and integers as the first arguments (general path-like objects also are supported in Python 3.6, but it doesn't matter). bool is a subtype of int, and False is equal to integer 0. 0 is a file descriptor of sys.stdin.

>>> import sys
>>> sys.stdin.fileno()
0

Thus open(False) is equivalent to open(sys.stdin.fileno()). It creates a file object that wraps the file descriptor 0. This is legitimate operation.

----------
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29297>
_______________________________________

From report at bugs.python.org  Tue Jan 17 10:20:27 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 17 Jan 2017 15:20:27 +0000
Subject: [issue20291] Argument Clinic should understand *args and **kwargs
 parameters
In-Reply-To: <1390010125.91.0.58212727704.issue20291@psf.upfronthosting.co.za>
Message-ID: <1484666427.33.0.160216155541.issue20291@psf.upfronthosting.co.za>


STINNER Victor added the comment:

FYI I started to work on a different Argument Clinic enhancement, issue #29299: "Argument Clinic: Fix signature of optional positional-only arguments".

----------
nosy: +haypo

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue20291>
_______________________________________

From report at bugs.python.org  Tue Jan 17 10:21:28 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Tue, 17 Jan 2017 15:21:28 +0000
Subject: [issue29289] Convert OrderedDict methods to Argument Clinic
In-Reply-To: <1484609084.15.0.406928886689.issue29289@psf.upfronthosting.co.za>
Message-ID: <1484666488.93.0.278340452244.issue29289@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

I have added other comments on Rietveld (mostly about docstrings).

----------
nosy: +serhiy.storchaka

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29289>
_______________________________________

From report at bugs.python.org  Tue Jan 17 10:21:44 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 17 Jan 2017 15:21:44 +0000
Subject: [issue29299] Argument Clinic: Fix signature of optional
 positional-only arguments
In-Reply-To: <1484666234.33.0.914406658402.issue29299@psf.upfronthosting.co.za>
Message-ID: <1484666504.97.0.903890829384.issue29299@psf.upfronthosting.co.za>


STINNER Victor added the comment:

See also issue #20291: "Argument Clinic should understand *args and **kwargs parameters".

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29299>
_______________________________________

From report at bugs.python.org  Tue Jan 17 10:25:17 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 17 Jan 2017 15:25:17 +0000
Subject: [issue29299] Argument Clinic: Fix signature of optional
 positional-only arguments
In-Reply-To: <1484666234.33.0.914406658402.issue29299@psf.upfronthosting.co.za>
Message-ID: <1484666717.3.0.527900261135.issue29299@psf.upfronthosting.co.za>


STINNER Victor added the comment:

This issue is blocking me to convert more functions to Argument Clinic. See for example attached getattr_ac.patch which converts getattr() to AC. Without ac_optional_positional.patch, AC generates the signature:

   "getattr($module, object, name, default=None, /)\n"

whereas the following signature is expected:

   "getattr($module, object, name[, default])\n"

----------
Added file: http://bugs.python.org/file46317/getattr_ac.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29299>
_______________________________________

From report at bugs.python.org  Tue Jan 17 10:29:28 2017
From: report at bugs.python.org (Eryk Sun)
Date: Tue, 17 Jan 2017 15:29:28 +0000
Subject: [issue29285] Unicode errors occur inside of multi-line comments
In-Reply-To: <1484586400.61.0.654989234385.issue29285@psf.upfronthosting.co.za>
Message-ID: <1484666968.19.0.201362820307.issue29285@psf.upfronthosting.co.za>


Eryk Sun added the comment:

> they do not appear in the byte code files

It's simple to coonfirm that unassigned string literals get thrown away.:

    >>> code = compile('"doc"\n"unused"\n"us"+"ed"', '', 'exec')
    >>> code.co_consts
    ('doc', 'us', 'ed', None, 'used')
    >>> dis.dis(code)
      1           0 LOAD_CONST               0 ('doc')
                  3 STORE_NAME               0 (__doc__)

      3           6 LOAD_CONST               4 ('used')
                  9 POP_TOP
                 10 LOAD_CONST               3 (None)
                 13 RETURN_VALUE

However, they're retained up to the abstract syntax tree:

    >>> print(ast.dump(ast.parse('"doc"\n"unused"\n"us"+"ed"', '', 'exec')))
    Module(body=[
        Expr(value=Str(s='doc')),
        Expr(value=Str(s='unused')),
        Expr(value=BinOp(left=Str(s='us'),
                         op=Add(),
                         right=Str(s='ed')))])

Sans the doc string behavior, this applies to literals in general. Thus even though the compiler discards these objects, the literal still has to be parsed like any other.

----------
nosy: +eryksun

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29285>
_______________________________________

From report at bugs.python.org  Tue Jan 17 10:34:50 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Tue, 17 Jan 2017 15:34:50 +0000
Subject: [issue29299] Argument Clinic: Fix signature of optional
 positional-only arguments
In-Reply-To: <1484666234.33.0.914406658402.issue29299@psf.upfronthosting.co.za>
Message-ID: <1484667290.64.0.518377339265.issue29299@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

The problem is that

  func(mandatory_arg1, mandatory_arg2[, optional_arg3[, optinal_arg4]])

is not compatible with the inspect module.

In many case a meaningful default value was added if this is possible. For example the Python default shown in the signature can be set to '', 'utf-8' or 'strict' while the C default value is NULL for performance. If the parameter is upper index in the sequence it can be set to sys.maxsize (Py_SSIZE_T_MAX in C).

This is not always possible. For example there is not default value for dict.pop().

----------
nosy: +yselivanov

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29299>
_______________________________________

From report at bugs.python.org  Tue Jan 17 10:49:52 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Tue, 17 Jan 2017 15:49:52 +0000
Subject: [issue20291] Argument Clinic should understand *args and **kwargs
 parameters
In-Reply-To: <1390010125.91.0.58212727704.issue20291@psf.upfronthosting.co.za>
Message-ID: <1484668192.66.0.112928393956.issue20291@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

Thank you for your examples Raymond, but they don't directly related to this issue, implementing support of var-positional and var-keyword parameters. I believe that it is possible to solve it, and the solution is complex, but is not extremal hard. I'm working on the first part of this.

Your examples show other, more hard issue. It looks to me that the only solution of that issue is to add support of multiple signatures for functions. But this can break the API of the inspect module.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue20291>
_______________________________________

From report at bugs.python.org  Tue Jan 17 10:52:04 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Tue, 17 Jan 2017 15:52:04 +0000
Subject: [issue29299] Argument Clinic: Fix signature of optional
 positional-only arguments
In-Reply-To: <1484666234.33.0.914406658402.issue29299@psf.upfronthosting.co.za>
Message-ID: <1484668324.34.0.523159351971.issue29299@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

Please don't change this part of Argument Clinic without Larry. There were several attempts to solve this problem, I don't like current status, but this is Larry's decision.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29299>
_______________________________________

From report at bugs.python.org  Tue Jan 17 11:22:34 2017
From: report at bugs.python.org (Ammar Askar)
Date: Tue, 17 Jan 2017 16:22:34 +0000
Subject: [issue29292] Missing a parameter in PyEval_EvalCodeEx doc
In-Reply-To: <1484640380.78.0.065326592654.issue29292@psf.upfronthosting.co.za>
Message-ID: <1484670154.86.0.129286533174.issue29292@psf.upfronthosting.co.za>


Ammar Askar added the comment:

It looks like a basic description of kwdefs was added as part of this commit: https://github.com/python/cpython/commit/7811a5d1c93f2aa0b357444eeb3f1ddc242ac57a

"keywords and defaults," however, the kwdefs argument was never added to the prototype.

I've attached a patch which adds the argument to the prototype and clarifies the language a little bit.

----------
keywords: +patch
nosy: +ammar2
Added file: http://bugs.python.org/file46318/kwdefs_docs.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29292>
_______________________________________

From report at bugs.python.org  Tue Jan 17 11:31:45 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 17 Jan 2017 16:31:45 +0000
Subject: [issue29300] Modify the _struct module to use FASTCALL and Argument
 Clinic
Message-ID: <1484670705.59.0.0193443230555.issue29300@psf.upfronthosting.co.za>


New submission from STINNER Victor:

Attached patch modify the _struct module to use FASTCALL and Argument Clinic.

AC requires a summary line, so I duplicated the first sentence of iter_unpack() docstring:

iter_unpack(fmt, buffer, /)
    Return an iterator yielding tuples unpacked from the given bytes.
    
    Return an iterator yielding tuples unpacked from the given bytes
    source according to the format string, like a repeated invocation of
    unpack_from().  Requires that the bytes length be a multiple of the
    format struct size.

----------
components: Argument Clinic
files: struct_fastcall.patch
keywords: patch
messages: 285663
nosy: haypo, inada.naoki, larry, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Modify the _struct module to use FASTCALL and Argument Clinic
type: performance
versions: Python 3.7
Added file: http://bugs.python.org/file46319/struct_fastcall.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29300>
_______________________________________

From report at bugs.python.org  Tue Jan 17 11:36:12 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Tue, 17 Jan 2017 16:36:12 +0000
Subject: [issue29292] Missing a parameter in PyEval_EvalCodeEx doc
In-Reply-To: <1484640380.78.0.065326592654.issue29292@psf.upfronthosting.co.za>
Message-ID: <1484670972.18.0.758022274816.issue29292@psf.upfronthosting.co.za>


Xiang Zhang added the comment:

> "keywords and defaults," however, the kwdefs argument was never added to the prototype.

I don't think this is about the missing "kwdefs". I think "arrays of arguments, keywords and defaults" is a whole part describing "PyObject **args, int argcount, PyObject **kws, int kwcount, PyObject **defs, int defcount". "kwdefs" is a dict containing the default values of keyword-only parameters in Py3? I am not sure.

----------
nosy: +haypo

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29292>
_______________________________________

From report at bugs.python.org  Tue Jan 17 11:38:44 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 17 Jan 2017 16:38:44 +0000
Subject: [issue29301] decimal: Use FASTCALL and/or Argument Clinic
Message-ID: <1484671124.21.0.493200110844.issue29301@psf.upfronthosting.co.za>


New submission from STINNER Victor:

I'm currently working on the isuse #29259: "Add tp_fastcall to PyTypeObject: support FASTCALL calling convention for all callable objects". I used bm_telco of the performance benchmark suite to check which functions still require to create a temporary tuple for positional arguments and a temporary dict for keyword arguments. I found 3 remaining functions which have an impact on the result of the benchmark:

* print(): optimized by the issue #29296
* _struct.unpack(): I just created the issue #29300 "Modify the _struct module to use FASTCALL and Argument Clinic"
* _decimal.Decimal.quantize()

I would like to know if Stephan would be ok to modify the _decimal module to use FASTCALL. I know that recently he reverted changes to keep the same code base on Python 3.5, 3.6 and 3.7.

With 4 changes (tp_fastcall #29259, print #29296, unpack #29300 and this issue), bm_telco becomes 22% faster which is not negligible!

   20.9 ms +- 0.5 ms => 16.4 ms +- 0.5 ms

Attached decimal.patch patch is the strict minimum change to optimize bm_telco, but I would prefer to change all _decimal functions and methods using METH_VARARGS and METH_VARARGS|METH_KEYWORDS to convert them to METH_FASTCALL.

The best would be to use Argument Clinic. AC exists since Python 3.5, so it should be possible to keep the same code base on Python 3.5-3.7, only generated code would be different.

----------
files: decimal.patch
keywords: patch
messages: 285665
nosy: haypo, skrah
priority: normal
severity: normal
status: open
title: decimal: Use FASTCALL and/or Argument Clinic
type: performance
versions: Python 3.7
Added file: http://bugs.python.org/file46320/decimal.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29301>
_______________________________________

From report at bugs.python.org  Tue Jan 17 11:38:48 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 17 Jan 2017 16:38:48 +0000
Subject: [issue29301] decimal: Use FASTCALL and/or Argument Clinic
In-Reply-To: <1484671124.21.0.493200110844.issue29301@psf.upfronthosting.co.za>
Message-ID: <1484671128.42.0.961076456728.issue29301@psf.upfronthosting.co.za>


Changes by STINNER Victor <victor.stinner at gmail.com>:


----------
nosy: +serhiy.storchaka

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29301>
_______________________________________

From report at bugs.python.org  Tue Jan 17 11:42:21 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 17 Jan 2017 16:42:21 +0000
Subject: [issue29259] Add tp_fastcall to PyTypeObject: support FASTCALL
 calling convention for all callable objects
In-Reply-To: <1484310274.99.0.832491900546.issue29259@psf.upfronthosting.co.za>
Message-ID: <1484671341.97.0.757279804141.issue29259@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Back on performances: if you combined tp_fastcall-2.patch + 3 minor changes to use METH_FASTCALL, the bm_telco benchmark becomes 22% faster, which is significant for a benchmark:
http://bugs.python.org/issue29301#msg285665

"20.9 ms +- 0.5 ms => 16.4 ms +- 0.5 ms"

It also means that 22% of the benchmark time is currently spent in handling function arguments! I expected that most of the time would be spent on maths and I/O!

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29259>
_______________________________________

From report at bugs.python.org  Tue Jan 17 11:57:09 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 17 Jan 2017 16:57:09 +0000
Subject: [issue29259] Add tp_fastcall to PyTypeObject: support FASTCALL
 calling convention for all callable objects
In-Reply-To: <1484310274.99.0.832491900546.issue29259@psf.upfronthosting.co.za>
Message-ID: <1484672229.61.0.323151323612.issue29259@psf.upfronthosting.co.za>


STINNER Victor added the comment:

I checked the effect of individual patches:

* tp_fastcall-2
* print
* struct
* decimal

tp_fastcall-2 + print + struct + decimal: 16.3 ms +- 0.6 ms
tp_fastcall-2 + struct + decimal: 21.2 ms +- 0.3 ms
tp_fastcall-2 + print: 16.7 ms +- 0.2 ms

Oh wow, I didn't expect that print would be the bottleneck of this benchmark!? There is a single print() in the hotcode of bm_telco:

   print(t, file=outfil)

Maybe it's the expensive price of creating a temporary dictionary?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29259>
_______________________________________

From report at bugs.python.org  Tue Jan 17 12:03:15 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 17 Jan 2017 17:03:15 +0000
Subject: [issue29296] convert print() to METH_FASTCALL
In-Reply-To: <1484652871.69.0.785445814249.issue29296@psf.upfronthosting.co.za>
Message-ID: <1484672595.93.0.722204056743.issue29296@psf.upfronthosting.co.za>


STINNER Victor added the comment:

I reviewed print-fastcall.patch: LGTM, but I proposed a minor change.


Serhiy Storchaka: "The performance of print() is not critical. It usually involves slow formatting and IO."

I also had the same understanding of print(), but I just analyzed performances of the bm_telco benchmark, and it seems just like handling function parameters of print() take 20% of the runtime!?

   http://bugs.python.org/issue29259#msg285667

bm_telco reference (unpatched) => with issue #29259 tp_fastcall-2.patch and print-fastcall.patch:

   20.9 ms +- 0.5 ms => 16.7 ms +- 0.2 ms

print-fastcall.patch makes bm_telco 20% faster! Just to make sure, I ran again bm_telco only with tp_fastcall-2.patch:

   telco: Median +- std dev: 21.4 ms +- 0.8 ms

Maybe we should optimize _PyStack_AsDict(), but that's a different topic ;-)

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29296>
_______________________________________

From report at bugs.python.org  Tue Jan 17 12:07:07 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Tue, 17 Jan 2017 17:07:07 +0000
Subject: [issue29300] Modify the _struct module to use FASTCALL and Argument
 Clinic
In-Reply-To: <1484670705.59.0.0193443230555.issue29300@psf.upfronthosting.co.za>
Message-ID: <1484672827.16.0.252865014657.issue29300@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

Why not just convert the struct module to Argument Clinic?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29300>
_______________________________________

From report at bugs.python.org  Tue Jan 17 12:10:02 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 17 Jan 2017 17:10:02 +0000
Subject: [issue29301] decimal: Use FASTCALL and/or Argument Clinic
In-Reply-To: <1484671124.21.0.493200110844.issue29301@psf.upfronthosting.co.za>
Message-ID: <1484673002.84.0.91683808833.issue29301@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Oh wait, I'm not sure that attached patch has a significant impact on performances. It seems like the speedup mostly comes from the print patch:
http://bugs.python.org/issue29296#msg285668

But well, it is still interesting to use METH_FASTCALL in decimal ;-)

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29301>
_______________________________________

From report at bugs.python.org  Tue Jan 17 12:12:19 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Tue, 17 Jan 2017 17:12:19 +0000
Subject: [issue29301] decimal: Use FASTCALL and/or Argument Clinic
In-Reply-To: <1484671124.21.0.493200110844.issue29301@psf.upfronthosting.co.za>
Message-ID: <1484673139.2.0.781265818516.issue29301@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

See msg207652 in issue20177.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29301>
_______________________________________

From report at bugs.python.org  Tue Jan 17 12:15:21 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Tue, 17 Jan 2017 17:15:21 +0000
Subject: [issue29296] convert print() to METH_FASTCALL
In-Reply-To: <1484652871.69.0.785445814249.issue29296@psf.upfronthosting.co.za>
Message-ID: <1484673321.19.0.517246517438.issue29296@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

Please wait until the support of var-positional arguments be added in Argument Clinic. After that print() could be just converted to Argument Clinic.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29296>
_______________________________________

From report at bugs.python.org  Tue Jan 17 12:29:46 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Tue, 17 Jan 2017 17:29:46 +0000
Subject: [issue29301] decimal: Use FASTCALL and/or Argument Clinic
In-Reply-To: <1484671124.21.0.493200110844.issue29301@psf.upfronthosting.co.za>
Message-ID: <1484674186.86.0.31400832183.issue29301@psf.upfronthosting.co.za>


Raymond Hettinger added the comment:

This should not go in without Stephan Krah's approval.  This code is finely tuned and carefully arranged.

----------
assignee:  -> skrah
nosy: +rhettinger

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29301>
_______________________________________

From report at bugs.python.org  Tue Jan 17 13:08:30 2017
From: report at bugs.python.org (INADA Naoki)
Date: Tue, 17 Jan 2017 18:08:30 +0000
Subject: [issue29259] Add tp_fastcall to PyTypeObject: support FASTCALL
 calling convention for all callable objects
In-Reply-To: <1484310274.99.0.832491900546.issue29259@psf.upfronthosting.co.za>
Message-ID: <1484676510.94.0.916196331348.issue29259@psf.upfronthosting.co.za>


INADA Naoki added the comment:

> Maybe it's the expensive price of creating a temporary dictionary?

Please see the patch on issue29295.
PyDict_GetItemString() is bad part, too.
It's create str object, calc hash, lookup dict, and dealloc the str object.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29259>
_______________________________________

From report at bugs.python.org  Tue Jan 17 13:36:41 2017
From: report at bugs.python.org (John Taylor)
Date: Tue, 17 Jan 2017 18:36:41 +0000
Subject: [issue29285] Unicode errors occur inside of multi-line comments
In-Reply-To: <1484586400.61.0.654989234385.issue29285@psf.upfronthosting.co.za>
Message-ID: <1484678201.68.0.151922957908.issue29285@psf.upfronthosting.co.za>


John Taylor added the comment:

OP here, thanks for replying to this.  I used Zach's suggestion of placing an 'r' in front of triple-quotes.  This accomplishes my goal.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29285>
_______________________________________

From report at bugs.python.org  Tue Jan 17 15:04:23 2017
From: report at bugs.python.org (Robert Collins)
Date: Tue, 17 Jan 2017 20:04:23 +0000
Subject: [issue9216] FIPS support for hashlib
In-Reply-To: <1278721335.16.0.522410247151.issue9216@psf.upfronthosting.co.za>
Message-ID: <1484683463.1.0.06360604555.issue9216@psf.upfronthosting.co.za>


Robert Collins added the comment:

@doug - I don't see how a separate fips module *wouldn't* solve it:
 - code that uses md5 in security contexts wouldn't be able to call it from the fips module, which is the needed outcome
 - code that uses md5 and isn't fips compliant would be importing from the non-fips module, and thats as auditable as looking for a 'usedforsecurity=False' flag
 - auditors can assume that code that doesn't use the fips module


And its way less messy: remember we're going to have this flag passed to every hashlib invocation from every project in order to *opt out* of the FIPS restrictions. Because, over time, FIPS will change, so noone can assume that any given function is and will remain FIPS compatible: and this flag is going to percolate up into e.g. the HMAC module.

I think thats pretty ugly: want to calculate the sha of a blob to look it up in git? sha1sum(file.read(), usedforsecurity=False)

Separately I wonder about the impact on higher layers - are they ready to be parameterised by objects, or do they look things up by name - and thus need to start accepting this new parameter and passing it down?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue9216>
_______________________________________

From report at bugs.python.org  Tue Jan 17 15:19:00 2017
From: report at bugs.python.org (Antoine Pitrou)
Date: Tue, 17 Jan 2017 20:19:00 +0000
Subject: [issue9216] FIPS support for hashlib
In-Reply-To: <1278721335.16.0.522410247151.issue9216@psf.upfronthosting.co.za>
Message-ID: <1484684340.8.0.616410570777.issue9216@psf.upfronthosting.co.za>


Antoine Pitrou added the comment:

The separate module idea is an interesting one, though I wonder if it aligns with users' goals.  Perhaps some users simply want to set the OPENSSL_FORCE_FIPS_MODE environment variable and then run existing Python code with it to ensure that code is FIPS-compliant.  A separate module assumes that the developer is the one who makes the decision of running in FIPS compliance mode or not.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue9216>
_______________________________________

From report at bugs.python.org  Tue Jan 17 15:26:40 2017
From: report at bugs.python.org (Ian Cordasco)
Date: Tue, 17 Jan 2017 20:26:40 +0000
Subject: [issue9216] FIPS support for hashlib
In-Reply-To: <1278721335.16.0.522410247151.issue9216@psf.upfronthosting.co.za>
Message-ID: <1484684800.72.0.185947775792.issue9216@psf.upfronthosting.co.za>


Ian Cordasco added the comment:

So I see the argument on both sides of this discussion. Having those optional arguments for all the functions seems like an obvious blocker. If a submodule is a blocker, what if we provide a context-manager to signal this?

----------
nosy: +icordasc

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue9216>
_______________________________________

From report at bugs.python.org  Tue Jan 17 15:29:47 2017
From: report at bugs.python.org (Antoine Pitrou)
Date: Tue, 17 Jan 2017 20:29:47 +0000
Subject: [issue9216] FIPS support for hashlib
In-Reply-To: <1278721335.16.0.522410247151.issue9216@psf.upfronthosting.co.za>
Message-ID: <1484684987.0.0.485135592027.issue9216@psf.upfronthosting.co.za>


Antoine Pitrou added the comment:

AFAICT from David's patch, there isn't a new argument in all hashlib functions but only in the digest constructors.  Someone might want to correct me.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue9216>
_______________________________________

From report at bugs.python.org  Tue Jan 17 15:42:06 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 17 Jan 2017 20:42:06 +0000
Subject: [issue29301] decimal: Use FASTCALL and/or Argument Clinic
In-Reply-To: <1484671124.21.0.493200110844.issue29301@psf.upfronthosting.co.za>
Message-ID: <1484685726.75.0.49339629128.issue29301@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Raymond Hettinger: "This should not go in without Stephan Krah's approval.  This code is finely tuned and carefully arrang"

Sure, that's why I opened this issue.


Serhiy Storchaka: "See msg207652 in issue20177."

Oh, I didn't know that Stefan Krah already gave his opinion.

Since the performance impact is unclear, I prefer to close this issue. I will come back if argument parsing is more clearly identified as a performance bottleneck.

----------
resolution:  -> rejected
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29301>
_______________________________________

From report at bugs.python.org  Tue Jan 17 15:45:33 2017
From: report at bugs.python.org (Yolanda)
Date: Tue, 17 Jan 2017 20:45:33 +0000
Subject: [issue9216] FIPS support for hashlib
In-Reply-To: <1278721335.16.0.522410247151.issue9216@psf.upfronthosting.co.za>
Message-ID: <1484685933.9.0.0466080231111.issue9216@psf.upfronthosting.co.za>


Yolanda added the comment:

@rbtcollins, so you mean the apps using it, shall be "fips aware" ? That will be the point of your separate module?
So...

if fips_enabled then
    use fips.md5
else
    use normal.md5

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue9216>
_______________________________________

From report at bugs.python.org  Tue Jan 17 16:02:46 2017
From: report at bugs.python.org (Mark Dickinson)
Date: Tue, 17 Jan 2017 21:02:46 +0000
Subject: [issue29282] Fused multiply-add: proposal to add math.fma()
In-Reply-To: <1484560897.19.0.0955404368118.issue29282@psf.upfronthosting.co.za>
Message-ID: <1484686966.76.0.501675235194.issue29282@psf.upfronthosting.co.za>


Mark Dickinson added the comment:

And here's a patch.

----------
keywords: +patch
Added file: http://bugs.python.org/file46321/math_fma.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29282>
_______________________________________

From report at bugs.python.org  Tue Jan 17 16:07:29 2017
From: report at bugs.python.org (Doug Hellmann)
Date: Tue, 17 Jan 2017 21:07:29 +0000
Subject: [issue9216] FIPS support for hashlib
In-Reply-To: <1278721335.16.0.522410247151.issue9216@psf.upfronthosting.co.za>
Message-ID: <1484687249.82.0.474504731641.issue9216@psf.upfronthosting.co.za>


Doug Hellmann added the comment:

@Robert, I thought you were proposing a hashlib.fips module that did not include md5() at all. If it does include the function, and the function does whatever is needed to disable the "die when using MD5" on a FIPS system, then I agree it would work. 

Your point about the FIPS standard changing and needing to include more hash types in the future is good.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue9216>
_______________________________________

From report at bugs.python.org  Tue Jan 17 16:08:42 2017
From: report at bugs.python.org (Doug Hellmann)
Date: Tue, 17 Jan 2017 21:08:42 +0000
Subject: [issue9216] FIPS support for hashlib
In-Reply-To: <1278721335.16.0.522410247151.issue9216@psf.upfronthosting.co.za>
Message-ID: <1484687322.53.0.85131532687.issue9216@psf.upfronthosting.co.za>


Doug Hellmann added the comment:

@Antoine - The idea behind introducing some API mechanism is exactly as you say, to let the developer say "this use of this algorithm is not related to security" to tell FIPS systems to not be pedantic.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue9216>
_______________________________________

From report at bugs.python.org  Tue Jan 17 16:15:21 2017
From: report at bugs.python.org (Mark Dickinson)
Date: Tue, 17 Jan 2017 21:15:21 +0000
Subject: [issue29282] Fused multiply-add: proposal to add math.fma()
In-Reply-To: <1484560897.19.0.0955404368118.issue29282@psf.upfronthosting.co.za>
Message-ID: <1484687721.15.0.432212053092.issue29282@psf.upfronthosting.co.za>


Changes by Mark Dickinson <dickinsm at gmail.com>:


----------
stage:  -> patch review

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29282>
_______________________________________

From report at bugs.python.org  Tue Jan 17 16:47:28 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Tue, 17 Jan 2017 21:47:28 +0000
Subject: [issue29282] Fused multiply-add: proposal to add math.fma()
In-Reply-To: <1484560897.19.0.0955404368118.issue29282@psf.upfronthosting.co.za>
Message-ID: <1484689648.14.0.880691844917.issue29282@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

LGTM except that needed the versionadded directive and What's New entry. And maybe few additional tests.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29282>
_______________________________________

From report at bugs.python.org  Tue Jan 17 17:17:33 2017
From: report at bugs.python.org (Eric Lafontaine)
Date: Tue, 17 Jan 2017 22:17:33 +0000
Subject: [issue28879] smtplib send_message should add Date header if it is
 missing, per RFC5322
In-Reply-To: <1480952895.22.0.418354723664.issue28879@psf.upfronthosting.co.za>
Message-ID: <1484691453.79.0.802191986939.issue28879@psf.upfronthosting.co.za>


Eric Lafontaine added the comment:

Hi all,

The IETF didn't answer yet :(.  

I'll await your news regarding this patch ("issue_28879_V4.patch").

I would like to have feedback if I need to change something.

Thanks a lot in advance,
Eric Lafontaine

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28879>
_______________________________________

From report at bugs.python.org  Tue Jan 17 17:49:42 2017
From: report at bugs.python.org (Alexander Mohr)
Date: Tue, 17 Jan 2017 22:49:42 +0000
Subject: [issue29302] add contextlib.AsyncExitStack
Message-ID: <1484693382.32.0.211448130443.issue29302@psf.upfronthosting.co.za>


New submission from Alexander Mohr:

ExitStack is a really useful class and would be a great to have an async version.  I've gone ahead and created an implementation based on the existing Python 3.5.2 implementation.  Let me know what you guys think. I think it would be possible to combine most of the two classes together if you guys think it would be useful.  Let me know if I can/should create a github PR and where to do that.

----------
components: Library (Lib)
files: exit_stack.py
messages: 285687
nosy: thehesiod
priority: normal
severity: normal
status: open
title: add contextlib.AsyncExitStack
type: enhancement
versions: Python 3.6, Python 3.7
Added file: http://bugs.python.org/file46322/exit_stack.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29302>
_______________________________________

From report at bugs.python.org  Tue Jan 17 18:26:15 2017
From: report at bugs.python.org (anonymous2017)
Date: Tue, 17 Jan 2017 23:26:15 +0000
Subject: [issue29303] asyncio.Lock.acquire() does not always yield
Message-ID: <1484695575.39.0.807179616569.issue29303@psf.upfronthosting.co.za>


New submission from anonymous2017:

Details here: http://stackoverflow.com/questions/41708609/unfair-scheduling-bad-lock-optimization-in-asyncio-event-loop

Essentially there should be a `yield` before this line otherwise a coroutine that only acquires and releases a lock will never yield to other coroutines.

----------
components: asyncio
messages: 285688
nosy: anonymous2017, gvanrossum, yselivanov
priority: normal
severity: normal
status: open
title: asyncio.Lock.acquire() does not always yield
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29303>
_______________________________________

From report at bugs.python.org  Tue Jan 17 18:27:54 2017
From: report at bugs.python.org (Guido van Rossum)
Date: Tue, 17 Jan 2017 23:27:54 +0000
Subject: [issue29303] asyncio.Lock.acquire() does not always yield
In-Reply-To: <1484695575.39.0.807179616569.issue29303@psf.upfronthosting.co.za>
Message-ID: <1484695674.29.0.270618350866.issue29303@psf.upfronthosting.co.za>


Guido van Rossum added the comment:

Locks are not meant to have predictable behavior like that. They are meant to protect against concurrent access. If you want fairness you have to look elsewhere.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29303>
_______________________________________

From report at bugs.python.org  Tue Jan 17 18:40:54 2017
From: report at bugs.python.org (anonymous2017)
Date: Tue, 17 Jan 2017 23:40:54 +0000
Subject: [issue29303] asyncio.Lock.acquire() does not always yield
In-Reply-To: <1484695575.39.0.807179616569.issue29303@psf.upfronthosting.co.za>
Message-ID: <1484696454.51.0.148130613513.issue29303@psf.upfronthosting.co.za>


anonymous2017 added the comment:

I may have mis-characterized the original report. Rephrasing just to make sure it is correctly addressed:

First, it is not about fairness. Sorry for the original characterization. I was trying to understand what was happening.

Second, the precise issue is whether a `yield from` can be relied upon for cooperative multitasking. Should other co-routines should be prevented from running or delayed if one routine is running and acquiring and releasing a lock using `yield from`.

It seemed at first sight that `yield from <anything>` should give the scheduler a chance to consider other routines. But the lock code above and including line #171 does not give control to the scheduler in a special case: https://github.com/python/cpython/blob/master/Lib/asyncio/locks.py#L171

Code Sample 1 (shows b() being delayed until a() is fully complete despite a() yielding many times)
=============
import asyncio

lock = asyncio.Lock()

def a ():
 yield from lock.acquire()
 for i in range(10):
  print('b: ' + str(i))
  if i % 2 == 0:
   lock.release()
   yield from lock.acquire()
 lock.release()

async def b ():
 print('hello')

asyncio.get_event_loop().run_until_complete(asyncio.gather(a(), b()))

print('done')

Code Sample 2
=============
(shows interleaving if an additional INITIAL yield from asyncio.sleep is inserted; removing the initial sleep removes the interleaving)

import asyncio

lock = asyncio.Lock()

def a ():
 yield from lock.acquire()
 yield from asyncio.sleep(0)
 for i in range(10):
  print('a: ' + str(i))
  if i % 2 == 0:
   lock.release()
   yield from lock.acquire()
 lock.release()

def b ():
 yield from lock.acquire()
 yield from asyncio.sleep(0)
 for i in range(10):
  print('b: ' + str(i))
  if i % 2 == 0:
   lock.release()
   yield from lock.acquire()
 lock.release()

asyncio.get_event_loop().run_until_complete(asyncio.gather(a(), b()))

print('done')


Thank you for your kind consideration.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29303>
_______________________________________

From report at bugs.python.org  Tue Jan 17 18:59:48 2017
From: report at bugs.python.org (Guido van Rossum)
Date: Tue, 17 Jan 2017 23:59:48 +0000
Subject: [issue29303] asyncio.Lock.acquire() does not always yield
In-Reply-To: <1484695575.39.0.807179616569.issue29303@psf.upfronthosting.co.za>
Message-ID: <1484697588.8.0.357792209623.issue29303@psf.upfronthosting.co.za>


Guido van Rossum added the comment:

No, `yield from` (or, in Python 3.5+, `await`) is not meant to bound back to the scheduler. If the target is a coroutine that itself doesn't yield, it is a *feature* that the scheduler is bypassed.

If you want to force a trip through the scheduler, use `asyncio.sleep(0)`.

----------
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29303>
_______________________________________

From report at bugs.python.org  Tue Jan 17 19:59:35 2017
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 18 Jan 2017 00:59:35 +0000
Subject: [issue29296] convert print() to METH_FASTCALL
In-Reply-To: <1484652871.69.0.785445814249.issue29296@psf.upfronthosting.co.za>
Message-ID: <1484701175.8.0.00676993961074.issue29296@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Serhiy Storchaka: "Please wait until the support of var-positional arguments be added in Argument Clinic. After that print() could be just converted to Argument Clinic."

Why can't we start with METH_FASTCALL and convert to AC later? It seems like print-fastcall.patch will be a major performance enhancement (at least for the very specific bm_telco benchmark), once tp_fastcall field will be added.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29296>
_______________________________________

From report at bugs.python.org  Tue Jan 17 20:25:16 2017
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 18 Jan 2017 01:25:16 +0000
Subject: [issue29259] Add tp_fastcall to PyTypeObject: support FASTCALL
 calling convention for all callable objects
In-Reply-To: <1484310274.99.0.832491900546.issue29259@psf.upfronthosting.co.za>
Message-ID: <1484702716.13.0.663890274902.issue29259@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Ok, I fixed all remaining issues. The new patch version 3 is now waiting for your review ;-)


Patch version 3:

* Add Py_TPFLAGS_HAVE_FINALIZE and Py_TPFLAGS_HAVE_FASTCALL to Py_TPFLAGS_DEFAULT
* Don't read or write tp_fastcall if the type doesn't have the Py_TPFLAGS_HAVE_FASTCALL feature
* fastcall_wrapper() looks into base classes to find a tp_fastcall field: use the first base class with the Py_TPFLAGS_HAVE_FASTCALL feature
* Fix how tp_call and tp_fastcall are inherited
* Use designated initializers for  tp_fastcall, as asked by Naoki (and discussed on python-dev)
* Rebase the change and revert now uselesss or unwanted changes
* No more FIXME ;-)

About the stable ABI: since the version 3, it should just work thanks to the change on Py_TPFLAGS_DEFAULT. Code compiled with Python 3.7 benefit directly of tp_fastcall. Code compiled with Python 3.6 or older will get fastcall_wrapper() which finds tp_fastcall in base classes.

----------
Added file: http://bugs.python.org/file46323/tp_fastcall-3.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29259>
_______________________________________

From report at bugs.python.org  Tue Jan 17 20:56:12 2017
From: report at bugs.python.org (Eric Lafontaine)
Date: Wed, 18 Jan 2017 01:56:12 +0000
Subject: [issue28879] smtplib send_message should add Date header if it is
 missing, per RFC5322
In-Reply-To: <1480952895.22.0.418354723664.issue28879@psf.upfronthosting.co.za>
Message-ID: <1484704572.71.0.798975397356.issue28879@psf.upfronthosting.co.za>


Eric Lafontaine added the comment:

Hi all,

I've received an answer from the IETF Pete Resnick.  The answer does say however that there is no guaranteed way of getting the right headers if the message doesn't respect the standard; 

"[...] 
In this case, if the trace fields were not present, you would not be able to clearly distinguish. Certainly you know which blocks the Resent-Date and Resent-From belong to, and in this case you know which blocks the Resent-Message-ID belong to (since there are two of them and only two Resent-* blocks), and you know which block the Resent-Bcc belongs to (because it comes between the first Resent-Date and Resent-Message-ID), but you can't tell which block the Resent-To: belongs to. And if the either Resent-Message-ID was missing, you would be unable to tell where the Resent-Bcc or Resent-Message-ID belongs. This is simply a weakness in the standard.

However, the trace fields should exist, and that should divide the Resent-* blocks. As it says in 3.6:

header fields SHOULD NOT be reordered when a message is transported
or transformed. More importantly, the trace header fields and resent
header fields MUST NOT be reordered, and SHOULD be kept in blocks
prepended to the message.

I hope that helps.


pr
-- 
Pete Resnick http://www.qualcomm.com/~presnick/
Qualcomm Technologies, Inc. - +1 (858)651-4478
"
______________

In other words, if we were to take an e-mail that would have more than one Resent- headers, there should be traces in between the resent block.  If there are no traces for a Resent- block, and we can detect there are 2 blocks in what is supposed to be a block , we should raise an error.  

However, if we were to do the implementation and make it to detect "blocks" of resents (with the position of the resent for example).  We could use the first block as the one we should be using and see if there are 2 Resent-Dates or 2 Resent-From in that block only.

What do you think David?  I would like to know :).

P.s. I can forward the e-mail to does who want to have it :).

Regards,
Eric Lafontaine

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28879>
_______________________________________

From report at bugs.python.org  Tue Jan 17 21:36:13 2017
From: report at bugs.python.org (INADA Naoki)
Date: Wed, 18 Jan 2017 02:36:13 +0000
Subject: [issue29304] dict: simplify lookup function
Message-ID: <1484706973.14.0.775713839146.issue29304@psf.upfronthosting.co.za>


New submission from INADA Naoki:

All lookdict* functions are implemented like pseudo language:

```
lookup()
if not collision:
    return result

while True:
    perturb_shift()
    lookup()
    if not collision:
        return result
```

This patch changes it as:

```
while True:
    lookup()
    if not collision:
        return result
    perturb_shift()
```

It removes 100 lines of code. Good. But how about performance?

When this patch is applied to 4a534c45bbf6:

```
$ ../python.patched -m perf compare_to default.json patched2.json -G --min-speed=2
Slower (4):
- xml_etree_generate: 271 ms +- 6 ms -> 283 ms +- 9 ms: 1.04x slower (+4%)
- nqueens: 263 ms +- 4 ms -> 272 ms +- 3 ms: 1.04x slower (+4%)
- scimark_monte_carlo: 272 ms +- 10 ms -> 280 ms +- 14 ms: 1.03x slower (+3%)
- scimark_lu: 435 ms +- 23 ms -> 446 ms +- 32 ms: 1.03x slower (+3%)

Faster (7):
- call_method: 15.2 ms +- 0.2 ms -> 14.7 ms +- 0.4 ms: 1.04x faster (-4%)
- call_simple: 14.4 ms +- 0.2 ms -> 13.9 ms +- 0.3 ms: 1.04x faster (-4%)
- xml_etree_iterparse: 227 ms +- 9 ms -> 219 ms +- 7 ms: 1.04x faster (-3%)
- scimark_sor: 527 ms +- 10 ms -> 510 ms +- 11 ms: 1.03x faster (-3%)
- call_method_slots: 14.7 ms +- 0.5 ms -> 14.3 ms +- 0.2 ms: 1.03x faster (-3%)
- genshi_text: 90.2 ms +- 1.1 ms -> 87.8 ms +- 1.1 ms: 1.03x faster (-3%)
- django_template: 403 ms +- 5 ms -> 394 ms +- 4 ms: 1.02x faster (-2%)

Benchmark hidden because not significant (53): 2to3, ...
```

And when this patch applied to 1a97b10cb420 :

```
$ ../python.patched -m perf compare_to default.json patched.json -G --min-speed=2                                                                                                               
Slower (6):
- call_simple: 13.5 ms +- 0.5 ms -> 14.4 ms +- 0.4 ms: 1.07x slower (+7%)
- xml_etree_generate: 270 ms +- 6 ms -> 287 ms +- 5 ms: 1.06x slower (+6%)
- xml_etree_process: 240 ms +- 6 ms -> 247 ms +- 4 ms: 1.03x slower (+3%)
- regex_compile: 429 ms +- 3 ms -> 440 ms +- 5 ms: 1.03x slower (+3%)
- call_method_unknown: 16.1 ms +- 0.2 ms -> 16.5 ms +- 0.3 ms: 1.02x slower (+2%)
- logging_simple: 31.2 us +- 0.4 us -> 32.0 us +- 0.3 us: 1.02x slower (+2%)

Faster (8):
- genshi_text: 90.6 ms +- 1.4 ms -> 87.6 ms +- 1.2 ms: 1.03x faster (-3%)
- scimark_sor: 513 ms +- 11 ms -> 497 ms +- 12 ms: 1.03x faster (-3%)
- genshi_xml: 200 ms +- 2 ms -> 194 ms +- 2 ms: 1.03x faster (-3%)
- unpickle_pure_python: 857 us +- 21 us -> 835 us +- 13 us: 1.03x faster (-3%)
- python_startup_no_site: 9.95 ms +- 0.02 ms -> 9.74 ms +- 0.02 ms: 1.02x faster (-2%)
- json_dumps: 29.7 ms +- 0.4 ms -> 29.1 ms +- 0.4 ms: 1.02x faster (-2%)
- xml_etree_iterparse: 225 ms +- 9 ms -> 220 ms +- 5 ms: 1.02x faster (-2%)
- chameleon: 31.1 ms +- 0.3 ms -> 30.5 ms +- 0.5 ms: 1.02x faster (-2%)

Benchmark hidden because not significant (50): 2to3, ...
```

I can't see any stable and significant performance regression.
I'll try to create some micro benchmarks.

----------
files: dictlook-refactoring.patch
keywords: patch
messages: 285695
nosy: inada.naoki
priority: normal
severity: normal
status: open
title: dict: simplify lookup function
Added file: http://bugs.python.org/file46324/dictlook-refactoring.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29304>
_______________________________________

From report at bugs.python.org  Tue Jan 17 22:26:45 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Wed, 18 Jan 2017 03:26:45 +0000
Subject: [issue29304] dict: simplify lookup function
In-Reply-To: <1484706973.14.0.775713839146.issue29304@psf.upfronthosting.co.za>
Message-ID: <1484710005.27.0.629200015735.issue29304@psf.upfronthosting.co.za>


Xiang Zhang added the comment:

Ahh, I got the same idea before. But then I persuaded myself that the first "lookup()" was deliberately extracted from the loop since it's highly possible there is no collision and you can hit the result (empty or not) the first time.

----------
nosy: +xiang.zhang

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29304>
_______________________________________

From report at bugs.python.org  Tue Jan 17 22:39:55 2017
From: report at bugs.python.org (R. David Murray)
Date: Wed, 18 Jan 2017 03:39:55 +0000
Subject: [issue28879] smtplib send_message should add Date header if it is
 missing, per RFC5322
In-Reply-To: <1480952895.22.0.418354723664.issue28879@psf.upfronthosting.co.za>
Message-ID: <1484710795.57.0.367806584438.issue28879@psf.upfronthosting.co.za>


R. David Murray added the comment:

I think this constitutes the heuristic I was talking about in that comment, that will get it right 99+% of the time.  Strict mode should raise an error, but strict is not the default in the email package.

I probably won't have time to do any review for a while yet, I'm afraid.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28879>
_______________________________________

From report at bugs.python.org  Tue Jan 17 22:54:41 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Wed, 18 Jan 2017 03:54:41 +0000
Subject: [issue29304] dict: simplify lookup function
In-Reply-To: <1484706973.14.0.775713839146.issue29304@psf.upfronthosting.co.za>
Message-ID: <1484711681.69.0.451669061629.issue29304@psf.upfronthosting.co.za>


Raymond Hettinger added the comment:

I worry you guys are rapidly sloshing around and churning code that was developed slowly and methodically over many years.  A lot of people have looked at an tweaked the dictionary code, but now it seems like there is a rewrite-everything festival.  When looking at potential changes, please consider that the people who came before you knew what they were doing.  If there is a pre-loop step that could have been folded into a single loop to save a little code, it isn't there because the people who wrote it are daft.  That expansion was made for a reason.

Even if you get an improved timing here or there, consider that it may vary from compiler to compiler, from os to os, and may vary widely across different kinds of applications.  Branch prediction and cache effect issues don't readily show up in simple timing loops.

Particularly as a new core dev, I recommend resisting the urge to rewrite everything you touch.  That will only destabilize Python.  In the past couple of years, two or three devs have churned an enormous amount of code.

----------
nosy: +rhettinger

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29304>
_______________________________________

From report at bugs.python.org  Tue Jan 17 23:46:59 2017
From: report at bugs.python.org (Roundup Robot)
Date: Wed, 18 Jan 2017 04:46:59 +0000
Subject: [issue28556] typing.py upgrades
In-Reply-To: <1477756009.46.0.442279756181.issue28556@psf.upfronthosting.co.za>
Message-ID: <20170118044656.7844.57853.2CC7CE12@psf.io>


Roundup Robot added the comment:

New changeset 1a9e12a852b2 by Guido van Rossum in branch '3.5':
Issue #28556: merge 5 more typing changes from upstream (#340, #344, #348, #349, #350)
https://hg.python.org/cpython/rev/1a9e12a852b2

New changeset fd889884fe08 by Guido van Rossum in branch '3.6':
Issue #28556: merge 5 more typing changes from upstream (#340, #344, #348, #349, #350) (3.5->3.6)
https://hg.python.org/cpython/rev/fd889884fe08

New changeset 794dad4b849f by Guido van Rossum in branch 'default':
Issue #28556: merge 5 more typing changes from upstream (#340, #344, #348, #349, #350) (3.6->3.7)
https://hg.python.org/cpython/rev/794dad4b849f

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28556>
_______________________________________

From report at bugs.python.org  Tue Jan 17 23:48:54 2017
From: report at bugs.python.org (Guido van Rossum)
Date: Wed, 18 Jan 2017 04:48:54 +0000
Subject: [issue29246] typing.Union raises RecursionError when comparing Union
 to other type
In-Reply-To: <1484175780.67.0.919413735976.issue29246@psf.upfronthosting.co.za>
Message-ID: <1484714934.4.0.610319937622.issue29246@psf.upfronthosting.co.za>


Guido van Rossum added the comment:

Fixed, see http://bugs.python.org/issue28556#msg281317

New changeset 75c7bc2c1ad8 by Guido van Rossum in branch '3.5':
Issue #28556: upstream improvements to docstrings and error messages by Ivan Levkivskyi (#331)
https://hg.python.org/cpython/rev/75c7bc2c1ad8

New changeset 294525aac5eb by Guido van Rossum in branch '3.6':
Issue #28556: upstream improvements to docstrings and error messages by Ivan Levkivskyi (#331) (3.5->3.6)
https://hg.python.org/cpython/rev/294525aac5eb

New changeset 30f154d9abf0 by Guido van Rossum in branch 'default':
Issue #28556: upstream improvements to docstrings and error messages by Ivan Levkivskyi (#331) (3.6->3.7)
https://hg.python.org/cpython/rev/30f154d9abf0

----------
resolution:  -> fixed
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29246>
_______________________________________

From report at bugs.python.org  Tue Jan 17 23:50:20 2017
From: report at bugs.python.org (Guido van Rossum)
Date: Wed, 18 Jan 2017 04:50:20 +0000
Subject: [issue28339] "TypeError: Parameterized generics cannot be used with
 class or instance checks" in test_functools after importing typing module
In-Reply-To: <1475421693.5.0.638809019729.issue28339@psf.upfronthosting.co.za>
Message-ID: <1484715020.35.0.896231025317.issue28339@psf.upfronthosting.co.za>


Guido van Rossum added the comment:

@levkivskyi Do you want to attempt a better fix in tome for 3.6.1?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28339>
_______________________________________

From report at bugs.python.org  Wed Jan 18 00:17:56 2017
From: report at bugs.python.org (Larry Hastings)
Date: Wed, 18 Jan 2017 05:17:56 +0000
Subject: [issue28339] "TypeError: Parameterized generics cannot be used with
 class or instance checks" in test_functools after importing typing module
In-Reply-To: <1475421693.5.0.638809019729.issue28339@psf.upfronthosting.co.za>
Message-ID: <1484716676.34.0.646302733619.issue28339@psf.upfronthosting.co.za>


Changes by Larry Hastings <larry at hastings.org>:


----------
nosy:  -larry

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28339>
_______________________________________

From report at bugs.python.org  Wed Jan 18 00:42:22 2017
From: report at bugs.python.org (INADA Naoki)
Date: Wed, 18 Jan 2017 05:42:22 +0000
Subject: [issue29304] dict: simplify lookup function
In-Reply-To: <1484706973.14.0.775713839146.issue29304@psf.upfronthosting.co.za>
Message-ID: <1484718142.72.0.36341437894.issue29304@psf.upfronthosting.co.za>


INADA Naoki added the comment:

Raymond, I understand your worries.  I won't commit this until I do more precise survey.

But why I trying this is not only I find duplicated code.
I think benefit of this technique is reduced by historical reason.


In Python 2.7, there are only two lookup functions: lookdict and lookdict_string.
Both functions support dict containing dummy entry.

In the first comparing part in the lookdict_string():

    if (ep->me_key == NULL || ep->me_key == key)
        return ep;
    if (ep->me_key == dummy)
        freeslot = ep;
    else {
        if (ep->me_hash == hash && _PyString_Eq(ep->me_key, key))
            return ep;
        freeslot = NULL;
    }

And similar code in same function but in loop:

        if (ep->me_key == NULL)
            return freeslot == NULL ? ep : freeslot;
        if (ep->me_key == key
            || (ep->me_hash == hash
            && ep->me_key != dummy
            && _PyString_Eq(ep->me_key, key)))
            return ep;
        if (ep->me_key == dummy && freeslot == NULL)
            freeslot = ep;

First part can ignore freeslot variable is NULL or not.  But second lookup can't.
Since most lookup is done at first try, this technique may had significant effect.


But for now, we're having five lookdict functions (if we include lookdict_index).

Three of them (lookdict_unicode_nodummy, lookdict_split, and lookdict_index) cares only dict without dummies.
They don't have freeslot variable.  So first part and second part is almost same.
Performance benefit from this technique may be negligible.

About remaining two functions (lookdict_unicode and lookdict), this technique may have still effect.
But performance of them are not so important now, compared with Python 2.7.
They were used for all name lookup in Python 2.7, but they are fallback of lookdict_split and lookdict_unicode_nodummy now.

On the other hand, having 2.5x (2 -> 5) lookdict function means maintenance cost of duplicated code is increased.


At first, I'll start three functions which doesn't take care of dummies.
I think there are almost zero performance regression.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29304>
_______________________________________

From report at bugs.python.org  Wed Jan 18 01:06:31 2017
From: report at bugs.python.org (INADA Naoki)
Date: Wed, 18 Jan 2017 06:06:31 +0000
Subject: [issue29304] dict: simplify lookup function
In-Reply-To: <1484706973.14.0.775713839146.issue29304@psf.upfronthosting.co.za>
Message-ID: <1484719591.54.0.839460381801.issue29304@psf.upfronthosting.co.za>


Changes by INADA Naoki <songofacandy at gmail.com>:


Removed file: http://bugs.python.org/file46324/dictlook-refactoring.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29304>
_______________________________________

From report at bugs.python.org  Wed Jan 18 01:09:20 2017
From: report at bugs.python.org (INADA Naoki)
Date: Wed, 18 Jan 2017 06:09:20 +0000
Subject: [issue29304] dict: simplify lookup function
In-Reply-To: <1484706973.14.0.775713839146.issue29304@psf.upfronthosting.co.za>
Message-ID: <1484719760.48.0.32964031209.issue29304@psf.upfronthosting.co.za>


INADA Naoki added the comment:

I've attached wrong file.  This patch is first version I wanted to post.
It dedupe all five functions.

----------
Added file: http://bugs.python.org/file46325/dict-refactor.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29304>
_______________________________________

From report at bugs.python.org  Wed Jan 18 01:22:28 2017
From: report at bugs.python.org (INADA Naoki)
Date: Wed, 18 Jan 2017 06:22:28 +0000
Subject: [issue29259] Add tp_fastcall to PyTypeObject: support FASTCALL
 calling convention for all callable objects
In-Reply-To: <1484310274.99.0.832491900546.issue29259@psf.upfronthosting.co.za>
Message-ID: <1484720548.0.0.556540468482.issue29259@psf.upfronthosting.co.za>


INADA Naoki added the comment:

@haypo, Rietveld didn't accept your patch.  (I don't know what format Rietveld accepts...)

Is PR 65 is same to the third patch?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29259>
_______________________________________

From report at bugs.python.org  Wed Jan 18 02:47:24 2017
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 18 Jan 2017 07:47:24 +0000
Subject: [issue29259] Add tp_fastcall to PyTypeObject: support FASTCALL
 calling convention for all callable objects
In-Reply-To: <1484310274.99.0.832491900546.issue29259@psf.upfronthosting.co.za>
Message-ID: <1484725644.37.0.956594667748.issue29259@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Oops, I had a local commit for testing purpose. I regenerated the patch: tp_fastcall-4.patch.

> Is PR 65 is same to the third patch?

Yes, patches are generated from my Git pull request:
https://github.com/python/cpython/pull/65

The Git history became a little bit more ugly after many changes :-) I should try to rework it to have better changes. I will likely do that before pushing changes once the change will be accepted.

----------
Added file: http://bugs.python.org/file46326/tp_fastcall-4.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29259>
_______________________________________

From report at bugs.python.org  Wed Jan 18 03:24:07 2017
From: report at bugs.python.org (Yolanda)
Date: Wed, 18 Jan 2017 08:24:07 +0000
Subject: [issue9216] FIPS support for hashlib
In-Reply-To: <1278721335.16.0.522410247151.issue9216@psf.upfronthosting.co.za>
Message-ID: <1484727847.96.0.653110074013.issue9216@psf.upfronthosting.co.za>


Yolanda added the comment:

@rbtcollins, even if we go with a FIPS aware module, we'd still need to detect if md5 was used for security purposes.
If we build a system that detects FIPS enablement, call md5 say ... for generating a password, and then the python fips_md5 call is masking it, we'd be breaking FIPS rules.
I still see the point of the used_for_security flag. Maybe reverting the flag, set used_for_security to False because the normal usage of md5 shall be for hashes and non security stuff?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue9216>
_______________________________________

From report at bugs.python.org  Wed Jan 18 03:41:01 2017
From: report at bugs.python.org (Babe Hardy)
Date: Wed, 18 Jan 2017 08:41:01 +0000
Subject: [issue29305] encoding to ascii in client.py
Message-ID: <1484728861.39.0.489354661051.issue29305@psf.upfronthosting.co.za>


New submission from Babe Hardy:

used urlopen('...{}'.format(v)).read() with v being a string
when v is of a non-ascii format:
>>UnicodeEncodeError: 'ascii' codec can't encode characters<<
in line 984 of putrequest

after changing
>>self._output(request.encode('ascii'))<<
into
>>self._output(request.encode('utf-8'))<<

my script worked again

----------
components: Library (Lib)
messages: 285707
nosy: Babe Hardy
priority: normal
severity: normal
status: open
title: encoding to ascii in client.py
type: compile error
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29305>
_______________________________________

From report at bugs.python.org  Wed Jan 18 03:44:17 2017
From: report at bugs.python.org (INADA Naoki)
Date: Wed, 18 Jan 2017 08:44:17 +0000
Subject: [issue29259] Add tp_fastcall to PyTypeObject: support FASTCALL
 calling convention for all callable objects
In-Reply-To: <1484310274.99.0.832491900546.issue29259@psf.upfronthosting.co.za>
Message-ID: <1484729057.85.0.753581365159.issue29259@psf.upfronthosting.co.za>


INADA Naoki added the comment:

I commented at the pull request.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29259>
_______________________________________

From report at bugs.python.org  Wed Jan 18 04:04:45 2017
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 18 Jan 2017 09:04:45 +0000
Subject: [issue29306] Check usage of Py_EnterRecursiveCall() and
 Py_LeaveRecursiveCall() in new FASTCALL functions
Message-ID: <1484730285.0.0.960009012423.issue29306@psf.upfronthosting.co.za>


New submission from STINNER Victor:

I added the following new functions to Python 3.6:

* _PyObject_FastCallDict()
* _PyObject_FastCallKeywords()
* _PyFunction_FastCallDict()
* _PyFunction_FastCallKeywords()
* _PyCFunction_FastCallDict()
* _PyCFunction_FastCallKeywords()

I'm not sure that these functions update correctly the "recursion_depth" counter using Py_EnterRecursiveCall() and Py_LeaveRecursiveCall().

----------
components: Interpreter Core
messages: 285709
nosy: haypo, inada.naoki
priority: normal
severity: normal
status: open
title: Check usage of Py_EnterRecursiveCall() and Py_LeaveRecursiveCall() in new FASTCALL functions
type: behavior
versions: Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29306>
_______________________________________

From report at bugs.python.org  Wed Jan 18 04:32:59 2017
From: report at bugs.python.org (Roundup Robot)
Date: Wed, 18 Jan 2017 09:32:59 +0000
Subject: [issue29259] Add tp_fastcall to PyTypeObject: support FASTCALL
 calling convention for all callable objects
In-Reply-To: <1484310274.99.0.832491900546.issue29259@psf.upfronthosting.co.za>
Message-ID: <20170118093256.11319.66323.2B9D8468@psf.io>


Roundup Robot added the comment:

New changeset 261b108b9468 by Victor Stinner in branch 'default':
Remove unused func parameter of _PyStack_UnpackDict()
https://hg.python.org/cpython/rev/261b108b9468

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29259>
_______________________________________

From report at bugs.python.org  Wed Jan 18 04:39:51 2017
From: report at bugs.python.org (Roundup Robot)
Date: Wed, 18 Jan 2017 09:39:51 +0000
Subject: [issue29263] Implement LOAD_METHOD/CALL_METHOD for C functions
In-Reply-To: <1484323792.44.0.217164349535.issue29263@psf.upfronthosting.co.za>
Message-ID: <20170118093948.56913.70922.CB53961C@psf.io>


Roundup Robot added the comment:

New changeset 0b219348ec9e by Victor Stinner in branch 'default':
Optimize methoddescr_call(): avoid temporary PyCFunction
https://hg.python.org/cpython/rev/0b219348ec9e

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29263>
_______________________________________

From report at bugs.python.org  Wed Jan 18 04:39:51 2017
From: report at bugs.python.org (Roundup Robot)
Date: Wed, 18 Jan 2017 09:39:51 +0000
Subject: [issue29259] Add tp_fastcall to PyTypeObject: support FASTCALL
 calling convention for all callable objects
In-Reply-To: <1484310274.99.0.832491900546.issue29259@psf.upfronthosting.co.za>
Message-ID: <20170118093948.56913.64409.2F046534@psf.io>


Roundup Robot added the comment:

New changeset 0b219348ec9e by Victor Stinner in branch 'default':
Optimize methoddescr_call(): avoid temporary PyCFunction
https://hg.python.org/cpython/rev/0b219348ec9e

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29259>
_______________________________________

From report at bugs.python.org  Wed Jan 18 05:10:41 2017
From: report at bugs.python.org (INADA Naoki)
Date: Wed, 18 Jan 2017 10:10:41 +0000
Subject: [issue29306] Check usage of Py_EnterRecursiveCall() and
 Py_LeaveRecursiveCall() in new FASTCALL functions
In-Reply-To: <1484730285.0.0.960009012423.issue29306@psf.upfronthosting.co.za>
Message-ID: <1484734241.81.0.416772281293.issue29306@psf.upfronthosting.co.za>


INADA Naoki added the comment:

I think basic idea is "wrap unknown function call at least once, and preferably only once".

_PyEval_EvalFrameDefault() calls Py_EnterRecursiveCall("").
So wrapping PyFunction_(Fast)Call* with Py_EnterRecursiveCall() seems redundant.

On the other hand, PyCFunction may calls method of extension module,
and it can cause recursive call.
So I think PyCFunction_*Call* methods calling function pointer it has
(e.g. _PyCFunction_FastCallKeywords) should call Py_EnterRecursiveCall().

And caller of them can skip Py_EnterRecursiveCall().
For example, _PyObject_FastCallDict() may call _PyFunction_FastCallDict() or
_PyCFunction_FastCallDict(), or tp_fastcall (via _Py_RawFastCallDict) or tp_call.
It can wrap all, but wrapping only tp_fastcall and tp_call is better.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29306>
_______________________________________

From report at bugs.python.org  Wed Jan 18 05:28:05 2017
From: report at bugs.python.org (Roundup Robot)
Date: Wed, 18 Jan 2017 10:28:05 +0000
Subject: [issue27830] Add _PyObject_FastCallKeywords(): avoid the creation of
 a temporary dictionary for keyword arguments
In-Reply-To: <1471905864.17.0.99232719056.issue27830@psf.upfronthosting.co.za>
Message-ID: <20170118102802.7534.58349.89EF1A2A@psf.io>


Roundup Robot added the comment:

New changeset 1b7a5bdd05ed by Victor Stinner in branch 'default':
_PyObject_FastCallKeywords() now checks the result
https://hg.python.org/cpython/rev/1b7a5bdd05ed

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue27830>
_______________________________________

From report at bugs.python.org  Wed Jan 18 05:28:05 2017
From: report at bugs.python.org (Roundup Robot)
Date: Wed, 18 Jan 2017 10:28:05 +0000
Subject: [issue29259] Add tp_fastcall to PyTypeObject: support FASTCALL
 calling convention for all callable objects
In-Reply-To: <1484310274.99.0.832491900546.issue29259@psf.upfronthosting.co.za>
Message-ID: <20170118102802.7534.66839.EA6DF96D@psf.io>


Roundup Robot added the comment:

New changeset 1b7a5bdd05ed by Victor Stinner in branch 'default':
_PyObject_FastCallKeywords() now checks the result
https://hg.python.org/cpython/rev/1b7a5bdd05ed

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29259>
_______________________________________

From report at bugs.python.org  Wed Jan 18 05:56:23 2017
From: report at bugs.python.org (Jeroen Van Goey)
Date: Wed, 18 Jan 2017 10:56:23 +0000
Subject: [issue29307] ModuleNotFoundError when using literal string
 interpolation with invalid format specifier
Message-ID: <1484736983.43.0.107395329278.issue29307@psf.upfronthosting.co.za>


New submission from Jeroen Van Goey:

I installed Python 3.6 from J Fernyhough's PPA by doing

sudo add-apt-repository ppa:jonathonf/python-3.6
sudo apt-get update
sudo apt-get install python3.6

I made a string, using the new literal string interpolation, but I supplied an invalid format specifier. I not only got the expected "ValueError: Invalid format specifier", but also the unexpected "ModuleNotFoundError: No module named 'apt_pkg'".


$ python3.6
Python 3.6.0 (default, Dec 29 2016, 21:40:36) 
[GCC 5.4.1 20161202] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> value = 4 * 20
>>> f'the value is {value:%A}'
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: Invalid format specifier
Error in sys.excepthook:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/apport_python_hook.py", line 63, in apport_excepthook
    from apport.fileutils import likely_packaged, get_recent_crashes
  File "/usr/lib/python3/dist-packages/apport/__init__.py", line 5, in <module>
    from apport.report import Report
  File "/usr/lib/python3/dist-packages/apport/report.py", line 30, in <module>
    import apport.fileutils
  File "/usr/lib/python3/dist-packages/apport/fileutils.py", line 23, in <module>
    from apport.packaging_impl import impl as packaging
  File "/usr/lib/python3/dist-packages/apport/packaging_impl.py", line 23, in <module>
    import apt
  File "/usr/lib/python3/dist-packages/apt/__init__.py", line 23, in <module>
    import apt_pkg
ModuleNotFoundError: No module named 'apt_pkg'

Original exception was:
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: Invalid format specifier

----------
messages: 285716
nosy: jeroen-vangoey
priority: normal
severity: normal
status: open
title: ModuleNotFoundError when using literal string interpolation with invalid format specifier
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29307>
_______________________________________

From report at bugs.python.org  Wed Jan 18 06:14:27 2017
From: report at bugs.python.org (Martin Panter)
Date: Wed, 18 Jan 2017 11:14:27 +0000
Subject: [issue20559] urllib/http fail to sanitize a non-ascii url
In-Reply-To: <1391834064.13.0.416589140445.issue20559@psf.upfronthosting.co.za>
Message-ID: <1484738067.66.0.810845690291.issue20559@psf.upfronthosting.co.za>


Martin Panter added the comment:

See also Issue 3991 with proposals for handling non-ASCII as new features.

----------
nosy: +martin.panter

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue20559>
_______________________________________

From report at bugs.python.org  Wed Jan 18 06:19:17 2017
From: report at bugs.python.org (Ivan Levkivskyi)
Date: Wed, 18 Jan 2017 11:19:17 +0000
Subject: [issue28339] "TypeError: Parameterized generics cannot be used with
 class or instance checks" in test_functools after importing typing module
In-Reply-To: <1475421693.5.0.638809019729.issue28339@psf.upfronthosting.co.za>
Message-ID: <1484738357.65.0.077783918183.issue28339@psf.upfronthosting.co.za>


Ivan Levkivskyi added the comment:

The problem is the only good way to fix this I see now is patching two lines in functools and abc using the public ``typing_inspect`` API proposed in http://bugs.python.org/issue29262 (I added it as a dependency). The latter could take some time, so that I am not 100% sure about 3.6.1.

----------
dependencies: +Provide a way to check for *real* typing.Union instances
type:  -> behavior

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28339>
_______________________________________

From report at bugs.python.org  Wed Jan 18 06:25:50 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Wed, 18 Jan 2017 11:25:50 +0000
Subject: [issue29307] ModuleNotFoundError when using literal string
 interpolation with invalid format specifier
In-Reply-To: <1484736983.43.0.107395329278.issue29307@psf.upfronthosting.co.za>
Message-ID: <1484738750.17.0.232908967254.issue29307@psf.upfronthosting.co.za>


Xiang Zhang added the comment:

It seems to be vendor's issue not CPython itself. This same issue also happens in Ubuntu 16.10's Python 3.6. Raise any exception can cause this:

Python 3.6.0b2 (default, Oct 11 2016, 05:27:10) 
[GCC 6.2.0 20161005] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> raise Exception
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
Exception
Error in sys.excepthook:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/apport_python_hook.py", line 63, in apport_excepthook
    from apport.fileutils import likely_packaged, get_recent_crashes
  File "/usr/lib/python3/dist-packages/apport/__init__.py", line 5, in <module>
    from apport.report import Report
  File "/usr/lib/python3/dist-packages/apport/report.py", line 30, in <module>
    import apport.fileutils
  File "/usr/lib/python3/dist-packages/apport/fileutils.py", line 23, in <module>
    from apport.packaging_impl import impl as packaging
  File "/usr/lib/python3/dist-packages/apport/packaging_impl.py", line 23, in <module>
    import apt
  File "/usr/lib/python3/dist-packages/apt/__init__.py", line 23, in <module>
    import apt_pkg
ModuleNotFoundError: No module named 'apt_pkg'

Original exception was:
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
Exception
>>> 

Also see https://bugs.launchpad.net/ubuntu/+source/python3.6/+bug/1631367.

----------
nosy: +doko, xiang.zhang

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29307>
_______________________________________

From report at bugs.python.org  Wed Jan 18 06:32:14 2017
From: report at bugs.python.org (Armin Rigo)
Date: Wed, 18 Jan 2017 11:32:14 +0000
Subject: [issue11992] sys.settrace doesn't disable tracing if a local trace
 function returns None
In-Reply-To: <1304474333.76.0.969797512596.issue11992@psf.upfronthosting.co.za>
Message-ID: <1484739134.56.0.662490173894.issue11992@psf.upfronthosting.co.za>


Armin Rigo added the comment:

Confirmed.  More interestingly, nowadays (at least in 3.5) test_pdb.py depends on this bug.  If we really clear f->f_trace when the trace function returns None, then test_pdb_until_command_for_generator() fails.  This is because pdb.py incorrectly thinks there is no breakpoint in the generator function, and returns None.  This doesn't actually clear anything, and so it works anyway.

I'd suggest to fix the documentation to reflect the actual behavior of all versions from 2.3 to (at least) 3.5.

----------
nosy: +arigo

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue11992>
_______________________________________

From report at bugs.python.org  Wed Jan 18 06:40:38 2017
From: report at bugs.python.org (Martin Panter)
Date: Wed, 18 Jan 2017 11:40:38 +0000
Subject: [issue3991] urllib.request.urlopen does not handle non-ASCII
 characters
In-Reply-To: <1222627636.82.0.587488225038.issue3991@psf.upfronthosting.co.za>
Message-ID: <1484739638.23.0.415721724297.issue3991@psf.upfronthosting.co.za>


Martin Panter added the comment:

Issue 9679: Focusses on encoding just the DNS name
Issue 20559: Maybe a duplicate, or opportunity for better documentation or error message as a bug fix?

Andreas?s patch just proposes a new function called quote_uri(). It would need documentation. We already have a quote() and quote_plus() function. Since it sounds like this is for IRIs (https://tools.ietf.org/html/rfc3987), would it be more appropriate to call it quote_iri()?

See revision cb09fdef19f5, especially the quote(safe=...) parameter, for how I avoided the double encoding problem.

----------
dependencies: +unicode DNS names in urllib, urlopen
nosy: +martin.panter
stage: test needed -> patch review
versions: +Python 3.7 -Python 3.2, Python 3.3, Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue3991>
_______________________________________

From report at bugs.python.org  Wed Jan 18 06:53:01 2017
From: report at bugs.python.org (Martin Panter)
Date: Wed, 18 Jan 2017 11:53:01 +0000
Subject: [issue29305] encoding to ascii in http/client.py
In-Reply-To: <1484728861.39.0.489354661051.issue29305@psf.upfronthosting.co.za>
Message-ID: <1484740381.61.0.496715318542.issue29305@psf.upfronthosting.co.za>


Martin Panter added the comment:

In general, HTTP URLs are supposed to be ASCII only. Newer protocols (e.g. RTSP which is based on HTTP) specifically allow UTF-8 encoding. But it would be wrong for Python?s HTTP library to assume UTF-8 is wanted everywhere. Especially in a domain name (e.g. in the full-URL request to a proxy), which should not be UTF-8 encoded. 

I suggest to work on handling IRIs (<https://tools.ietf.org/html/rfc3987>, basically Unicode URLs) in higher-level places like ?urllib?. See Issue 3991.

----------
nosy: +martin.panter
resolution:  -> rejected
status: open -> closed
superseder:  -> urllib.request.urlopen does not handle non-ASCII characters
title: encoding to ascii in client.py -> encoding to ascii in http/client.py
type: compile error -> enhancement

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29305>
_______________________________________

From report at bugs.python.org  Wed Jan 18 07:30:13 2017
From: report at bugs.python.org (Roundup Robot)
Date: Wed, 18 Jan 2017 12:30:13 +0000
Subject: =?utf-8?b?W2lzc3VlMjkyNzRdIENoYW5nZSDigJx0ZXN0cyBjYXNlc+KAnSDihpIg4oCc?=
 =?utf-8?q?test_cases=E2=80=9D?=
In-Reply-To: <1484379059.0.0.00296348115264.issue29274@psf.upfronthosting.co.za>
Message-ID: <20170118123009.56146.58791.4C9BFF20@psf.io>


Roundup Robot added the comment:

New changeset f2fe00653d07 by Martin Panter in branch '3.5':
Issue #29274: tests cases ? test cases
https://hg.python.org/cpython/rev/f2fe00653d07

New changeset 145da99b3df2 by Martin Panter in branch '3.6':
Issue 29274: Merge doc fixes from 3.5
https://hg.python.org/cpython/rev/145da99b3df2

New changeset f093f88f8a5c by Martin Panter in branch 'default':
Issue 29274: Merge doc fixes from 3.6
https://hg.python.org/cpython/rev/f093f88f8a5c

New changeset 9b22d52a6d4b by Martin Panter in branch '2.7':
Issue #29274: tests cases ? test cases
https://hg.python.org/cpython/rev/9b22d52a6d4b

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29274>
_______________________________________

From report at bugs.python.org  Wed Jan 18 07:50:33 2017
From: report at bugs.python.org (Roundup Robot)
Date: Wed, 18 Jan 2017 12:50:33 +0000
Subject: [issue29259] Add tp_fastcall to PyTypeObject: support FASTCALL
 calling convention for all callable objects
In-Reply-To: <1484310274.99.0.832491900546.issue29259@psf.upfronthosting.co.za>
Message-ID: <20170118125030.7203.51489.0B984ED5@psf.io>


Roundup Robot added the comment:

New changeset a3865d0c1844 by Victor Stinner in branch 'default':
Fix Python 2.6 support in python-gdb.py
https://hg.python.org/cpython/rev/a3865d0c1844

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29259>
_______________________________________

From report at bugs.python.org  Wed Jan 18 08:00:54 2017
From: report at bugs.python.org (R. David Murray)
Date: Wed, 18 Jan 2017 13:00:54 +0000
Subject: [issue29307] ModuleNotFoundError when using literal string
 interpolation with invalid format specifier
In-Reply-To: <1484736983.43.0.107395329278.issue29307@psf.upfronthosting.co.za>
Message-ID: <1484744454.46.0.714914102402.issue29307@psf.upfronthosting.co.za>


R. David Murray added the comment:

Yes, this appears to be the vendor's failure reporting infrastructure that is failing.  Why they'd want a report for every traceback at the interactive prompt is beyond me, but that appears to be what they are trying to do.

----------
nosy: +r.david.murray
resolution:  -> third party
stage:  -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29307>
_______________________________________

From report at bugs.python.org  Wed Jan 18 08:23:00 2017
From: report at bugs.python.org (Roundup Robot)
Date: Wed, 18 Jan 2017 13:23:00 +0000
Subject: [issue29259] Add tp_fastcall to PyTypeObject: support FASTCALL
 calling convention for all callable objects
In-Reply-To: <1484310274.99.0.832491900546.issue29259@psf.upfronthosting.co.za>
Message-ID: <20170118132257.111706.42578.D02758AF@psf.io>


Roundup Robot added the comment:

New changeset a241817424e5 by Victor Stinner in branch 'default':
Fix _PyMethodDef_RawFastCallDict() argument parsing
https://hg.python.org/cpython/rev/a241817424e5

New changeset a8d35309dcc0 by Victor Stinner in branch 'default':
PyCFunction_Call() now calls _PyCFunction_FastCallDict()
https://hg.python.org/cpython/rev/a8d35309dcc0

New changeset ee6e1b1151a8 by Victor Stinner in branch 'default':
_PyObject_FastCallKeywords() now checks !PyErr_Occurred()
https://hg.python.org/cpython/rev/ee6e1b1151a8

New changeset 11039ece46b9 by Victor Stinner in branch 'default':
Rephrase !PyErr_Occurred() comment: may=>can
https://hg.python.org/cpython/rev/11039ece46b9

New changeset 8cc5d78d9b18 by Victor Stinner in branch 'default':
Cleanup _PyMethodDef_RawFastCallDict()
https://hg.python.org/cpython/rev/8cc5d78d9b18

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29259>
_______________________________________

From report at bugs.python.org  Wed Jan 18 08:37:04 2017
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 18 Jan 2017 13:37:04 +0000
Subject: [issue29306] Check usage of Py_EnterRecursiveCall() and
 Py_LeaveRecursiveCall() in new FASTCALL functions
In-Reply-To: <1484730285.0.0.960009012423.issue29306@psf.upfronthosting.co.za>
Message-ID: <1484746624.35.0.62109232204.issue29306@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Naoki: "On the other hand, PyCFunction may calls method of extension module, and it can cause recursive call. So I think PyCFunction_*Call* methods calling function pointer it has (e.g. _PyCFunction_FastCallKeywords) should call Py_EnterRecursiveCall()."

In Python 3.5, C functions are only calls inside Py_EnterRecursiveCall() if calls from PyObject_Call(). There are many other ways to call C functions which don't use Py_EnterRecursiveCall(). Examples:

* call_function() => call the C function
* call_function() => PyCFunction_Call() => call the C function
* call_function() => do_call() => PyCFunction_Call() => call the C function
* ext_do_call() => PyCFunction_Call() => call the C function

call_function() and do_call() have a special case for PyCFunction, otherwise they call the generic PyObject_Call() which uses Py_EnterRecursiveCall().

I agree that calling C functions with Py_EnterRecursiveCall() would help to prevent stack overflow crashs.

Attached enter_recursive_call.patch patch:

* _PyMethodDef_RawFastCallDict() (internal helper of _PyCFunction_FastCallDict()) and _PyCFunction_FastCallKeywords() now use Py_EnterRecursiveCall()
* _PyObject_FastCallKeywords(): move Py_EnterRecursiveCall() closer to the final call to simplify error handling
* Modify _PyObject_FastCallDict() to not call _PyFunction_FastCallDict() nor _PyCFunction_FastCallDict() inside Py_EnterRecursiveCall(), since these functions already use Py_EnterRecursiveCall() internally

----------
keywords: +patch
Added file: http://bugs.python.org/file46327/enter_recursive_call.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29306>
_______________________________________

From report at bugs.python.org  Wed Jan 18 09:38:11 2017
From: report at bugs.python.org (Xavier de Gaye)
Date: Wed, 18 Jan 2017 14:38:11 +0000
Subject: [issue28997] test_readline.test_nonascii fails on Android
In-Reply-To: <1481965369.72.0.409169776064.issue28997@psf.upfronthosting.co.za>
Message-ID: <1484750291.92.0.511679623979.issue28997@psf.upfronthosting.co.za>


Xavier de Gaye added the comment:

PEP 538 [1] coerces the C locale to UTF-8 by setting the locale environment variables (LC_ALL, LANG).  The PEP has an implementation at issue 28180 as pep538_coerce_legacy_c_locale_v3.diff, and the patch fixes test_nonascii when run on the Android emulators (api 21 and 24).  This is as expected as now both Python and Readline are in agreement and use the same encoding.
I think we should wait for the resolution of PEP 538 and of issue 28684. If this does not happen, I agree we should skip the test with the message you are proposing Martin.

[1] https://www.python.org/dev/peps/pep-0538

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28997>
_______________________________________

From report at bugs.python.org  Wed Jan 18 09:39:37 2017
From: report at bugs.python.org (Jack Bennett)
Date: Wed, 18 Jan 2017 14:39:37 +0000
Subject: [issue29308] venv should match virtualenv VIRTUAL_ENV_DISABLE_PROMPT
 config
Message-ID: <1484750377.25.0.809506276992.issue29308@psf.upfronthosting.co.za>


New submission from Jack Bennett:

virtualenv has https://virtualenv.pypa.io/en/latest/reference/#envvar-VIRTUAL_ENV_DISABLE_PROMPT to block activate from trying to overwrite the prompt function. 

Users of venv can't do that, 3.6 added prompt but even None defaults to the current folder name.

If I may suggest either respecting the virtualenv variable or perhaps prompt=False

Specifically we've ran into this in powershell while making cmder stricter on the prompt over here: https://github.com/cmderdev/cmder/issues/1207

Thanks, Jack

----------
components: Library (Lib)
messages: 285729
nosy: Jack Bennett
priority: normal
severity: normal
status: open
title: venv should match virtualenv VIRTUAL_ENV_DISABLE_PROMPT config
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29308>
_______________________________________

From report at bugs.python.org  Wed Jan 18 09:41:50 2017
From: report at bugs.python.org (Barry A. Warsaw)
Date: Wed, 18 Jan 2017 14:41:50 +0000
Subject: [issue29307] ModuleNotFoundError when using literal string
 interpolation with invalid format specifier
In-Reply-To: <1484736983.43.0.107395329278.issue29307@psf.upfronthosting.co.za>
Message-ID: <20170118094146.18e67b4f@subdivisions.wooz.org>


Barry A. Warsaw added the comment:

On Jan 18, 2017, at 10:56 AM, Jeroen Van Goey wrote:

>sudo add-apt-repository ppa:jonathonf/python-3.6
>sudo apt-get update
>sudo apt-get install python3.6
>
>I made a string, using the new literal string interpolation, but I supplied
>an invalid format specifier. I not only got the expected "ValueError: Invalid
>format specifier", but also the unexpected "ModuleNotFoundError: No module
>named 'apt_pkg'".

Please understand that installing Python 3.6 from a random PPA does *not*
provide full support for this version of the interpreter.  Python 3.6 is not
yet a supported version in any version of Ubuntu (which I'm assuming your
using), although we are working on it for 17.04.

Very often, you can install a new Python 3 interpreter package and many things
will work because the Ubuntu infrastructure shares pure-Python modules across
all installed Python 3's.  Technically speaking, they will all have
/usr/lib/python3/dist-packages on their sys.path so any third party
pure-Python module built for a support version of Python 3 will be importable
by any (package-built) installed version of Python 3.

But that 1) is a long way from saying that those third-party modules will
work; 2) does *not* include any packages containing C extension modules, which
must be rebuilt for the specific interpreter version.

Supporting a new version of Python is a long process, for which we are just
starting.  Please engage with ubuntu-devel at ubuntu.com for details.

Ubuntu does install a standard exception handler so that when Python
applications and such crash, we can gather crash statistics, so that we can
devote resources to fixing common problems and regressions.  apport (which you
see in the traceback) is that crash reporting infrastructure.  apport calls
apt_pkg, which is an (C++) extension module and thus won't have been built for
the version of Python 3.6 you installed from that PPA, unless of course the
PPA owner (who I don't know) has also done an archive-wide Python 3 rebuild.
Since I'm in the process of setting that up, and I know it's quite a bit of
work, I doubt that's been done for this rather random PPA.

The ubuntu-devel mailing list is a better place to discuss the ongoing work to
bring Python 3.6 as a supported version on Ubuntu.

----------
nosy: +barry

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29307>
_______________________________________

From report at bugs.python.org  Wed Jan 18 09:54:13 2017
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 18 Jan 2017 14:54:13 +0000
Subject: [issue29259] Add tp_fastcall to PyTypeObject: support FASTCALL
 calling convention for all callable objects
In-Reply-To: <1484310274.99.0.832491900546.issue29259@psf.upfronthosting.co.za>
Message-ID: <1484751253.51.0.459247966222.issue29259@psf.upfronthosting.co.za>


STINNER Victor added the comment:

See also issue #29306: "Check usage of Py_EnterRecursiveCall() and Py_LeaveRecursiveCall() in new FASTCALL functions".

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29259>
_______________________________________

From report at bugs.python.org  Wed Jan 18 09:54:38 2017
From: report at bugs.python.org (Chi Hsuan Yen)
Date: Wed, 18 Jan 2017 14:54:38 +0000
Subject: [issue29308] venv should match virtualenv VIRTUAL_ENV_DISABLE_PROMPT
 config
In-Reply-To: <1484750377.25.0.809506276992.issue29308@psf.upfronthosting.co.za>
Message-ID: <1484751278.81.0.133302860971.issue29308@psf.upfronthosting.co.za>


Changes by Chi Hsuan Yen <yan12125 at gmail.com>:


----------
nosy: +vinay.sajip

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29308>
_______________________________________

From report at bugs.python.org  Wed Jan 18 09:58:33 2017
From: report at bugs.python.org (=?utf-8?q?Romain_S=C3=A9bille?=)
Date: Wed, 18 Jan 2017 14:58:33 +0000
Subject: [issue29309] Interpreter hang when interrupting a
 loop.run_in_executor() future
Message-ID: <1484751513.93.0.173979168861.issue29309@psf.upfronthosting.co.za>


New submission from Romain S?bille:

Hi,

I stumble on this today and can't find any reasons or previous issue as to why this happen so I thought of submitting the issue here.

I attached a script (run_in_executor.py) to reproduce the problem, the original problem arise when waiting to read on a socket but I have the same result with time.sleep().

The output:
$ python3.6 run_in_executor.py
Start
Going to sleep
^CClosing loop
Traceback (most recent call last):
  File "run_in_executor.py", line 15, in <module>
    loop.run_until_complete(sleep_in_executor(loop))
  File "/usr/local/lib/python3.6/asyncio/base_events.py", line 454, in run_until_complete
    self.run_forever()
  File "/usr/local/lib/python3.6/asyncio/base_events.py", line 421, in run_forever
    self._run_once()
  File "/usr/local/lib/python3.6/asyncio/base_events.py", line 1389, in _run_once
    event_list = self._selector.select(timeout)
  File "/usr/local/lib/python3.6/selectors.py", line 445, in select
    fd_event_list = self._epoll.poll(timeout, max_ev)
KeyboardInterrupt
^CError in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/concurrent/futures/thread.py", line 39, in _python_exit
    t.join()
  File "/usr/local/lib/python3.6/threading.py", line 1056, in join
    self._wait_for_tstate_lock()
  File "/usr/local/lib/python3.6/threading.py", line 1072, in _wait_for_tstate_lock
    elif lock.acquire(block, timeout):
KeyboardInterrupt

Current behavior:
We have to send two ^C before we go back to the shell.

Expected behavior:
Only one ^C to go back to the shell.

Python versions tested:
- Python 3.4.2
- Python 3.4.6
- Python 3.5.3
- Python 3.6.0

>From what I have gathered the error is within the ThreadPoolExecutor cleaning function but I don't see why there are still living workers, the Executor.shutdown() take care of that.
Unfortunately I don't really know what to look, is a AsyncIO bug ? A ThreadPoolExecutor bug ? Or something else.

----------
components: asyncio
files: run_in_executor.py
messages: 285732
nosy: gvanrossum, rsebille, yselivanov
priority: normal
severity: normal
status: open
title: Interpreter hang when interrupting a loop.run_in_executor() future
type: behavior
versions: Python 3.4, Python 3.5, Python 3.6
Added file: http://bugs.python.org/file46328/run_in_executor.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29309>
_______________________________________

From report at bugs.python.org  Wed Jan 18 10:01:34 2017
From: report at bugs.python.org (Jeroen Van Goey)
Date: Wed, 18 Jan 2017 15:01:34 +0000
Subject: [issue29307] ModuleNotFoundError when using literal string
 interpolation with invalid format specifier
In-Reply-To: <1484736983.43.0.107395329278.issue29307@psf.upfronthosting.co.za>
Message-ID: <1484751694.51.0.998172069946.issue29307@psf.upfronthosting.co.za>


Jeroen Van Goey added the comment:

Thanks Barry for your extensive explanation!

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29307>
_______________________________________

From report at bugs.python.org  Wed Jan 18 10:04:53 2017
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 18 Jan 2017 15:04:53 +0000
Subject: [issue29309] Interpreter hang when interrupting a
 loop.run_in_executor() future
In-Reply-To: <1484751513.93.0.173979168861.issue29309@psf.upfronthosting.co.za>
Message-ID: <1484751893.18.0.663135679486.issue29309@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Sending SIGTERM (CTRL+c, KeyboardInterrupt) while Python is waiting in threading.Thread.join() is not supported.

You have a thread running time.sleep(1). On Linux, this function calls the select() syscall. So you have the main thread waiting for the thread state lock, and a thread waiting in select(). If SIGTERM is received in the main thread, you enter a weird state of the threading module.

Python has a signal.pthread_kill() which might help to only send a signal to a specific thread, but I don't know exactly how Python behaves, since Python wants to execute Python signal handles in the main thread...

----------
nosy: +haypo

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29309>
_______________________________________

From report at bugs.python.org  Wed Jan 18 10:16:00 2017
From: report at bugs.python.org (Xavier de Gaye)
Date: Wed, 18 Jan 2017 15:16:00 +0000
Subject: [issue28180] sys.getfilesystemencoding() should default to utf-8
In-Reply-To: <1474024622.65.0.838685461995.issue28180@psf.upfronthosting.co.za>
Message-ID: <1484752560.88.0.512769974572.issue28180@psf.upfronthosting.co.za>


Xavier de Gaye added the comment:

pep538_coerce_legacy_c_locale_v3.diff fixes issue 28997 on Android (api 21 and 24). This issue is raised because there is an inconsistency between Python on Android that considers the locale encoding to be always UTF-8 and GNU Readline that does not accept eight-bit characters when LANG is not set (on Android).

On Android, setlocale(CATEGORY, "") does not look for the locale environment variables (LANG, ...) but sets the 'C' locale instead, so the patch does not fully behave as expected and the 'Py_Initialize detected' warning is emitted. Here is the output of an interactive session on Android:

root at generic_x86:/data/data/org.bitbucket.pyona # python
Python detected LC_CTYPE=C, forcing LC_ALL & LANG to C.UTF-8 (set PYTHONALLOWCLOCALE to disable this locale coercion behaviour).
Py_Initialize detected LC_CTYPE=C, which limits Unicode compatibility. Some libraries and operating system interfaces may not work correctly. Set `PYTHONALLOWCLOCALE=1 LC_CTYPE=C` to configure a similar environment when running Python directly.
Python 3.7.0a0 (default:0503024831ad+, Jan 18 2017, 11:34:53)
[GCC 4.2.1 Compatible Android Clang 3.8.256229 ] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import locale, os
>>> os.environ['LANG']
'C.UTF-8'
>>> locale.getdefaultlocale()
('en_US', 'UTF-8')
>>> locale.setlocale(locale.LC_CTYPE)
'C'
>>> locale.setlocale(locale.LC_ALL, 'en_US.UTF-8')
'C.UTF-8'
>>> locale.setlocale(locale.LC_CTYPE)
'C.UTF-8'

The attached android_setlocale.patch fixes the following problems when applied after pep538_coerce_legacy_c_locale_v3.diff:
* No 'Py_Initialize detected' warning is emitted.
* locale.setlocale(locale.LC_CTYPE) returns now 'C.UTF-8'.

----------
nosy: +xdegaye
Added file: http://bugs.python.org/file46329/android_setlocale.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28180>
_______________________________________

From report at bugs.python.org  Wed Jan 18 10:24:02 2017
From: report at bugs.python.org (Xavier de Gaye)
Date: Wed, 18 Jan 2017 15:24:02 +0000
Subject: [issue28997] test_readline.test_nonascii fails on Android
In-Reply-To: <1481965369.72.0.409169776064.issue28997@psf.upfronthosting.co.za>
Message-ID: <1484753042.1.0.0961607615725.issue28997@psf.upfronthosting.co.za>


Xavier de Gaye added the comment:

> I think we should wait for the resolution of PEP 538 and of issue 28684

s/issue 28684/issue 28180/

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28997>
_______________________________________

From report at bugs.python.org  Wed Jan 18 10:57:06 2017
From: report at bugs.python.org (Matthias Klose)
Date: Wed, 18 Jan 2017 15:57:06 +0000
Subject: [issue1294959] Problems with /usr/lib64 builds.
Message-ID: <1484755026.92.0.456254217935.issue1294959@psf.upfronthosting.co.za>


Matthias Klose added the comment:

I don't think we need a PEP for this, however I would like to name the new macro libsubdir, or sublibdir, to make it clear it's just the libdir component. Or make the value absolute, like it already is for libdir ...

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue1294959>
_______________________________________

From report at bugs.python.org  Wed Jan 18 11:00:13 2017
From: report at bugs.python.org (Barry A. Warsaw)
Date: Wed, 18 Jan 2017 16:00:13 +0000
Subject: [issue1294959] Problems with /usr/lib64 builds.
In-Reply-To: <1484755026.92.0.456254217935.issue1294959@psf.upfronthosting.co.za>
Message-ID: <20170118110010.71142ef3@subdivisions.wooz.org>


Barry A. Warsaw added the comment:

On Jan 18, 2017, at 03:57 PM, Matthias Klose wrote:

>I don't think we need a PEP for this

Correct.  Generally we don't need PEPs for build system changes.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue1294959>
_______________________________________

From report at bugs.python.org  Wed Jan 18 11:05:14 2017
From: report at bugs.python.org (Roundup Robot)
Date: Wed, 18 Jan 2017 16:05:14 +0000
Subject: [issue28556] typing.py upgrades
In-Reply-To: <1477756009.46.0.442279756181.issue28556@psf.upfronthosting.co.za>
Message-ID: <20170118160511.111675.41139.4C9C6019@psf.io>


Roundup Robot added the comment:

New changeset 59829d7b7838 by Guido van Rossum in branch '3.5':
Issue #28556: allow default values in class form of NamedTuple -- Jelle Zijlstra
https://hg.python.org/cpython/rev/59829d7b7838

New changeset 2159f36ccd6b by Guido van Rossum in branch '3.6':
Issue #28556: allow default values in class form of NamedTuple -- Jelle Zijlstra (3.5->3.6)
https://hg.python.org/cpython/rev/2159f36ccd6b

New changeset f0e1c49233ff by Guido van Rossum in branch 'default':
Issue #28556: allow default values in class form of NamedTuple -- Jelle Zijlstra (3.6->3.7)
https://hg.python.org/cpython/rev/f0e1c49233ff

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28556>
_______________________________________

From report at bugs.python.org  Wed Jan 18 11:09:17 2017
From: report at bugs.python.org (Jelle Zijlstra)
Date: Wed, 18 Jan 2017 16:09:17 +0000
Subject: [issue29310] Document typing.NamedTuple default argument syntax
Message-ID: <1484755757.54.0.970081096642.issue29310@psf.upfronthosting.co.za>


New submission from Jelle Zijlstra:

Document https://github.com/python/typing/pull/338

I should have a patch in the next few days; creating this issue to remind myself.

----------
assignee: Jelle Zijlstra
components: Documentation
messages: 285740
nosy: Jelle Zijlstra
priority: normal
severity: normal
status: open
title: Document typing.NamedTuple default argument syntax
versions: Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29310>
_______________________________________

From report at bugs.python.org  Wed Jan 18 11:17:16 2017
From: report at bugs.python.org (Ivan Levkivskyi)
Date: Wed, 18 Jan 2017 16:17:16 +0000
Subject: [issue29310] Document typing.NamedTuple default argument syntax
In-Reply-To: <1484755757.54.0.970081096642.issue29310@psf.upfronthosting.co.za>
Message-ID: <1484756236.39.0.563986145196.issue29310@psf.upfronthosting.co.za>


Changes by Ivan Levkivskyi <levkivskyi at gmail.com>:


----------
nosy: +levkivskyi

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29310>
_______________________________________

From report at bugs.python.org  Wed Jan 18 11:22:29 2017
From: report at bugs.python.org (Guido van Rossum)
Date: Wed, 18 Jan 2017 16:22:29 +0000
Subject: [issue29310] Document typing.NamedTuple default argument syntax
In-Reply-To: <1484755757.54.0.970081096642.issue29310@psf.upfronthosting.co.za>
Message-ID: <1484756549.05.0.0545635911033.issue29310@psf.upfronthosting.co.za>


Changes by Guido van Rossum <guido at python.org>:


----------
nosy: +gvanrossum

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29310>
_______________________________________

From report at bugs.python.org  Wed Jan 18 11:23:46 2017
From: report at bugs.python.org (Roundup Robot)
Date: Wed, 18 Jan 2017 16:23:46 +0000
Subject: [issue29259] Add tp_fastcall to PyTypeObject: support FASTCALL
 calling convention for all callable objects
In-Reply-To: <1484310274.99.0.832491900546.issue29259@psf.upfronthosting.co.za>
Message-ID: <20170118162343.55951.69864.3A5AAB26@psf.io>


Roundup Robot added the comment:

New changeset 127401469628 by Victor Stinner in branch 'default':
Update and enhance python-gdb.py
https://hg.python.org/cpython/rev/127401469628

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29259>
_______________________________________

From report at bugs.python.org  Wed Jan 18 11:54:10 2017
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 18 Jan 2017 16:54:10 +0000
Subject: [issue29311] Argument Clinic: convert dict methods
Message-ID: <1484758450.31.0.717694851278.issue29311@psf.upfronthosting.co.za>


New submission from STINNER Victor:

The dict type has multiple methods which use METH_VARARGS or METH_VARARGS|METH_KEYWORDS calling convention, whereras there is no a new faster METH_FASTCALL calling convention which avoids temporary tuple/dict to pass arguments.

----------
components: Argument Clinic
messages: 285742
nosy: haypo, inada.naoki, larry
priority: normal
severity: normal
status: open
title: Argument Clinic: convert dict methods
type: performance
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29311>
_______________________________________

From report at bugs.python.org  Wed Jan 18 11:57:41 2017
From: report at bugs.python.org (Ammar Askar)
Date: Wed, 18 Jan 2017 16:57:41 +0000
Subject: [issue29292] Missing a parameter in PyEval_EvalCodeEx doc
In-Reply-To: <1484640380.78.0.065326592654.issue29292@psf.upfronthosting.co.za>
Message-ID: <1484758661.14.0.0434573151935.issue29292@psf.upfronthosting.co.za>


Ammar Askar added the comment:

You're completely right, the kwdefs is referring to default arguments for keyword-only-arguments from this PEP: https://www.python.org/dev/peps/pep-3102/

Where as that line is probably referring to "defs", the defaults for normal parameters.

I'll upload an amended patch.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29292>
_______________________________________

From report at bugs.python.org  Wed Jan 18 12:07:42 2017
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 18 Jan 2017 17:07:42 +0000
Subject: [issue29312] Use FASTCALL in dict.update()
Message-ID: <1484759262.53.0.625567482548.issue29312@psf.upfronthosting.co.za>


New submission from STINNER Victor:

Follow-up of the issue #29311 "Argument Clinic: convert dict methods".

The dict.update() method hs a special prototype:

   def update(arg=None **kw): ...

I don't think that Argument Clinic supports it right now, so I propose to first optimize dict_update() to use METH_FASTCALL.

Attached patch is a first step: convert kwnames tuple to a dict.

A second step would be to avoid the temporary dict and update the dict using args + kwnames directly.

----------
files: dict_update_fastcall.patch
keywords: patch
messages: 285744
nosy: haypo, inada.naoki
priority: normal
severity: normal
status: open
title: Use FASTCALL in dict.update()
type: enhancement
versions: Python 3.7
Added file: http://bugs.python.org/file46330/dict_update_fastcall.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29312>
_______________________________________

From report at bugs.python.org  Wed Jan 18 12:10:15 2017
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 18 Jan 2017 17:10:15 +0000
Subject: [issue29312] Use FASTCALL in dict.update()
In-Reply-To: <1484759262.53.0.625567482548.issue29312@psf.upfronthosting.co.za>
Message-ID: <1484759415.16.0.279439377775.issue29312@psf.upfronthosting.co.za>


STINNER Victor added the comment:

My patch doesn't use _PyStack_AsDict() since this function currently fails with an assertion error if a key is not exactly a string (PyUnicode_CheckExact). dict_update_common() already checks if all keys are string.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29312>
_______________________________________

From report at bugs.python.org  Wed Jan 18 12:13:20 2017
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 18 Jan 2017 17:13:20 +0000
Subject: [issue29311] Argument Clinic: convert dict methods
In-Reply-To: <1484758450.31.0.717694851278.issue29311@psf.upfronthosting.co.za>
Message-ID: <1484759600.06.0.185485960527.issue29311@psf.upfronthosting.co.za>


STINNER Victor added the comment:

dict.patch converts two methods to Argument Clinic:

* get()
* setdefault()

pop() requires the issue #29299, its signature must not show any default value for the second parameter.

dict.update() is special, so I created a dedicated issue: issue #29312.

----------
keywords: +patch
Added file: http://bugs.python.org/file46331/dict.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29311>
_______________________________________

From report at bugs.python.org  Wed Jan 18 12:13:27 2017
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 18 Jan 2017 17:13:27 +0000
Subject: [issue29312] Use FASTCALL in dict.update()
In-Reply-To: <1484759262.53.0.625567482548.issue29312@psf.upfronthosting.co.za>
Message-ID: <1484759607.64.0.795225168641.issue29312@psf.upfronthosting.co.za>


Changes by STINNER Victor <victor.stinner at gmail.com>:


----------
components: +Argument Clinic
nosy: +larry

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29312>
_______________________________________

From report at bugs.python.org  Wed Jan 18 12:14:13 2017
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 18 Jan 2017 17:14:13 +0000
Subject: [issue29312] Use FASTCALL in dict.update()
In-Reply-To: <1484759262.53.0.625567482548.issue29312@psf.upfronthosting.co.za>
Message-ID: <1484759653.96.0.538061722669.issue29312@psf.upfronthosting.co.za>


STINNER Victor added the comment:

See also issue #20291: "Argument Clinic should understand *args and **kwargs parameters".

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29312>
_______________________________________

From report at bugs.python.org  Wed Jan 18 12:16:19 2017
From: report at bugs.python.org (Ammar Askar)
Date: Wed, 18 Jan 2017 17:16:19 +0000
Subject: [issue29292] Missing a parameter in PyEval_EvalCodeEx doc
In-Reply-To: <1484640380.78.0.065326592654.issue29292@psf.upfronthosting.co.za>
Message-ID: <1484759779.11.0.186256883182.issue29292@psf.upfronthosting.co.za>


Changes by Ammar Askar <ammar at ammaraskar.com>:


Added file: http://bugs.python.org/file46332/kwdefs_docs.diff2

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29292>
_______________________________________

From report at bugs.python.org  Wed Jan 18 12:22:06 2017
From: report at bugs.python.org (=?utf-8?q?Romain_S=C3=A9bille?=)
Date: Wed, 18 Jan 2017 17:22:06 +0000
Subject: [issue29309] Interpreter hang when interrupting a
 loop.run_in_executor() future
In-Reply-To: <1484751513.93.0.173979168861.issue29309@psf.upfronthosting.co.za>
Message-ID: <1484760126.34.0.121934776288.issue29309@psf.upfronthosting.co.za>


Romain S?bille added the comment:

Ok, thanks for the explanation!

One thing still bother me, run_in_executor() allow us to launch blocking operations but without blocking the main thread, so there is a high chance that we wait inside one of the pool's worker.
Does this mean run_in_executor() shouldn't be used for long/indefinite blocking operation?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29309>
_______________________________________

From report at bugs.python.org  Wed Jan 18 14:38:39 2017
From: report at bugs.python.org (eszense)
Date: Wed, 18 Jan 2017 19:38:39 +0000
Subject: [issue29313] msi by bdist_msi will fail execute install-scripts if
 space in present in python path
Message-ID: <1484768319.35.0.107822358832.issue29313@psf.upfronthosting.co.za>


New submission from eszense:

If python.exe path contains space, msi generated with bdist_msi command with --install-script=abc.py will fail with msg:

C:\\Program Files\\Python35\\python.exe: can't open file 'C:\\Program': [Error 2] No such file or directory

Patch attached for your consideration

----------
components: Distutils
files: patch.diff
keywords: patch
messages: 285749
nosy: dstufft, eric.araujo, eszense
priority: normal
severity: normal
status: open
title: msi by bdist_msi will fail execute install-scripts if space in present in python path
type: crash
Added file: http://bugs.python.org/file46333/patch.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29313>
_______________________________________

From report at bugs.python.org  Wed Jan 18 15:07:47 2017
From: report at bugs.python.org (R. David Murray)
Date: Wed, 18 Jan 2017 20:07:47 +0000
Subject: [issue29314] asyncio.async deprecation warning is missing stacklevel=2
Message-ID: <1484770067.77.0.974182324744.issue29314@psf.upfronthosting.co.za>


New submission from R. David Murray:

Title pretty much says it all.  The warning isn't very helpful currently, since it just reports the line in tasks.py where the deprecation warning is generated.

----------
messages: 285750
nosy: r.david.murray
priority: normal
severity: normal
status: open
title: asyncio.async deprecation warning is missing stacklevel=2
versions: Python 3.5, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29314>
_______________________________________

From report at bugs.python.org  Wed Jan 18 15:08:03 2017
From: report at bugs.python.org (R. David Murray)
Date: Wed, 18 Jan 2017 20:08:03 +0000
Subject: [issue29314] asyncio.async deprecation warning is missing stacklevel=2
In-Reply-To: <1484770067.77.0.974182324744.issue29314@psf.upfronthosting.co.za>
Message-ID: <1484770083.12.0.655852971833.issue29314@psf.upfronthosting.co.za>


Changes by R. David Murray <rdmurray at bitdance.com>:


----------
components: +asyncio
nosy: +gvanrossum, yselivanov
stage:  -> needs patch
type:  -> behavior

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29314>
_______________________________________

From report at bugs.python.org  Wed Jan 18 15:21:15 2017
From: report at bugs.python.org (Mike Lissner)
Date: Wed, 18 Jan 2017 20:21:15 +0000
Subject: [issue29315] \b requires raw strings or to be escaped. Update docs
 with that hint?
Message-ID: <1484770875.58.0.864108991771.issue29315@psf.upfronthosting.co.za>


New submission from Mike Lissner:

I just ran into a funny corner case I imagine others are aware of. When you write "\b" in Python, it is a single character: "\x08". So if you try to write a regex like:

words = '\b(.*)\b'

That won't work. But using a raw string will:

words = r'\b(.*)\b'

As will escaping it in this horrible fashion:

words = '\\b(.*)\\b'

I believe this doesn't affect any of the other regex flags, so I wonder if it's worth adding something to the docs to warn about this. I just spent a bunch of time trying to figure out why it seemed like \b wasn't working. A little tip in the docs would have gone a LONG way.

----------
assignee: docs at python
components: Documentation
messages: 285751
nosy: Mike.Lissner, docs at python
priority: normal
severity: normal
status: open
title: \b requires raw strings or to be escaped. Update docs with that hint?
type: enhancement
versions: Python 2.7, Python 3.3, Python 3.4, Python 3.5, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29315>
_______________________________________

From report at bugs.python.org  Wed Jan 18 15:40:26 2017
From: report at bugs.python.org (Martin Panter)
Date: Wed, 18 Jan 2017 20:40:26 +0000
Subject: =?utf-8?b?W2lzc3VlMjkyNzRdIENoYW5nZSDigJx0ZXN0cyBjYXNlc+KAnSDihpIg4oCc?=
 =?utf-8?q?test_cases=E2=80=9D?=
In-Reply-To: <1484379059.0.0.00296348115264.issue29274@psf.upfronthosting.co.za>
Message-ID: <1484772026.06.0.215235116988.issue29274@psf.upfronthosting.co.za>


Martin Panter added the comment:

Thanks for the feedback

----------
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29274>
_______________________________________

From report at bugs.python.org  Wed Jan 18 15:44:41 2017
From: report at bugs.python.org (=?utf-8?q?S=C3=A9bastien_Taylor?=)
Date: Wed, 18 Jan 2017 20:44:41 +0000
Subject: [issue24960] Can't use lib2to3 with embeddable zip file.
In-Reply-To: <1440869782.83.0.207113001402.issue24960@psf.upfronthosting.co.za>
Message-ID: <1484772281.63.0.624081654008.issue24960@psf.upfronthosting.co.za>


Changes by S?bastien Taylor <sebastientaylor at gmail.com>:


----------
nosy: +S?bastien Taylor

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24960>
_______________________________________

From report at bugs.python.org  Wed Jan 18 15:55:26 2017
From: report at bugs.python.org (Paul Moore)
Date: Wed, 18 Jan 2017 20:55:26 +0000
Subject: [issue24960] Can't use lib2to3 with embeddable zip file.
In-Reply-To: <1440869782.83.0.207113001402.issue24960@psf.upfronthosting.co.za>
Message-ID: <1484772926.31.0.528907110327.issue24960@psf.upfronthosting.co.za>


Paul Moore added the comment:

I'm not sure I see why you'd want to run lib2to3 with the embedded distribution anyway. Surely you'd do any 2to3 conversion in a normal development installation, and then simply bundle the resulting converted scripts with your application and the embedded interpreter?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24960>
_______________________________________

From report at bugs.python.org  Wed Jan 18 16:11:44 2017
From: report at bugs.python.org (Roundup Robot)
Date: Wed, 18 Jan 2017 21:11:44 +0000
Subject: [issue29198] AsyncGenerator is missing from typing
In-Reply-To: <1483825061.03.0.644790904886.issue29198@psf.upfronthosting.co.za>
Message-ID: <20170118211141.111675.35581.D533A10D@psf.io>


Roundup Robot added the comment:

New changeset 87e8139420ed by Guido van Rossum in branch '3.5':
Issue #29198: add AsyncGenerator (Jelle Zijlstra)
https://hg.python.org/cpython/rev/87e8139420ed

New changeset 5b771c662c00 by Guido van Rossum in branch '3.6':
Issue #29198: add AsyncGenerator (Jelle Zijlstra) (3.5->3.6)
https://hg.python.org/cpython/rev/5b771c662c00

New changeset 07523532039d by Guido van Rossum in branch 'default':
Issue #29198: add AsyncGenerator (Jelle Zijlstra) (3.6->3.7)
https://hg.python.org/cpython/rev/07523532039d

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29198>
_______________________________________

From report at bugs.python.org  Wed Jan 18 16:11:54 2017
From: report at bugs.python.org (R. David Murray)
Date: Wed, 18 Jan 2017 21:11:54 +0000
Subject: [issue29315] \b requires raw strings or to be escaped. Update docs
 with that hint?
In-Reply-To: <1484770875.58.0.864108991771.issue29315@psf.upfronthosting.co.za>
Message-ID: <1484773914.32.0.227732256789.issue29315@psf.upfronthosting.co.za>


R. David Murray added the comment:

One should always use raw strings for regex expressions, and this is already documented in the introduction to the regex module.  Further, in 3.5 using \ in front of characters that aren't special produces a warning, which should reduce the frequency of this mistake.

I don't see that there's anything to do here.

----------
nosy: +r.david.murray
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29315>
_______________________________________

From report at bugs.python.org  Wed Jan 18 16:14:35 2017
From: report at bugs.python.org (=?utf-8?q?S=C3=A9bastien_Taylor?=)
Date: Wed, 18 Jan 2017 21:14:35 +0000
Subject: [issue24960] Can't use lib2to3 with embeddable zip file.
In-Reply-To: <1440869782.83.0.207113001402.issue24960@psf.upfronthosting.co.za>
Message-ID: <1484774075.22.0.836818300958.issue24960@psf.upfronthosting.co.za>


S?bastien Taylor added the comment:

Are there existing tools for bundling the python35.zip (and adding additional packages)?  I personally ran into this issue by using pip to add additional modules to my embeddable python but if there's a better way to migrate from devel to embeddable python I'd rather not have to use pip for this.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24960>
_______________________________________

From report at bugs.python.org  Wed Jan 18 16:18:08 2017
From: report at bugs.python.org (Guido van Rossum)
Date: Wed, 18 Jan 2017 21:18:08 +0000
Subject: [issue29198] AsyncGenerator is missing from typing
In-Reply-To: <1483825061.03.0.644790904886.issue29198@psf.upfronthosting.co.za>
Message-ID: <1484774288.62.0.206508961595.issue29198@psf.upfronthosting.co.za>


Guido van Rossum added the comment:

So the changes to typing.py have been committed. Can someone commit the right doc patches?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29198>
_______________________________________

From report at bugs.python.org  Wed Jan 18 16:20:07 2017
From: report at bugs.python.org (Guido van Rossum)
Date: Wed, 18 Jan 2017 21:20:07 +0000
Subject: [issue29316] Can we keep typing.py provisional for the duration of
 the Python 3.6 release cycle?
Message-ID: <1484774407.47.0.689245055951.issue29316@psf.upfronthosting.co.za>


New submission from Guido van Rossum:

The 3.6 what's new (https://docs.python.org/3/whatsnew/3.6.html#whatsnew36-typing) claims typing.py is no longer provisional, but I don't think it's quite ready yet. There are a number of things that I'd like to merge into typing.py over the duration of 3.6's lifetime. Nothing backwards incompatible, but definitely new features.

See also https://github.com/python/typing/issues/247

----------
messages: 285758
nosy: gvanrossum
priority: normal
severity: normal
status: open
title: Can we keep typing.py provisional for the duration of the Python 3.6 release cycle?

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29316>
_______________________________________

From report at bugs.python.org  Wed Jan 18 16:22:01 2017
From: report at bugs.python.org (Ivan Levkivskyi)
Date: Wed, 18 Jan 2017 21:22:01 +0000
Subject: [issue29316] Can we keep typing.py provisional for the duration of
 the Python 3.6 release cycle?
In-Reply-To: <1484774407.47.0.689245055951.issue29316@psf.upfronthosting.co.za>
Message-ID: <1484774521.66.0.777017415293.issue29316@psf.upfronthosting.co.za>


Ivan Levkivskyi added the comment:

I don't have any strong opinion on this. If others don't object then I am perfectly fine with keeping typing provisional for 3.6.

----------
nosy: +levkivskyi

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29316>
_______________________________________

From report at bugs.python.org  Wed Jan 18 16:33:41 2017
From: report at bugs.python.org (=?utf-8?q?=C5=81ukasz_Langa?=)
Date: Wed, 18 Jan 2017 21:33:41 +0000
Subject: [issue29316] Can we keep typing.py provisional for the duration of
 the Python 3.6 release cycle?
In-Reply-To: <1484774407.47.0.689245055951.issue29316@psf.upfronthosting.co.za>
Message-ID: <1484775221.06.0.388493274788.issue29316@psf.upfronthosting.co.za>


?ukasz Langa added the comment:

There are three specific areas I can identify that are missing from the typing module at the moment:
* support for structural typing (protocols)
* support for typing heterogenous dictionaries (dictionary "schema")
* support for typing arbitrary callables

----------
components: +Library (Lib)
keywords: +3.6regression
nosy: +lukasz.langa
versions: +Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29316>
_______________________________________

From report at bugs.python.org  Wed Jan 18 16:40:12 2017
From: report at bugs.python.org (Ivan Levkivskyi)
Date: Wed, 18 Jan 2017 21:40:12 +0000
Subject: [issue29316] Can we keep typing.py provisional for the duration of
 the Python 3.6 release cycle?
In-Reply-To: <1484774407.47.0.689245055951.issue29316@psf.upfronthosting.co.za>
Message-ID: <1484775612.1.0.841392123089.issue29316@psf.upfronthosting.co.za>


Ivan Levkivskyi added the comment:

There are also minor things like NoReturn, typing_inspect helpers(?) etc. 

The main good point I see for keeping typing provisional is we could iterate fast and come up with more "settled" and robust API in time for 3.7.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29316>
_______________________________________

From report at bugs.python.org  Wed Jan 18 16:53:24 2017
From: report at bugs.python.org (Ammar Askar)
Date: Wed, 18 Jan 2017 21:53:24 +0000
Subject: [issue29281] json.loads documentation missing "versionchanged"
 statement
In-Reply-To: <1484558399.93.0.792457597374.issue29281@psf.upfronthosting.co.za>
Message-ID: <1484776404.51.0.521919939604.issue29281@psf.upfronthosting.co.za>


Ammar Askar added the comment:

To anyone more experienced with python documentation, is behavior like this actually supposed to be documented? For some more historical context,

the support for bytes in the json module was removed in this issue: https://bugs.python.org/issue4136

and then re-added in this one: 
https://bugs.python.org/issue17909

----------
nosy: +ammar2

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29281>
_______________________________________

From report at bugs.python.org  Wed Jan 18 17:14:02 2017
From: report at bugs.python.org (R. David Murray)
Date: Wed, 18 Jan 2017 22:14:02 +0000
Subject: [issue29281] json.loads documentation missing "versionchanged"
 statement
In-Reply-To: <1484558399.93.0.792457597374.issue29281@psf.upfronthosting.co.za>
Message-ID: <1484777642.97.0.727715900325.issue29281@psf.upfronthosting.co.za>


R. David Murray added the comment:

Yes, this should be documented with a versionchanged.  The removal was for python3 so we didn't need a versionchanged there; it was just a difference from python2, and the python3 docs were "restarted" with (almost) no back references to python2.

----------
nosy: +r.david.murray

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29281>
_______________________________________

From report at bugs.python.org  Wed Jan 18 18:33:31 2017
From: report at bugs.python.org (mark)
Date: Wed, 18 Jan 2017 23:33:31 +0000
Subject: [issue29317] test_copyxattr_symlinks fails
Message-ID: <1484782411.92.0.520036190929.issue29317@psf.upfronthosting.co.za>


New submission from mark:

======================================================================
ERROR: test_copyxattr_symlinks (test.test_shutil.TestShutil)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/Python-3.6.0/Lib/test/test_shutil.py", line 505, in test_copyxattr_symlinks
    os.setxattr(src, 'trusted.foo', b'42')
PermissionError: [Errno 1] Operation not permitted: '/tmp/tmpvlu10qdm/foo'

----------------------------------------------------------------------

The problem is that "trusted" attributes (in this case trusted.foo) are visible and accessible only to processes that have the CAP_SYS_ADMIN capability. The current "skipUnless" guard is wrong: being root (UID 0) doesn't necessarely imply you have CAP_SYS_ADMIN. For instance this test (and thus "make test") will always fail in a Docker container unless it's started with "--cap-add SYS_ADMIN" (which, in general, is not the best thing to do).

----------
components: Tests
messages: 285764
nosy: marktay
priority: normal
severity: normal
status: open
title: test_copyxattr_symlinks fails
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29317>
_______________________________________

From report at bugs.python.org  Wed Jan 18 19:39:28 2017
From: report at bugs.python.org (Berker Peksag)
Date: Thu, 19 Jan 2017 00:39:28 +0000
Subject: [issue29316] Can we keep typing.py provisional for the duration of
 the Python 3.6 release cycle?
In-Reply-To: <1484774407.47.0.689245055951.issue29316@psf.upfronthosting.co.za>
Message-ID: <1484786368.69.0.0603738834039.issue29316@psf.upfronthosting.co.za>


Changes by Berker Peksag <berker.peksag at gmail.com>:


----------
nosy: +ned.deily

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29316>
_______________________________________

From report at bugs.python.org  Wed Jan 18 19:57:26 2017
From: report at bugs.python.org (INADA Naoki)
Date: Thu, 19 Jan 2017 00:57:26 +0000
Subject: [issue29311] Argument Clinic: convert dict methods
In-Reply-To: <1484758450.31.0.717694851278.issue29311@psf.upfronthosting.co.za>
Message-ID: <1484787446.95.0.911040332872.issue29311@psf.upfronthosting.co.za>


INADA Naoki added the comment:

LGTM

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29311>
_______________________________________

From report at bugs.python.org  Wed Jan 18 20:38:56 2017
From: report at bugs.python.org (INADA Naoki)
Date: Thu, 19 Jan 2017 01:38:56 +0000
Subject: [issue29312] Use FASTCALL in dict.update()
In-Reply-To: <1484759262.53.0.625567482548.issue29312@psf.upfronthosting.co.za>
Message-ID: <1484789936.83.0.525538969006.issue29312@psf.upfronthosting.co.za>


INADA Naoki added the comment:

Using FASTCALL for methods accepting **kwargs can't skip creating dict in most cases.  Because they accepts dict.

Even worth, when calling it like `d.update(**config)` (yes, it's abuse of
**, but it can be happen in some C methods), FASTCALL may unpack the passed
dict, and pack it again.

So, when considering METH_FASTCALL, supporting **kwargs is lowest priority.
(Off course, supporting it by AC with METH_KEYWORDS is nice to have)

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29312>
_______________________________________

From report at bugs.python.org  Wed Jan 18 23:19:05 2017
From: report at bugs.python.org (Josh Rosenberg)
Date: Thu, 19 Jan 2017 04:19:05 +0000
Subject: [issue29293] Missing parameter "n" on
 multiprocessing.Condition.notify()
In-Reply-To: <1484642636.98.0.126375980974.issue29293@psf.upfronthosting.co.za>
Message-ID: <1484799545.49.0.0576972559449.issue29293@psf.upfronthosting.co.za>


Josh Rosenberg added the comment:

Looks like, despite what the multiprocessing.Condition docs say (claiming it's an alias for threading.Condition), at least in Python 3.5, it's a completely separate animal from multiprocessing.synchronize, and the notify method on it doesn't take any parameters.

Seems like an obvious thing to fix; the parameter is defaulted anyway, so existing code should continue to work, and it makes multiprocessing swap in for threading more seamlessly.

----------
nosy: +josh.r

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29293>
_______________________________________

From report at bugs.python.org  Thu Jan 19 01:48:40 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Thu, 19 Jan 2017 06:48:40 +0000
Subject: [issue29312] Use FASTCALL in dict.update()
In-Reply-To: <1484759262.53.0.625567482548.issue29312@psf.upfronthosting.co.za>
Message-ID: <1484808520.47.0.0842364750678.issue29312@psf.upfronthosting.co.za>


Raymond Hettinger added the comment:

I like the other AC changes to dict in 29311, but this one seems like it shouldn't be done.  There is too much twisting around existing code to force it to use AC and the benefit will be almost nothing.   dict.update() is mainly used with a list of tuples argument or with another mapping.  The O(1) time spent on the method call is inconsequential compared to the O(n) step of looping over all the inputs and putting them in the dict.  Accordingly, I think this method should be skipped, leaving the current clear, stable, fast-enough code in-place.

----------
nosy: +rhettinger

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29312>
_______________________________________

From report at bugs.python.org  Thu Jan 19 02:09:26 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Thu, 19 Jan 2017 07:09:26 +0000
Subject: [issue29316] Can we keep typing.py provisional for the duration of
 the Python 3.6 release cycle?
In-Reply-To: <1484774407.47.0.689245055951.issue29316@psf.upfronthosting.co.za>
Message-ID: <1484809766.32.0.372602538576.issue29316@psf.upfronthosting.co.za>


Raymond Hettinger added the comment:

+1 for keeping this provisional.  The code is still maturing and needs to be iterated while we gain experience with it.

----------
nosy: +rhettinger

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29316>
_______________________________________

From report at bugs.python.org  Thu Jan 19 04:25:21 2017
From: report at bugs.python.org (=?utf-8?q?R=C3=A9mi_Cardona?=)
Date: Thu, 19 Jan 2017 09:25:21 +0000
Subject: [issue29309] Interpreter hang when interrupting a
 loop.run_in_executor() future
In-Reply-To: <1484751513.93.0.173979168861.issue29309@psf.upfronthosting.co.za>
Message-ID: <1484817921.31.0.0965746052254.issue29309@psf.upfronthosting.co.za>


Changes by R?mi Cardona <remi2402 at free.fr>:


----------
nosy: +RemiCardona

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29309>
_______________________________________

From report at bugs.python.org  Thu Jan 19 05:03:50 2017
From: report at bugs.python.org (STINNER Victor)
Date: Thu, 19 Jan 2017 10:03:50 +0000
Subject: [issue29312] Use FASTCALL in dict.update()
In-Reply-To: <1484759262.53.0.625567482548.issue29312@psf.upfronthosting.co.za>
Message-ID: <1484820230.47.0.226616121409.issue29312@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Oops, I forgot a DECREF: fixed in the patch version 2.

--

Oh wait, I misunderstood how dict.update() is called. In fact, they are two bytecodes to call a function with keyword arguments.

(1) Using **kwargs:

>>> def f():
...  d.update(**d2)
... 
>>> dis.dis(f)
  2           0 LOAD_GLOBAL              0 (d)
              2 LOAD_ATTR                1 (update)
              4 BUILD_TUPLE              0
              6 LOAD_GLOBAL              2 (d2)
              8 CALL_FUNCTION_EX         1
             10 POP_TOP
             12 LOAD_CONST               0 (None)
             14 RETURN_VALUE


(2) Using a list of key=value:

>>> def g():
...  d.update(x=1, y=2)
... 
>>> dis.dis(g)
  2           0 LOAD_GLOBAL              0 (d)
              2 LOAD_ATTR                1 (update)
              4 LOAD_CONST               1 (1)
              6 LOAD_CONST               2 (2)
              8 LOAD_CONST               3 (('x', 'y'))
             10 CALL_FUNCTION_KW         2
             12 POP_TOP
             14 LOAD_CONST               0 (None)
             16 RETURN_VALUE


The problem is that the dict.update() method has a single implementation, the C dict_update() function.


For (2), there is a speedup, but it's minor:

$ ./python -m perf timeit -s 'd={"x": 1, "y": 2}' 'd.update(x=1, y=2)' -p10 --compare-to=../default-ref/python 
Median +- std dev: [ref] 185 ns +- 62 ns -> [patched] 177 ns +- 2 ns: 1.05x faster (-5%)


For (1), I expected that **kwargs would be unpacked *before* calling dict.update(), but kwargs is passed unchanged to dict.update() directly! With my patch, CALL_FUNCTION_EX calls PyCFunction_Call() which uses _PyStack_UnpackDict() to create kwnames and then dict_update() rebuilds a new temporary dictionary. It's completely inefficient! As Raymond expected, it's much slower:

haypo at smithers$ ./python -m perf timeit -s 'd={"x": 1, "y": 2}; d2=dict(d)' 'd.update(**d2)' -p10 --compare-to=../default-ref/python
Median +- std dev: [ref] 114 ns +- 1 ns -> [patched] 232 ns +- 21 ns: 2.04x slower (+104%)


I expect that (1) dict.update(**kwargs) is more common than (2) dict.update(x=1, y=2). Moreover, the speedup for (2) is low (5%), so I prefer to reject this issue.

--

Naoki: "So, when considering METH_FASTCALL, supporting **kwargs is lowest priority. (Off course, supporting it by AC with METH_KEYWORDS is nice to have)"

Hum, dict.update() is the first function that I found that really wants a Python dict at the end.

For dict1.update(**dict2), METH_VARARGS|METH_KEYWORDS is already optimal.

So I don't think that it's worth it to micro-optimize the way to pass positional arguments. The common case is to call dict1.update(dict2) which requires to build a temporary tuple of 1 item. PyTuple_New() uses a free list for such small tuple, so it should be fast enough.

I found a few functions which pass through keyword arguments, but they are "proxy". I'm converting all METH_VARARGS|METH_KEYWORDS to METH_FASTCALL, so most functions will expects a kwnames tuple at the end of the call for keyword arguments. In this case, using METH_FASTCALL for the proxy is optimum for func(x=1, y=2) (CALL_FUNCTION_KW), but slower for func(**kwargs) (CALL_FUNCTION_EX).

With METH_FASTCALL, CALL_FUNCTION_EX requires to unpack the dictionary if I understood correctly.

----------
status: open -> closed
Added file: http://bugs.python.org/file46334/dict_update_fastcall-2.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29312>
_______________________________________

From report at bugs.python.org  Thu Jan 19 05:05:27 2017
From: report at bugs.python.org (Evan Andrews)
Date: Thu, 19 Jan 2017 10:05:27 +0000
Subject: [issue28595] shlex.shlex should not augment wordchars
In-Reply-To: <1478166136.94.0.995190174918.issue28595@psf.upfronthosting.co.za>
Message-ID: <1484820327.18.0.206973207399.issue28595@psf.upfronthosting.co.za>


Evan Andrews added the comment:

Attaching an updated patch now that the two linked issues are resolved.

----------
Added file: http://bugs.python.org/file46335/shlex2.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28595>
_______________________________________

From report at bugs.python.org  Thu Jan 19 06:21:27 2017
From: report at bugs.python.org (Xavier de Gaye)
Date: Thu, 19 Jan 2017 11:21:27 +0000
Subject: [issue26865] Meta-issue: support of the android platform
In-Reply-To: <1461685011.99.0.617135447086.issue26865@psf.upfronthosting.co.za>
Message-ID: <1484824887.23.0.878244346789.issue26865@psf.upfronthosting.co.za>


Xavier de Gaye added the comment:

issue #28180: sys.getfilesystemencoding() should default to utf-8

----------
dependencies: +sys.getfilesystemencoding() should default to utf-8

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue26865>
_______________________________________

From report at bugs.python.org  Thu Jan 19 06:33:01 2017
From: report at bugs.python.org (STINNER Victor)
Date: Thu, 19 Jan 2017 11:33:01 +0000
Subject: [issue29318] Optimize _PyFunction_FastCallDict() for **kwargs
Message-ID: <1484825581.48.0.790581995103.issue29318@psf.upfronthosting.co.za>


New submission from STINNER Victor:

Example:
---
def func(x, y):
    print(x, y)

def proxy2(func, **kw):
    func(**kw)

def proxy1(func, **kw):
    proxy2(func, **kw)
---

The "proxy2(func, **kw)" call in proxy1() is currently inefficient: _PyFunction_FastCallDict() converts the dictionary into a C array [key1, value1, key2, value2, ...] and then _PyEval_EvalCodeWithName() rebuilds the dictionary from the C array.

Since "func(*args, **kw)" proxies are common in Python, especially to call the parent constructor when overriding __init__, I think that it would be interesting to optimize this code path.

I first expected that it was a regression of FASTCALL, but Python < 3.6 doesn't optimize this code neither.

----------
components: Interpreter Core
messages: 285773
nosy: haypo, inada.naoki
priority: normal
severity: normal
status: open
title: Optimize _PyFunction_FastCallDict() for **kwargs
type: performance
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29318>
_______________________________________

From report at bugs.python.org  Thu Jan 19 06:34:29 2017
From: report at bugs.python.org (STINNER Victor)
Date: Thu, 19 Jan 2017 11:34:29 +0000
Subject: [issue29312] Use FASTCALL in dict.update()
In-Reply-To: <1484759262.53.0.625567482548.issue29312@psf.upfronthosting.co.za>
Message-ID: <1484825669.98.0.062364263253.issue29312@psf.upfronthosting.co.za>


STINNER Victor added the comment:

When analyzing how FASTCALL handles "func(**kwargs)" calls for Python functions, I identified a missed optimization. I created the issue #29318: "Optimize _PyFunction_FastCallDict() for **kwargs".

----------
resolution:  -> rejected

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29312>
_______________________________________

From report at bugs.python.org  Thu Jan 19 06:41:13 2017
From: report at bugs.python.org (Roundup Robot)
Date: Thu, 19 Jan 2017 11:41:13 +0000
Subject: [issue29259] Add tp_fastcall to PyTypeObject: support FASTCALL
 calling convention for all callable objects
In-Reply-To: <1484310274.99.0.832491900546.issue29259@psf.upfronthosting.co.za>
Message-ID: <20170119114110.14473.68716.3FAFD457@psf.io>


Roundup Robot added the comment:

New changeset bf6728085b01 by Victor Stinner in branch 'default':
_PyStack_AsDict() now checks kwnames != NULL
https://hg.python.org/cpython/rev/bf6728085b01

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29259>
_______________________________________

From report at bugs.python.org  Thu Jan 19 06:41:13 2017
From: report at bugs.python.org (Roundup Robot)
Date: Thu, 19 Jan 2017 11:41:13 +0000
Subject: [issue29311] Argument Clinic: convert dict methods
In-Reply-To: <1484758450.31.0.717694851278.issue29311@psf.upfronthosting.co.za>
Message-ID: <20170119114110.14473.73408.C7776731@psf.io>


Roundup Robot added the comment:

New changeset 00c63ee66e0c by Victor Stinner in branch 'default':
dict.get() and dict.setdefault() now use AC
https://hg.python.org/cpython/rev/00c63ee66e0c

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29311>
_______________________________________

From report at bugs.python.org  Thu Jan 19 06:44:38 2017
From: report at bugs.python.org (INADA Naoki)
Date: Thu, 19 Jan 2017 11:44:38 +0000
Subject: [issue29318] Optimize _PyFunction_FastCallDict() for **kwargs
In-Reply-To: <1484825581.48.0.790581995103.issue29318@psf.upfronthosting.co.za>
Message-ID: <1484826278.36.0.939398527826.issue29318@psf.upfronthosting.co.za>


INADA Naoki added the comment:

Since mutating kw dict shouldn't affect caller's dict, caller and callee can't share the dict.

One possible optimization is using PyDict_Copy() to copy the dict.
It can reduce number of hash() calls.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29318>
_______________________________________

From report at bugs.python.org  Thu Jan 19 06:45:26 2017
From: report at bugs.python.org (Roundup Robot)
Date: Thu, 19 Jan 2017 11:45:26 +0000
Subject: [issue29312] Use FASTCALL in dict.update()
In-Reply-To: <1484759262.53.0.625567482548.issue29312@psf.upfronthosting.co.za>
Message-ID: <20170119114523.14859.59655.2859D9F8@psf.io>


Roundup Robot added the comment:

New changeset e371686229e7 by Victor Stinner in branch 'default':
Add a note explaining why dict_update() doesn't use METH_FASTCALL
https://hg.python.org/cpython/rev/e371686229e7

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29312>
_______________________________________

From report at bugs.python.org  Thu Jan 19 06:47:14 2017
From: report at bugs.python.org (STINNER Victor)
Date: Thu, 19 Jan 2017 11:47:14 +0000
Subject: [issue20291] Argument Clinic should understand *args and **kwargs
 parameters
In-Reply-To: <1390010125.91.0.58212727704.issue20291@psf.upfronthosting.co.za>
Message-ID: <1484826434.14.0.155014142795.issue20291@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Once this feature will be implemented, print() should be modified to use Argument Clinic: see the issue #29296.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue20291>
_______________________________________

From report at bugs.python.org  Thu Jan 19 06:51:11 2017
From: report at bugs.python.org (Roundup Robot)
Date: Thu, 19 Jan 2017 11:51:11 +0000
Subject: [issue29296] convert print() to METH_FASTCALL
In-Reply-To: <1484652871.69.0.785445814249.issue29296@psf.upfronthosting.co.za>
Message-ID: <20170119115107.6961.68495.F03985DE@psf.io>


Roundup Robot added the comment:

New changeset 0327171f05dd by INADA Naoki in branch 'default':
Issue #29296: convert print() to METH_FASTCALL
https://hg.python.org/cpython/rev/0327171f05dd

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29296>
_______________________________________

From report at bugs.python.org  Thu Jan 19 06:55:37 2017
From: report at bugs.python.org (STINNER Victor)
Date: Thu, 19 Jan 2017 11:55:37 +0000
Subject: [issue29296] convert print() to METH_FASTCALL
In-Reply-To: <1484652871.69.0.785445814249.issue29296@psf.upfronthosting.co.za>
Message-ID: <1484826937.55.0.757554779801.issue29296@psf.upfronthosting.co.za>


STINNER Victor added the comment:

I pushed print-fastcall.patch since it's simple and has a significant impact on bm_telco benchmark (especially when tp_fastcall slot will be added).

I added a reminder (msg285779) in #20291 to convert print() once AC will support **kwargs.

Can we close the issue, or do you prefer to keep it open?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29296>
_______________________________________

From report at bugs.python.org  Thu Jan 19 07:09:18 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Thu, 19 Jan 2017 12:09:18 +0000
Subject: [issue29318] Optimize _PyFunction_FastCallDict() for **kwargs
In-Reply-To: <1484825581.48.0.790581995103.issue29318@psf.upfronthosting.co.za>
Message-ID: <1484827758.79.0.845517822162.issue29318@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

> It can reduce number of hash() calls.

Keys are strings (even interned strings), and hashes are cached.

In most cases kw is empty or very small. I doubt any optimization can have significant effect.

----------
nosy: +serhiy.storchaka

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29318>
_______________________________________

From report at bugs.python.org  Thu Jan 19 07:11:31 2017
From: report at bugs.python.org (Paul Moore)
Date: Thu, 19 Jan 2017 12:11:31 +0000
Subject: [issue29319] Embedded 3.6.0 distribution cannot run pyz files
Message-ID: <1484827891.73.0.559570067048.issue29319@psf.upfronthosting.co.za>


New submission from Paul Moore:

Trying to run a pyz file using the embedded distribution for 3.6.0, I get an error "Could not import runpy module".

To reproduce, see below:

>type .\__main__.py
print('Hello, world')
>zip test.pyz __main__.py
  adding: __main__.py (172 bytes security) (stored 0%)
>unzip .\python-3.6.0-embed-amd64.zip
...
>.\python.exe .\test.pyz
Could not import runpy module
ModuleNotFoundError: No module named 'runpy'

Running with the standard interpreter works fine:

>py -V
Python 3.6.0
>py .\test.pyz
Hello, world

----------
assignee: steve.dower
components: Windows
messages: 285783
nosy: paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Embedded 3.6.0 distribution cannot run pyz files
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29319>
_______________________________________

From report at bugs.python.org  Thu Jan 19 07:13:18 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Thu, 19 Jan 2017 12:13:18 +0000
Subject: [issue29296] convert print() to METH_FASTCALL
In-Reply-To: <1484652871.69.0.785445814249.issue29296@psf.upfronthosting.co.za>
Message-ID: <1484827998.47.0.746428626193.issue29296@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

> Why can't we start with METH_FASTCALL and convert to AC later?

It increases the maintenance burden. When the fastcall protocol be changed, this would require changing more handwritten code.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29296>
_______________________________________

From report at bugs.python.org  Thu Jan 19 07:17:01 2017
From: report at bugs.python.org (Paul Moore)
Date: Thu, 19 Jan 2017 12:17:01 +0000
Subject: [issue29319] Embedded 3.6.0 distribution cannot run pyz files
In-Reply-To: <1484827891.73.0.559570067048.issue29319@psf.upfronthosting.co.za>
Message-ID: <1484828221.35.0.774659403195.issue29319@psf.upfronthosting.co.za>


Paul Moore added the comment:

I just checked, and 3.6.0b1 (the only prerelease version I had available) has the same problem. 3.5.2 works fine.

I thought I'd had similar code working during the beta cycle, but I can't demonstrate that any more, so maybe my recollection is wrong :-(

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29319>
_______________________________________

From report at bugs.python.org  Thu Jan 19 07:53:40 2017
From: report at bugs.python.org (eszense)
Date: Thu, 19 Jan 2017 12:53:40 +0000
Subject: [issue29320] bdist_msi install_script fail to execute if alt python
 location specified
Message-ID: <1484830420.95.0.711337585743.issue29320@psf.upfronthosting.co.za>


New submission from eszense:

if install-script specified and
alternative python location selected during installation,
msiexec will fail with:
    
    "There is a problem with this Windows Installer
    package. A program required for this install to
    complete could not be run. Contact your support
    personnel or package vendor.

This is due to the lack of PythonExeX Custom Action setting
the path to python.exe

Patch attacked for your consideration

----------
files: patch.txt
messages: 285786
nosy: eszense
priority: normal
pull_requests: 18
severity: normal
status: open
title: bdist_msi install_script fail to execute if alt python location specified
type: crash
Added file: http://bugs.python.org/file46336/patch.txt

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29320>
_______________________________________

From report at bugs.python.org  Thu Jan 19 07:53:55 2017
From: report at bugs.python.org (eszense)
Date: Thu, 19 Jan 2017 12:53:55 +0000
Subject: [issue29320] bdist_msi install_script fail to execute if alt python
 location specified
In-Reply-To: <1484830420.95.0.711337585743.issue29320@psf.upfronthosting.co.za>
Message-ID: <1484830435.05.0.688733948222.issue29320@psf.upfronthosting.co.za>


Changes by eszense <eszense2015 at gmail.com>:


----------
components: +Distutils
nosy: +dstufft, eric.araujo

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29320>
_______________________________________

From report at bugs.python.org  Thu Jan 19 08:04:56 2017
From: report at bugs.python.org (Arne de Laat)
Date: Thu, 19 Jan 2017 13:04:56 +0000
Subject: [issue28911] Clarify the behaviour of assert_called_once_with
In-Reply-To: <1481233700.68.0.637550951007.issue28911@psf.upfronthosting.co.za>
Message-ID: <1484831096.7.0.627293028368.issue28911@psf.upfronthosting.co.za>


Arne de Laat added the comment:

Unfortunately there cant be commas in the method name, that would clarify the name. It should be read as 'assert called once, with ...' not 'assert called once with ...'.

I have often used the method to test something was called only once, and with the specified arguments.

It seems that it is more difficult (i.e. no single method) to check if only one call was made with the specified arguments (allowing for more calls with other arguments), perhaps that could be added. Something like 'assert_once_called_with' or 'assert_one_call_with'.

----------
nosy: +153957

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28911>
_______________________________________

From report at bugs.python.org  Thu Jan 19 08:35:38 2017
From: report at bugs.python.org (eszense)
Date: Thu, 19 Jan 2017 13:35:38 +0000
Subject: [issue29313] msi by bdist_msi will fail execute install-scripts if
 space in present in python path
In-Reply-To: <1484768319.35.0.107822358832.issue29313@psf.upfronthosting.co.za>
Message-ID: <1484832938.94.0.797930349572.issue29313@psf.upfronthosting.co.za>


eszense added the comment:

This issue can be fixed together with Issue 15797.

Patch attached for your consideration

----------
Added file: http://bugs.python.org/file46337/patch.txt

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29313>
_______________________________________

From report at bugs.python.org  Thu Jan 19 08:37:27 2017
From: report at bugs.python.org (eszense)
Date: Thu, 19 Jan 2017 13:37:27 +0000
Subject: [issue29313] msi by bdist_msi will fail execute install-scripts if
 space in present in python path
In-Reply-To: <1484768319.35.0.107822358832.issue29313@psf.upfronthosting.co.za>
Message-ID: <1484833047.41.0.339581871189.issue29313@psf.upfronthosting.co.za>


Changes by eszense <eszense2015 at gmail.com>:


Removed file: http://bugs.python.org/file46337/patch2.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29313>
_______________________________________

From report at bugs.python.org  Thu Jan 19 08:38:02 2017
From: report at bugs.python.org (eszense)
Date: Thu, 19 Jan 2017 13:38:02 +0000
Subject: [issue29313] msi by bdist_msi will fail execute install-scripts if
 space in present in python path
In-Reply-To: <1484768319.35.0.107822358832.issue29313@psf.upfronthosting.co.za>
Message-ID: <1484833082.94.0.492792970443.issue29313@psf.upfronthosting.co.za>


Changes by eszense <eszense2015 at gmail.com>:


Added file: http://bugs.python.org/file46338/patch.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29313>
_______________________________________

From report at bugs.python.org  Thu Jan 19 08:38:12 2017
From: report at bugs.python.org (Paul Moore)
Date: Thu, 19 Jan 2017 13:38:12 +0000
Subject: [issue29319] Embedded 3.6.0 distribution cannot run pyz files
In-Reply-To: <1484827891.73.0.559570067048.issue29319@psf.upfronthosting.co.za>
Message-ID: <1484833092.71.0.299149348027.issue29319@psf.upfronthosting.co.za>


Paul Moore added the comment:

Confirmed that it works with alpha 2, 3 and 4. But fails with beta 1 and the release version.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29319>
_______________________________________

From report at bugs.python.org  Thu Jan 19 09:26:43 2017
From: report at bugs.python.org (RK-5wWm9h)
Date: Thu, 19 Jan 2017 14:26:43 +0000
Subject: [issue29321] Wrong documentation for unicode and str comparison
Message-ID: <1484836003.73.0.224345826816.issue29321@psf.upfronthosting.co.za>


New submission from RK-5wWm9h:

PROBLEM (IN BRIEF):

In the currently published 2.7.13 The Python Language Reference manual, section 5.9 "Comparisons" (https://docs.python.org/2/reference/expressions.html#comparisons):

    "If both are numbers, they are converted to a common type. Otherwise, objects of different types always compare unequal..."

This an *incorrect (and misleading) statement*.


PROPOSED FIX:

Insert a new sentence, to give this resulting text:

    "If both are numbers, they are converted to a common type. If one is str and the other unicode, they are compared as below. Otherwise, objects of different types always compare unequal..."


DETAILS, JUSTIFICATION, CORRECTNESS, ETC:

The behaviour that a str and a unicode object -- despite being objects of different types -- may compare equal, is explicitly stated several paragraphs later:

    "* Strings are compared lexicographically using the numeric equivalents (the result of the built-in function ord()) of their characters. Unicode and 8-bit strings are fully interoperable in this behavior. [4]"

Text in the 2.7.13 The Python Standard Library (Library Reference manual) is careful to cover this unicode - str case (https://docs.python.org/2/library/stdtypes.html#comparisons):

    "Objects of different types, except different numeric types and different string types, never compare equal; such objects are ordered consistently but arbitrarily (so that sorting a heterogeneous array yields a consistent result)."


IMPACT AND RELATED BUG:

The current incorrect text is really misleading for anyone reading the Language Ref.  It's easy to see the categorical statement and stop reading because your question has been answered.

Further, the Library ref about unicode and str (The Python Standard Library (Library Reference manual) section 5.6 "Sequence Types": https://docs.python.org/2/library/stdtypes.html#sequence-types-str-unicode-list-tuple-bytearray-buffer-xrange), links here.  Link text: "(For full details see Comparisons in the language reference.)".

(Aside: That paragraph has a mistake similar to this present bug: it says "to compare equal, every element must compare equal and the two sequences must be of the same type"; I'll file a separate bug for it.)

PS: First time reporting a Python bug; following https://docs.python.org/2/bugs.html.  Hope I did ok!  :-)

----------
assignee: docs at python
components: Documentation
messages: 285790
nosy: RK-5wWm9h, docs at python
priority: normal
severity: normal
status: open
title: Wrong documentation for unicode and str comparison
type: behavior
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29321>
_______________________________________

From report at bugs.python.org  Thu Jan 19 09:39:52 2017
From: report at bugs.python.org (Michael J)
Date: Thu, 19 Jan 2017 14:39:52 +0000
Subject: [issue29322] SimpleCV error on Raspberry Pi
Message-ID: <1484836792.51.0.0433994997007.issue29322@psf.upfronthosting.co.za>


New submission from Michael J:

Hello,
I'm using a Microsoft LifeCam with SimpleCV on the ipython interpreter(or shell) on a Raspberry Pi 3 model B(with Raspbian, tell me if you need the exact OS version as it's quite complicated). 

I'm following the directions of a book: Raspberry Pi Cookbook by Simon Monk.
I get this error:
SimpleCV:1> c = Camera()

SimpleCV:2> i = c.getImage()

SimpleCV:3> i
SimpleCV:3: <SimpleCV.Image Object size:(640, 480), filename: (None), at memory location: (0x6efdfc88)>

SimpleCV:4> 

SimpleCV:4> i.show()
---------------------------------------------------------------------------
IOError                                   Traceback (most recent call last)
/usr/local/lib/python2.7/dist-packages/SimpleCV/Shell/Shell.pyc in <module>()
----> 1 i.show()

/usr/local/lib/python2.7/dist-packages/SimpleCV/ImageClass.pyc in show(self, type)
   5445               d = Display(displaytype='notebook')
   5446           else:
-> 5447               d = Display(self.size())
   5448           self.save(d)
   5449           return d

/usr/local/lib/python2.7/dist-packages/SimpleCV/Display.pyc in __init__(self, resolution, flags, title, displaytype, headless)
    156         if not displaytype == 'notebook':
    157             self.screen = pg.display.set_mode(resolution, flags)
--> 158         scvLogo = SimpleCV.Image("simplecv").scale(32,32)
    159         pg.display.set_icon(scvLogo.getPGSurface())
    160         if flags != pg.FULLSCREEN and flags != pg.NOFRAME:

/usr/local/lib/python2.7/dist-packages/SimpleCV/ImageClass.pyc in __init__(self, source, camera, colorSpace, verbose, sample, cv2image)
    785                     self._bitmap = cv.LoadImage(self.filename, iscolor=cv.CV_LOAD_IMAGE_COLOR)
    786                 except:
--> 787                     self._pil = pil.open(self.filename).convert("RGB")
    788                     self._bitmap = cv.CreateImageHeader(self._pil.size, cv.IPL_DEPTH_8U, 3)
    789                     cv.SetData(self._bitmap, self._pil.tostring())

/usr/local/lib/python2.7/dist-packages/PIL/Image.pyc in open(fp, mode)
   2280 
   2281     if filename:
-> 2282         fp = builtins.open(filename, "rb")
   2283 
   2284     try:

IOError: [Errno 2] No such file or directory: '/usr/local/lib/python2.7/dist-packages/SimpleCV/sampleimages/simplecv.png'

I(Michael J or "MJ" as I'm commonly called) have searched for the directory myself, and I can tell you, it's not there.

Tell me if you need more information, as I will be glad to assist.

I'm sorry for any inconvenience caused by me using ipython if it's unfamiliar.

----------
components: IO
messages: 285791
nosy: MJ
priority: normal
severity: normal
status: open
title: SimpleCV error on Raspberry Pi
type: behavior
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29322>
_______________________________________

From report at bugs.python.org  Thu Jan 19 09:40:31 2017
From: report at bugs.python.org (RK-5wWm9h)
Date: Thu, 19 Jan 2017 14:40:31 +0000
Subject: [issue29323] Wrong documentation (Library) for unicode and str
 comparison
Message-ID: <1484836831.64.0.870722472975.issue29323@psf.upfronthosting.co.za>


New submission from RK-5wWm9h:

PROBLEM (IN BRIEF):
In the currently published 2.7.13 The Python Standard Library (Library Reference manual) section 5.6 "Sequence Types" (https://docs.python.org/2/library/stdtypes.html#sequence-types-str-unicode-list-tuple-bytearray-buffer-xrange):

    "to compare equal, ... the two sequences must be of the same type"

This an *incorrect (and misleading) statement*, for the unicode and str case.


PROPOSED FIX:

Current full paragraph:

    "Sequence types also support comparisons. In particular, tuples and lists are compared lexicographically by comparing corresponding elements. This means that to compare equal, every element must compare equal and the two sequences must be of the same type and have the same length. (For full details see Comparisons in the language reference.)"

Proposed replacement text:

    "Sequence types also support comparisons. In particular, tuples and lists are compared lexicographically by comparing corresponding elements. This means that to compare equal, every element must compare equal and the two sequences 
must be of the same type and have the same length. (Unicode and str are treated as the same type here; for full details see Comparisons in the language reference.)"


DETAILS, JUSTIFICATION, CORRECTNESS, ETC:

The current incorrect text is really misleading.

The behaviour that a str and a unicode object -- despite being objects of different types -- may compare equal, is explicitly stated in the 2.7.13 The Python Language Reference manual, section 5.9 "Comparisons" (https://docs.python.org/2/reference/expressions.html#comparisons):

    "* Strings are compared lexicographically using the numeric equivalents (the result of the built-in function ord()) of their characters. Unicode and 8-bit strings are fully interoperable in this behavior. [4]"

(Aside: Incidentally an earlier paragraph in the Language Ref fails to cover the unicode and str case; see separately filed bug Issue 29321.)

----------
assignee: docs at python
components: Documentation
messages: 285792
nosy: RK-5wWm9h, docs at python
priority: normal
severity: normal
status: open
title: Wrong documentation (Library) for unicode and str comparison
type: behavior
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29323>
_______________________________________

From report at bugs.python.org  Thu Jan 19 09:42:45 2017
From: report at bugs.python.org (RK-5wWm9h)
Date: Thu, 19 Jan 2017 14:42:45 +0000
Subject: [issue29321] Wrong documentation (Language Ref) for unicode and str
 comparison
In-Reply-To: <1484836003.73.0.224345826816.issue29321@psf.upfronthosting.co.za>
Message-ID: <1484836965.86.0.450062397586.issue29321@psf.upfronthosting.co.za>


Changes by RK-5wWm9h <rkistruc at brocade.com>:


----------
title: Wrong documentation for unicode and str comparison -> Wrong documentation (Language Ref) for unicode and str comparison

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29321>
_______________________________________

From report at bugs.python.org  Thu Jan 19 09:45:23 2017
From: report at bugs.python.org (Stuart Berg)
Date: Thu, 19 Jan 2017 14:45:23 +0000
Subject: [issue21628] 2to3 does not fix zip in some cases
In-Reply-To: <1401643673.43.0.6506799869.issue21628@psf.upfronthosting.co.za>
Message-ID: <1484837123.6.0.146230289697.issue21628@psf.upfronthosting.co.za>


Stuart Berg added the comment:

Already closed, but FWIW, I think this was incorrectly marked as a duplicate.  Issue 20742 discusses a different issue related to lib2to3 and zip.

Meanwhile, this issue has been raised again in 28837, so I will continue the discussion there.  (I have a patch.)

----------
nosy: +stuarteberg

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue21628>
_______________________________________

From report at bugs.python.org  Thu Jan 19 10:01:06 2017
From: report at bugs.python.org (Christian Heimes)
Date: Thu, 19 Jan 2017 15:01:06 +0000
Subject: [issue29324] test_aead_aes_gcm fails on Kernel 4.9
Message-ID: <1484838066.61.0.93462051202.issue29324@psf.upfronthosting.co.za>


New submission from Christian Heimes:

$ ./python -m test -m test_aead_aes_gcm test_socket
Run tests sequentially
0:00:00 [1/1] test_socket
test test_socket failed -- Traceback (most recent call last):
  File "/home/heimes/dev/python/cpython/Lib/test/support/__init__.py", line 556, in wrapper
    return func(*args, **kw)
  File "/home/heimes/dev/python/cpython/Lib/test/test_socket.py", line 5515, in test_aead_aes_gcm
    res = op.recv(assoclen + len(plain) + taglen)
OSError: [Errno 22] Invalid argument

The tests were written and passed under Linux Kernel 4.7. I was under the assumption that the API is stable. But the most recent version 4.9 has changed the API for AEAD mode slightly. libkcapi project maintain a well written documentation of the Kernel crypto API,
https://github.com/smuellerDD/libkcapi/commit/be242c387b7030cbccae2c183107efa86d9a3cd6

Fedora 24 and 25 are affected. I'm going to update the tests. Downstream distributors: Feel free to disable the test in the mean time. The feature is not critical.

----------
assignee: christian.heimes
components: Extension Modules
messages: 285794
nosy: christian.heimes
priority: normal
severity: normal
status: open
title: test_aead_aes_gcm fails on Kernel 4.9
type: behavior
versions: Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29324>
_______________________________________

From report at bugs.python.org  Thu Jan 19 10:01:08 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Thu, 19 Jan 2017 15:01:08 +0000
Subject: [issue29322] SimpleCV error on Raspberry Pi
In-Reply-To: <1484836792.51.0.0433994997007.issue29322@psf.upfronthosting.co.za>
Message-ID: <1484838068.04.0.190605073814.issue29322@psf.upfronthosting.co.za>


Xiang Zhang added the comment:

Hi Michael, this bug tracker is meant for developing CPython. Your problem exists in your own codes or third party applications. You'd better report to the related parties.

----------
nosy: +xiang.zhang
resolution:  -> third party
stage:  -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29322>
_______________________________________

From report at bugs.python.org  Thu Jan 19 10:03:17 2017
From: report at bugs.python.org (Charalampos Stratakis)
Date: Thu, 19 Jan 2017 15:03:17 +0000
Subject: [issue29324] test_aead_aes_gcm fails on Kernel 4.9
In-Reply-To: <1484838066.61.0.93462051202.issue29324@psf.upfronthosting.co.za>
Message-ID: <1484838197.55.0.08956450628.issue29324@psf.upfronthosting.co.za>


Changes by Charalampos Stratakis <cstratak at redhat.com>:


----------
nosy: +cstratak

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29324>
_______________________________________

From report at bugs.python.org  Thu Jan 19 10:08:08 2017
From: report at bugs.python.org (Stuart Berg)
Date: Thu, 19 Jan 2017 15:08:08 +0000
Subject: [issue28837] 2to3 does not wrap zip correctly
In-Reply-To: <1480503898.29.0.0471565347052.issue28837@psf.upfronthosting.co.za>
Message-ID: <1484838488.69.0.267796428457.issue28837@psf.upfronthosting.co.za>


Stuart Berg added the comment:

In addition to zip(), this problem also affects map() and filter().

The problem is that the match patterns in FixZip, FixMap, and FixFilter do not allow for more than one "trailer" node.  (And even if they did, their transform() methods aren't expecting it.)

For example, in the following expression, 'zip' is followed by two 'trailers', which are '(a,b)', and [0]:

zip(a,b)[0]

... but FixZip.PATTERN only expects a single trailer (the argument list), so the presence of a second trailer prevents the match: https://git.io/vMDP9


(Here's the relevant line of the grammar: https://git.io/vMDPJ)

I've written a patch that fixes this problem for zip, map, and filter, with tests.  See attached.

BTW, this problem was previously reported in 21628, but that issue was incorrectly closed as a duplicate, so I'm submitting my patch here.

----------
keywords: +patch
nosy: +stuarteberg
Added file: http://bugs.python.org/file46339/fix-28837.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28837>
_______________________________________

From report at bugs.python.org  Thu Jan 19 10:39:01 2017
From: report at bugs.python.org (Christian Heimes)
Date: Thu, 19 Jan 2017 15:39:01 +0000
Subject: [issue29324] test_aead_aes_gcm fails on Kernel 4.9
In-Reply-To: <1484838066.61.0.93462051202.issue29324@psf.upfronthosting.co.za>
Message-ID: <1484840341.92.0.605342910924.issue29324@psf.upfronthosting.co.za>


Christian Heimes added the comment:

recv(64) works. I need to figure out why 64 and what's in the extra bytes.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29324>
_______________________________________

From report at bugs.python.org  Thu Jan 19 10:44:41 2017
From: report at bugs.python.org (Roundup Robot)
Date: Thu, 19 Jan 2017 15:44:41 +0000
Subject: [issue20186] Derby #18: Convert 31 sites to Argument Clinic across 23
 files
In-Reply-To: <1389140586.13.0.921060348978.issue20186@psf.upfronthosting.co.za>
Message-ID: <20170119154437.14473.18641.B669169F@psf.io>


Roundup Robot added the comment:

New changeset 3713f7de576d by Serhiy Storchaka in branch 'default':
Issue #20186: Converted the _operator module to Argument Clinic.
https://hg.python.org/cpython/rev/3713f7de576d

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue20186>
_______________________________________

From report at bugs.python.org  Thu Jan 19 10:46:05 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Thu, 19 Jan 2017 15:46:05 +0000
Subject: [issue20186] Derby #18: Convert 31 sites to Argument Clinic across 23
 files
In-Reply-To: <1389140586.13.0.921060348978.issue20186@psf.upfronthosting.co.za>
Message-ID: <1484840765.36.0.65988972331.issue20186@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

The only change I made is used the return converter in length_hint().

----------
versions: +Python 3.7 -Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue20186>
_______________________________________

From report at bugs.python.org  Thu Jan 19 10:49:59 2017
From: report at bugs.python.org (Steve Dower)
Date: Thu, 19 Jan 2017 15:49:59 +0000
Subject: [issue29319] Embedded 3.6.0 distribution cannot run pyz files
In-Reply-To: <1484827891.73.0.559570067048.issue29319@psf.upfronthosting.co.za>
Message-ID: <1484840999.59.0.917392184351.issue29319@psf.upfronthosting.co.za>


Steve Dower added the comment:

Does running with -v provide any more hints?

I'm also interested in whether the alphas work when you don't have a full install on the same machine. PC/getpathp.c changed for b1. But that's probably less important.

Perhaps runpy just never made it into the python36.zip file?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29319>
_______________________________________

From report at bugs.python.org  Thu Jan 19 11:08:24 2017
From: report at bugs.python.org (Paul Moore)
Date: Thu, 19 Jan 2017 16:08:24 +0000
Subject: [issue29319] Embedded 3.6.0 distribution cannot run pyz files
In-Reply-To: <1484827891.73.0.559570067048.issue29319@psf.upfronthosting.co.za>
Message-ID: <1484842104.05.0.649509275529.issue29319@psf.upfronthosting.co.za>


Paul Moore added the comment:

Sorry I should have thought of trying -v. The output (included below) doesn't seem to offer many hints, though. runpy.pyc is in python36.zip, I checked that.

I'll see if I can find a machine without Python installed to test that case.

>.\python.exe -v .\test.pyz
import _frozen_importlib # frozen
import _imp # builtin
import sys # builtin
import '_warnings' # <class '_frozen_importlib.BuiltinImporter'>
import '_thread' # <class '_frozen_importlib.BuiltinImporter'>
import '_weakref' # <class '_frozen_importlib.BuiltinImporter'>
import '_frozen_importlib_external' # <class '_frozen_importlib.FrozenImporter'>
import '_io' # <class '_frozen_importlib.BuiltinImporter'>
import 'marshal' # <class '_frozen_importlib.BuiltinImporter'>
import 'nt' # <class '_frozen_importlib.BuiltinImporter'>
import _thread # previously loaded ('_thread')
import '_thread' # <class '_frozen_importlib.BuiltinImporter'>
import _weakref # previously loaded ('_weakref')
import '_weakref' # <class '_frozen_importlib.BuiltinImporter'>
import 'winreg' # <class '_frozen_importlib.BuiltinImporter'>
# installing zipimport hook
import 'zipimport' # <class '_frozen_importlib.BuiltinImporter'>
# installed zipimport hook
# zipimport: found 609 names in 'C:\\Work\\Scratch\\test\\python36.zip'
import 'zlib' # <class '_frozen_importlib.BuiltinImporter'>
# zipimport: zlib available
# zipimport: zlib available
# zipimport: zlib available
# zipimport: zlib available
import '_codecs' # <class '_frozen_importlib.BuiltinImporter'>
import codecs # loaded from Zip C:\Work\Scratch\test\python36.zip\codecs.pyc
# zipimport: zlib available
# zipimport: zlib available
import encodings.aliases # loaded from Zip C:\Work\Scratch\test\python36.zip\encodings\aliases.pyc
import encodings # loaded from Zip C:\Work\Scratch\test\python36.zip\encodings\__init__.pyc
# zipimport: zlib available
# zipimport: zlib available
import encodings.utf_8 # loaded from Zip C:\Work\Scratch\test\python36.zip\encodings\utf_8.pyc
import '_signal' # <class '_frozen_importlib.BuiltinImporter'>
# zipimport: zlib available
# zipimport: zlib available
import encodings.latin_1 # loaded from Zip C:\Work\Scratch\test\python36.zip\encodings\latin_1.pyc
# zipimport: zlib available
# zipimport: zlib available
# zipimport: zlib available
# zipimport: zlib available
# zipimport: zlib available
# zipimport: zlib available
import _weakrefset # loaded from Zip C:\Work\Scratch\test\python36.zip\_weakrefset.pyc
import abc # loaded from Zip C:\Work\Scratch\test\python36.zip\abc.pyc
import io # loaded from Zip C:\Work\Scratch\test\python36.zip\io.pyc
Python 3.6.0 (v3.6.0:41df79263a11, Dec 23 2016, 08:06:12) [MSC v.1900 64 bit (AMD64)] on win32
# zipimport: zlib available
# zipimport: zlib available
import encodings.cp437 # loaded from Zip C:\Work\Scratch\test\python36.zip\encodings\cp437.pyc
# zipimport: found 1 names in '.\\test.pyz'
Could not import runpy module
Traceback (most recent call last):
  File "<frozen importlib._bootstrap>", line 961, in _find_and_load
  File "<frozen importlib._bootstrap>", line 948, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'runpy'
# clear builtins._
# clear sys.path
# clear sys.argv
# clear sys.ps1
# clear sys.ps2
# clear sys.last_type
# clear sys.last_value
# clear sys.last_traceback
# clear sys.path_hooks
# clear sys.path_importer_cache
# clear sys.meta_path
# clear sys.__interactivehook__
# clear sys.flags
# clear sys.float_info
# restore sys.stdin
# restore sys.stdout
# restore sys.stderr
# cleanup[2] removing builtins
# cleanup[2] removing sys
# cleanup[2] removing _frozen_importlib
# cleanup[2] removing _imp
# cleanup[2] removing _warnings
# cleanup[2] removing _thread
# cleanup[2] removing _weakref
# cleanup[2] removing _frozen_importlib_external
# cleanup[2] removing _io
# cleanup[2] removing marshal
# cleanup[2] removing nt
# cleanup[2] removing winreg
# cleanup[2] removing zipimport
# cleanup[2] removing zlib
# cleanup[2] removing encodings
# destroy encodings
# cleanup[2] removing codecs
# cleanup[2] removing _codecs
# cleanup[2] removing encodings.aliases
# cleanup[2] removing encodings.utf_8
# cleanup[2] removing _signal
# cleanup[2] removing __main__
# destroy __main__
# cleanup[2] removing encodings.latin_1
# cleanup[2] removing io
# destroy io
# cleanup[2] removing abc
# destroy abc
# cleanup[2] removing _weakrefset
# destroy _weakrefset
# cleanup[2] removing encodings.cp437
# destroy zipimport
# destroy zlib
# destroy _signal
# cleanup[3] wiping _frozen_importlib
# destroy _frozen_importlib_external
# cleanup[3] wiping _imp
# cleanup[3] wiping _warnings
# cleanup[3] wiping _thread
# cleanup[3] wiping _weakref
# cleanup[3] wiping _io
# cleanup[3] wiping marshal
# cleanup[3] wiping nt
# cleanup[3] wiping winreg
# cleanup[3] wiping codecs
# cleanup[3] wiping _codecs
# cleanup[3] wiping encodings.aliases
# cleanup[3] wiping encodings.utf_8
# cleanup[3] wiping encodings.latin_1
# cleanup[3] wiping encodings.cp437
# cleanup[3] wiping sys
# cleanup[3] wiping builtins
# destroy _imp
# destroy io
# destroy _warnings
# destroy marshal
# destroy nt
# destroy _thread
# destroy _weakref
# destroy winreg
# destroy _frozen_importlib

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29319>
_______________________________________

From report at bugs.python.org  Thu Jan 19 11:13:39 2017
From: report at bugs.python.org (Roundup Robot)
Date: Thu, 19 Jan 2017 16:13:39 +0000
Subject: [issue20186] Derby #18: Convert 31 sites to Argument Clinic across 23
 files
In-Reply-To: <1389140586.13.0.921060348978.issue20186@psf.upfronthosting.co.za>
Message-ID: <20170119161332.62768.91717.FDD5CD30@psf.io>


Roundup Robot added the comment:

New changeset 112f27b8c8ea by Serhiy Storchaka in branch 'default':
Issue #20186: Converted the math module to Argument Clinic.
https://hg.python.org/cpython/rev/112f27b8c8ea

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue20186>
_______________________________________

From report at bugs.python.org  Thu Jan 19 11:15:06 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Thu, 19 Jan 2017 16:15:06 +0000
Subject: [issue20186] Derby #18: Convert 31 sites to Argument Clinic across 23
 files
In-Reply-To: <1389140586.13.0.921060348978.issue20186@psf.upfronthosting.co.za>
Message-ID: <1484842506.13.0.804076091399.issue20186@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

issue20186.mathmodule.v2.patch needed just synchronizing docstrings.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue20186>
_______________________________________

From report at bugs.python.org  Thu Jan 19 11:22:29 2017
From: report at bugs.python.org (Martin Panter)
Date: Thu, 19 Jan 2017 16:22:29 +0000
Subject: [issue12067] Doc: remove errors about mixed-type comparisons.
In-Reply-To: <1305237573.86.0.646542413513.issue12067@psf.upfronthosting.co.za>
Message-ID: <1484842949.46.0.908727644325.issue12067@psf.upfronthosting.co.za>


Changes by Martin Panter <vadmium+py at gmail.com>:


----------
dependencies: +Wrong documentation (Language Ref) for unicode and str comparison

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue12067>
_______________________________________

From report at bugs.python.org  Thu Jan 19 11:30:50 2017
From: report at bugs.python.org (Brian Vandenberg)
Date: Thu, 19 Jan 2017 16:30:50 +0000
Subject: [issue29325] pysqlite: Evaluate removal of sqlite3_stmt_readonly
Message-ID: <1484843450.51.0.0079411502064.issue29325@psf.upfronthosting.co.za>


New submission from Brian Vandenberg:

I'm not sure where to request changes to pysqlite, so my apologies if this isn't the right place.

To begin with: I'll either end up building a newer version of sqlite myself or just accepting that pysqlite won't be part of this python installation.  However, I thought it might be useful to know that use of the function "sqlite3_stmt_readonly" is the only thing tying pysqlite to the current minimum requirement to use sqlite 3.7.4.

The currently available 'supported' version of sqlite for RHEL6 is 3.6.x and there's likely others out there who (for whatever reason) are stuck on an older release of sqlite and moving to the latest & greatest OS isn't [currently] feasible.

----------
components: Extension Modules
messages: 285804
nosy: phantal
priority: normal
severity: normal
status: open
title: pysqlite: Evaluate removal of sqlite3_stmt_readonly
type: enhancement

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29325>
_______________________________________

From report at bugs.python.org  Thu Jan 19 11:31:33 2017
From: report at bugs.python.org (Martin Panter)
Date: Thu, 19 Jan 2017 16:31:33 +0000
Subject: [issue29321] Wrong documentation (Language Ref) for unicode and str
 comparison
In-Reply-To: <1484836003.73.0.224345826816.issue29321@psf.upfronthosting.co.za>
Message-ID: <1484843493.0.0.556913284804.issue29321@psf.upfronthosting.co.za>


Martin Panter added the comment:

The Python 3 version of this was rewritten in Issue 12067. It would be good to port the new text to the Python 2 version, although that is not straightforward because of various differences between Python 2 and 3.

That doesn?t rule out making smaller more specific edits in the mean time. However your proposal still seems misleading to only mention str and unicode as special cases. It does not allow for str vs bytearray, set vs frozenset, or custom classes/types implementing their own __eq__() etc methods.

----------
nosy: +martin.panter

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29321>
_______________________________________

From report at bugs.python.org  Thu Jan 19 11:46:49 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Thu, 19 Jan 2017 16:46:49 +0000
Subject: [issue20186] Derby #18: Convert 31 sites to Argument Clinic across 23
 files
In-Reply-To: <1389140586.13.0.921060348978.issue20186@psf.upfronthosting.co.za>
Message-ID: <1484844409.39.0.879182903325.issue20186@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

issue20186.enumobject.patch LGTM too.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue20186>
_______________________________________

From report at bugs.python.org  Thu Jan 19 11:48:40 2017
From: report at bugs.python.org (Roundup Robot)
Date: Thu, 19 Jan 2017 16:48:40 +0000
Subject: [issue20186] Derby #18: Convert 31 sites to Argument Clinic across 23
 files
In-Reply-To: <1389140586.13.0.921060348978.issue20186@psf.upfronthosting.co.za>
Message-ID: <20170119164836.62481.89583.DEDC02C7@psf.io>


Roundup Robot added the comment:

New changeset e3db9bccff3f by Serhiy Storchaka in branch 'default':
Issue #20186: Converted builtins enumerate() and reversed() to Argument Clinic.
https://hg.python.org/cpython/rev/e3db9bccff3f

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue20186>
_______________________________________

From report at bugs.python.org  Thu Jan 19 11:49:52 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Thu, 19 Jan 2017 16:49:52 +0000
Subject: [issue20186] Derby #18: Convert 31 sites to Argument Clinic across 23
 files
In-Reply-To: <1389140586.13.0.921060348978.issue20186@psf.upfronthosting.co.za>
Message-ID: <1484844592.45.0.203474960597.issue20186@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

Sorry for committing patches for you Tal, but they were hanging so long time.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue20186>
_______________________________________

From report at bugs.python.org  Thu Jan 19 11:54:32 2017
From: report at bugs.python.org (Steve Dower)
Date: Thu, 19 Jan 2017 16:54:32 +0000
Subject: [issue29326] Blank lines in ._pth file are not ignored
Message-ID: <1484844872.11.0.819349056222.issue29326@psf.upfronthosting.co.za>


New submission from Steve Dower:

If a python._pth file includes a blank line, it gets treated as '\n' which is then appended to the directory and used as an entry in sys.path.

Empty lines should be ignored completely.

----------
components: Windows
messages: 285809
nosy: paul.moore, steve.dower, tim.golden, zach.ware
priority: low
severity: normal
stage: needs patch
status: open
title: Blank lines in ._pth file are not ignored
type: behavior
versions: Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29326>
_______________________________________

From report at bugs.python.org  Thu Jan 19 11:54:54 2017
From: report at bugs.python.org (Xavier de Gaye)
Date: Thu, 19 Jan 2017 16:54:54 +0000
Subject: [issue13886] readline-related test_builtin failure
In-Reply-To: <1327659819.71.0.475160045664.issue13886@psf.upfronthosting.co.za>
Message-ID: <1484844894.71.0.733739713658.issue13886@psf.upfronthosting.co.za>


Xavier de Gaye added the comment:

With input-readline.v3.patch, test_builtin runs with success on Android api 21.

With pep538_coerce_legacy_c_locale_v4.diff that implements PEP 538 in issue 28180, and with input-readline.v3.patch modified to have 'readline_encoding = locale.getpreferredencoding()' even when 'is_android' is True, test_builtin runs with success. This means that no specific Android handling would be needed if PEP 538 were to be adopted.

The new input-readline.v4.patch is a slight improvement over the previous patch and sets readline_encoding to 'UTF-8' on Android when test_builtin is run with the environment variable LANG set to 'en_US.UTF-8' and in that case the test exercises all the code paths including those with the readline module.  This is because locale.getdefaultlocale() returns ('en_US', 'UTF-8') on Android in that case and because both getdefaultlocale() and readline scan the environment to check for the locale.  This new patch is only useful if tests on Android are expected to be run also with one of the locale environment variables set to UTF-8 (and PEP 538 is rejected).

I have left some comments on Rietveld.

----------
Added file: http://bugs.python.org/file46340/input-readline.v4.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue13886>
_______________________________________

From report at bugs.python.org  Thu Jan 19 11:58:40 2017
From: report at bugs.python.org (Steve Dower)
Date: Thu, 19 Jan 2017 16:58:40 +0000
Subject: [issue29319] Embedded 3.6.0 distribution cannot run pyz files
In-Reply-To: <1484827891.73.0.559570067048.issue29319@psf.upfronthosting.co.za>
Message-ID: <1484845120.06.0.403235005965.issue29319@psf.upfronthosting.co.za>


Steve Dower added the comment:

I just tried it and it makes no difference.

Omitting the "._pth" file seems to fix it, but apart from issue29326 (benign) sys.path is fine.

Still digging, but don't worry about testing on other machines.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29319>
_______________________________________

From report at bugs.python.org  Thu Jan 19 12:00:59 2017
From: report at bugs.python.org (Roundup Robot)
Date: Thu, 19 Jan 2017 17:00:59 +0000
Subject: [issue29311] Argument Clinic: convert dict methods
In-Reply-To: <1484758450.31.0.717694851278.issue29311@psf.upfronthosting.co.za>
Message-ID: <20170119170055.6497.78133.EAFE84D9@psf.io>


Roundup Robot added the comment:

New changeset fe1d83fe29d6 by Serhiy Storchaka in branch 'default':
Issue #29311: Argument Clinic generates reasonable name for the parameter "default".
https://hg.python.org/cpython/rev/fe1d83fe29d6

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29311>
_______________________________________

From report at bugs.python.org  Thu Jan 19 12:02:26 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Thu, 19 Jan 2017 17:02:26 +0000
Subject: [issue29311] Argument Clinic: convert dict methods
In-Reply-To: <1484758450.31.0.717694851278.issue29311@psf.upfronthosting.co.za>
Message-ID: <1484845346.05.0.348015233303.issue29311@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

There are same problems with docstrings as in OrderedDict. The name "D" is not defined.

----------
nosy: +serhiy.storchaka

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29311>
_______________________________________

From report at bugs.python.org  Thu Jan 19 12:14:57 2017
From: report at bugs.python.org (Steve Dower)
Date: Thu, 19 Jan 2017 17:14:57 +0000
Subject: [issue29319] Embedded 3.6.0 distribution cannot run pyz files
In-Reply-To: <1484827891.73.0.559570067048.issue29319@psf.upfronthosting.co.za>
Message-ID: <1484846097.99.0.435362846525.issue29319@psf.upfronthosting.co.za>


Steve Dower added the comment:

Found it in Modules/main.c in RunMainFromImporter():

    /* argv0 is usable as an import source, so put it in sys.path[0]
       and import __main__ */
    sys_path = PySys_GetObject("path");
    if (sys_path == NULL) {
        PyErr_SetString(PyExc_RuntimeError, "unable to get sys.path");
        goto error;
    }
    if (PyList_SetItem(sys_path, 0, argv0)) {
        argv0 = NULL;
        goto error;
    }
    Py_INCREF(argv0);

When running with a ._pth file, we force the -I option, which removes the empty entry at sys.path[0]. Instead, it becomes the path to the .zip file with the standard library, so when RunMainFromImporter overwrites it blindly, it's cutting out the only hope it has of finding runpy.

You can see this in a normal install by doing:

    py -I -c "import sys; print(sys.path)"
    [correct output]

    py -Ii test.pyz
    [output from test.pyz]
    >>> import sys; sys.path
    [incorrect output]

So we need to stop blindly overwriting sys.path[0] here. I'm not sure what the best approach would be, but maybe Nick has a preference? Perhaps we should pass the zip file path into runpy._run_module_as_main so it can initialize __path__ with it rather than making a global change? Or maybe an insert rather than a set is the right way and I'm over-thinking this.

----------
assignee: steve.dower -> 
nosy: +ncoghlan
stage:  -> needs patch
versions: +Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29319>
_______________________________________

From report at bugs.python.org  Thu Jan 19 12:28:06 2017
From: report at bugs.python.org (R. David Murray)
Date: Thu, 19 Jan 2017 17:28:06 +0000
Subject: [issue29323] Wrong documentation (Library) for unicode and str
 comparison
In-Reply-To: <1484836831.64.0.870722472975.issue29323@psf.upfronthosting.co.za>
Message-ID: <1484846886.17.0.226166231676.issue29323@psf.upfronthosting.co.za>


R. David Murray added the comment:

Unicode and string *are* of the same type: basestring.  This is a specific example of the liskov substitution principle, so I don't think it should be called out explicitly in this section.

----------
nosy: +r.david.murray

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29323>
_______________________________________

From report at bugs.python.org  Thu Jan 19 12:30:53 2017
From: report at bugs.python.org (R. David Murray)
Date: Thu, 19 Jan 2017 17:30:53 +0000
Subject: [issue29323] Wrong documentation (Library) for unicode and str
 comparison
In-Reply-To: <1484836831.64.0.870722472975.issue29323@psf.upfronthosting.co.za>
Message-ID: <1484847053.39.0.482365580339.issue29323@psf.upfronthosting.co.za>


R. David Murray added the comment:

As per your other issue, though, the real issue is that the two objects must be *comparable*, not that they be of the same type, and the language should probably be updated to reflect that.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29323>
_______________________________________

From report at bugs.python.org  Thu Jan 19 12:31:08 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Thu, 19 Jan 2017 17:31:08 +0000
Subject: [issue29327] SystemError in sorted(iterable=[])
Message-ID: <1484847068.03.0.460461228187.issue29327@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka:

In Python 3.6:

>>> sorted(iterable=[])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
SystemError: Objects/tupleobject.c:81: bad argument to internal function

In Python 3.5 and 2.7:

>>> sorted(iterable=[])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'iterable' is an invalid keyword argument for this function

----------
components: Interpreter Core
keywords: 3.6regression
messages: 285817
nosy: serhiy.storchaka
priority: normal
severity: normal
status: open
title: SystemError in sorted(iterable=[])
type: behavior
versions: Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29327>
_______________________________________

From report at bugs.python.org  Thu Jan 19 12:31:17 2017
From: report at bugs.python.org (STINNER Victor)
Date: Thu, 19 Jan 2017 17:31:17 +0000
Subject: [issue29311] Argument Clinic: convert dict methods
In-Reply-To: <1484758450.31.0.717694851278.issue29311@psf.upfronthosting.co.za>
Message-ID: <1484847077.27.0.103117463265.issue29311@psf.upfronthosting.co.za>


STINNER Victor added the comment:

> Argument Clinic generates reasonable name for the parameter "default".
> -    default as failobj: object = None
> +    default: object = None

Thanks, that's a better name :-)

FYI I wanted to limit changes when converting to AC, the change is already big enough and hard to review.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29311>
_______________________________________

From report at bugs.python.org  Thu Jan 19 12:33:32 2017
From: report at bugs.python.org (STINNER Victor)
Date: Thu, 19 Jan 2017 17:33:32 +0000
Subject: [issue29311] Argument Clinic: convert dict methods
In-Reply-To: <1484758450.31.0.717694851278.issue29311@psf.upfronthosting.co.za>
Message-ID: <1484847212.35.0.809871320217.issue29311@psf.upfronthosting.co.za>


STINNER Victor added the comment:

> There are same problems with docstrings as in OrderedDict. The name "D" is not defined.

I copied the old docstring to AC.

Python 3.6 doc:
---
get(...)
    D.get(k[,d]) -> D[k] if k in D, else d.  d defaults to None.
---

D was already implicitly "self".


Python 3.7 (new) doc:
---
get(self, key, default=None, /)
    D.get(key[, default]) -> D[key] if key in D, else default.
---

What do you propose? Use self?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29311>
_______________________________________

From report at bugs.python.org  Thu Jan 19 12:37:46 2017
From: report at bugs.python.org (Stuart Berg)
Date: Thu, 19 Jan 2017 17:37:46 +0000
Subject: [issue28837] 2to3 does not wrap zip correctly
In-Reply-To: <1480503898.29.0.0471565347052.issue28837@psf.upfronthosting.co.za>
Message-ID: <1484847466.73.0.756782154776.issue28837@psf.upfronthosting.co.za>


Changes by Stuart Berg <bergs at janelia.hhmi.org>:


Removed file: http://bugs.python.org/file46339/fix-28837.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28837>
_______________________________________

From report at bugs.python.org  Thu Jan 19 12:38:35 2017
From: report at bugs.python.org (Roundup Robot)
Date: Thu, 19 Jan 2017 17:38:35 +0000
Subject: [issue29289] Convert OrderedDict methods to Argument Clinic
In-Reply-To: <1484609084.15.0.406928886689.issue29289@psf.upfronthosting.co.za>
Message-ID: <20170119173830.111581.76536.ADE4170F@psf.io>


Roundup Robot added the comment:

New changeset c144bf6c0ff7 by Serhiy Storchaka in branch 'default':
Issue #29289: Argument Clinic generates reasonable name for the parameter "default".
https://hg.python.org/cpython/rev/c144bf6c0ff7

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29289>
_______________________________________

From report at bugs.python.org  Thu Jan 19 12:39:29 2017
From: report at bugs.python.org (Stuart Berg)
Date: Thu, 19 Jan 2017 17:39:29 +0000
Subject: [issue28837] 2to3 does not wrap zip correctly
In-Reply-To: <1480503898.29.0.0471565347052.issue28837@psf.upfronthosting.co.za>
Message-ID: <1484847569.76.0.703676491051.issue28837@psf.upfronthosting.co.za>


Changes by Stuart Berg <bergs at janelia.hhmi.org>:


Added file: http://bugs.python.org/file46341/fix-28837.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28837>
_______________________________________

From report at bugs.python.org  Thu Jan 19 12:50:42 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Thu, 19 Jan 2017 17:50:42 +0000
Subject: [issue29311] Argument Clinic: convert dict methods
In-Reply-To: <1484758450.31.0.717694851278.issue29311@psf.upfronthosting.co.za>
Message-ID: <1484848242.44.0.65799031478.issue29311@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

There was a context in old docstrings removed in new docstrings.

It is not always possible just to copy docstrings in Argument Clinic. Sometimes rewriting docstrings is the hardest part of converting to Argument Clinic.

I suggest either restore the context that defines D, or better totally rewrite docstrings, getting the wording from the documentation.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29311>
_______________________________________

From report at bugs.python.org  Thu Jan 19 12:51:02 2017
From: report at bugs.python.org (Paul Moore)
Date: Thu, 19 Jan 2017 17:51:02 +0000
Subject: [issue29319] Embedded 3.6.0 distribution cannot run pyz files
In-Reply-To: <1484827891.73.0.559570067048.issue29319@psf.upfronthosting.co.za>
Message-ID: <1484848262.11.0.484701440326.issue29319@psf.upfronthosting.co.za>


Paul Moore added the comment:

Nice! Thanks for finding this. I don't suppose there's any chance this would qualify as a bugfix for 3.6.1? I've been holding off on working on https://github.com/pfmoore/pylaunch until 3.6 because 3.5 doesn't handle being in a subdirectory very well. It'd be a shame if this meant I could only really support 3.7+

But it is a pretty minor use case, so I'll live either way.

FWIW, I'm inclined to think we shouldn't be blindly overwriting things, at the very least we should check that what we're overwriting is what we expect. But my knowledge of this part of the interpreter is pretty much zero, so I'll defer to the experts.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29319>
_______________________________________

From report at bugs.python.org  Thu Jan 19 12:53:39 2017
From: report at bugs.python.org (Stuart Berg)
Date: Thu, 19 Jan 2017 17:53:39 +0000
Subject: [issue28837] 2to3 does not wrap zip correctly
In-Reply-To: <1480503898.29.0.0471565347052.issue28837@psf.upfronthosting.co.za>
Message-ID: <1484848419.06.0.186647556484.issue28837@psf.upfronthosting.co.za>


Stuart Berg added the comment:

Sorry for re-uploading the patch; I made some pep8 fixes.

----------
Added file: http://bugs.python.org/file46342/fix-28837.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28837>
_______________________________________

From report at bugs.python.org  Thu Jan 19 12:53:48 2017
From: report at bugs.python.org (Stuart Berg)
Date: Thu, 19 Jan 2017 17:53:48 +0000
Subject: [issue28837] 2to3 does not wrap zip correctly
In-Reply-To: <1480503898.29.0.0471565347052.issue28837@psf.upfronthosting.co.za>
Message-ID: <1484848428.12.0.173292844103.issue28837@psf.upfronthosting.co.za>


Changes by Stuart Berg <bergs at janelia.hhmi.org>:


Removed file: http://bugs.python.org/file46341/fix-28837.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28837>
_______________________________________

From report at bugs.python.org  Thu Jan 19 12:55:06 2017
From: report at bugs.python.org (Brett Cannon)
Date: Thu, 19 Jan 2017 17:55:06 +0000
Subject: [issue29316] Can we keep typing.py provisional for the duration of
 the Python 3.6 release cycle?
In-Reply-To: <1484774407.47.0.689245055951.issue29316@psf.upfronthosting.co.za>
Message-ID: <1484848506.94.0.289213585147.issue29316@psf.upfronthosting.co.za>


Brett Cannon added the comment:

Keeping it provisional sounds reasonable to me, especially if it helps land protocol support.

----------
nosy: +brett.cannon

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29316>
_______________________________________

From report at bugs.python.org  Thu Jan 19 12:58:01 2017
From: report at bugs.python.org (Steve Dower)
Date: Thu, 19 Jan 2017 17:58:01 +0000
Subject: [issue29319] Embedded 3.6.0 distribution cannot run pyz files
In-Reply-To: <1484827891.73.0.559570067048.issue29319@psf.upfronthosting.co.za>
Message-ID: <1484848681.5.0.771410209649.issue29319@psf.upfronthosting.co.za>


Steve Dower added the comment:

I'd say it definitely qualifies as a bug fix, even in 3.5 (which repros), assuming we don't break any existing APIs in the process.

----------
versions: +Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29319>
_______________________________________

From report at bugs.python.org  Thu Jan 19 13:01:51 2017
From: report at bugs.python.org (Stuart Berg)
Date: Thu, 19 Jan 2017 18:01:51 +0000
Subject: [issue28837] 2to3 does not wrap zip correctly
In-Reply-To: <1480503898.29.0.0471565347052.issue28837@psf.upfronthosting.co.za>
Message-ID: <1484848911.57.0.288386525449.issue28837@psf.upfronthosting.co.za>


Changes by Stuart Berg <bergs at janelia.hhmi.org>:


Removed file: http://bugs.python.org/file46342/fix-28837.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28837>
_______________________________________

From report at bugs.python.org  Thu Jan 19 13:02:26 2017
From: report at bugs.python.org (Stuart Berg)
Date: Thu, 19 Jan 2017 18:02:26 +0000
Subject: [issue28837] 2to3 does not wrap zip correctly
In-Reply-To: <1480503898.29.0.0471565347052.issue28837@psf.upfronthosting.co.za>
Message-ID: <1484848946.81.0.748027592793.issue28837@psf.upfronthosting.co.za>


Changes by Stuart Berg <bergs at janelia.hhmi.org>:


Added file: http://bugs.python.org/file46343/fix-28837.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28837>
_______________________________________

From report at bugs.python.org  Thu Jan 19 13:03:36 2017
From: report at bugs.python.org (Brett Cannon)
Date: Thu, 19 Jan 2017 18:03:36 +0000
Subject: [issue2771] Test issue
In-Reply-To: <1210005645.74.0.283923986194.issue2771@psf.upfronthosting.co.za>
Message-ID: <1484849016.83.0.648896003881.issue2771@psf.upfronthosting.co.za>


Changes by Brett Cannon <brett at python.org>:


----------
pull_requests: +19

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue2771>
_______________________________________

From report at bugs.python.org  Thu Jan 19 13:08:46 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Thu, 19 Jan 2017 18:08:46 +0000
Subject: [issue29327] SystemError in sorted(iterable=[])
In-Reply-To: <1484847068.03.0.460461228187.issue29327@psf.upfronthosting.co.za>
Message-ID: <1484849326.24.0.149153020809.issue29327@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

Seems this regression was introduced in issue27809.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29327>
_______________________________________

From report at bugs.python.org  Thu Jan 19 13:11:24 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Thu, 19 Jan 2017 18:11:24 +0000
Subject: [issue29327] SystemError in sorted(iterable=[])
In-Reply-To: <1484847068.03.0.460461228187.issue29327@psf.upfronthosting.co.za>
Message-ID: <1484849484.93.0.6820706927.issue29327@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

In debug build Python is crashed.

>>> sorted(iterable=[])
python: Objects/abstract.c:2317: _PyObject_FastCallDict: Assertion `nargs >= 0' failed.
Aborted (core dumped)

----------
type: behavior -> crash

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29327>
_______________________________________

From report at bugs.python.org  Thu Jan 19 13:24:32 2017
From: report at bugs.python.org (Elizabeth Myers)
Date: Thu, 19 Jan 2017 18:24:32 +0000
Subject: [issue29328] struct module should support variable-length strings
Message-ID: <1484850272.25.0.472309940141.issue29328@psf.upfronthosting.co.za>


New submission from Elizabeth Myers:

There was some discussion on python-ideas about this, and I figured it would be more productive to bring it here since to me this appears to be a glaring omission.

The struct module has no capability to support variable-length strings; this includes null-terminated and Pascal-ish strings with a different integer datatype (usually in binary) specifying length.

This unfortunate omission makes the struct module extremely unwieldy to use in situations where you need to unpack a lot of variable-length strings, especially iteratively; see https://mail.python.org/pipermail/python-ideas/2017-January/044328.html for why. For zero-terminated strings, it is essentially impossible.

It's worth noting many modern protocols use variable-length strings, including DHCP.

I therefore propose the following extensions to the struct module (details can be bikeshedded over :P):

- Z (uppercase) format specifier (I did not invent this idea, see https://github.com/stendec/netstruct - although that uses $), which states the preceding whole-number datatype is the length of a string that follows.
- z (lowercase) format specifier, which specifies a null-terminated (also known as C style) string. An optional length parameter can be added to specify the maximum search length.

These two additions will make the struct module much more usable in a wider variety of contexts.

----------
components: Library (Lib)
messages: 285828
nosy: Elizacat
priority: normal
severity: normal
status: open
title: struct module should support variable-length strings
type: enhancement
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29328>
_______________________________________

From report at bugs.python.org  Thu Jan 19 13:29:16 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Thu, 19 Jan 2017 18:29:16 +0000
Subject: [issue29327] SystemError or crash in sorted(iterable=[])
In-Reply-To: <1484847068.03.0.460461228187.issue29327@psf.upfronthosting.co.za>
Message-ID: <1484850556.67.0.59169702446.issue29327@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

Proposed patch fixes the bug.

----------
keywords: +patch
nosy: +haypo
stage:  -> patch review
title: SystemError in sorted(iterable=[]) -> SystemError or crash in sorted(iterable=[])
Added file: http://bugs.python.org/file46344/sorted.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29327>
_______________________________________

From report at bugs.python.org  Thu Jan 19 13:38:35 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Thu, 19 Jan 2017 18:38:35 +0000
Subject: [issue29328] struct module should support variable-length strings
In-Reply-To: <1484850272.25.0.472309940141.issue29328@psf.upfronthosting.co.za>
Message-ID: <1484851115.01.0.102687029481.issue29328@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

Could you provide some examples of using these format specifiers? I suppose that due to limitations of the struct module the way in which they can be implemented would be not particularly useful for you.

----------
nosy: +serhiy.storchaka

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29328>
_______________________________________

From report at bugs.python.org  Thu Jan 19 13:49:49 2017
From: report at bugs.python.org (Thomas Kluyver)
Date: Thu, 19 Jan 2017 18:49:49 +0000
Subject: [issue13285] signal module ignores external signal changes
In-Reply-To: <1319801278.56.0.946220269602.issue13285@psf.upfronthosting.co.za>
Message-ID: <1484851789.77.0.767448156201.issue13285@psf.upfronthosting.co.za>


Thomas Kluyver added the comment:

I'd like to make the case for a fix in the code again. Our use case is, I believe, the same as Vilya's. We want to temporarily set a signal handler from Python and then restore the previous handler. This is fairly straightforward for Python handler functions, and SIG_DFL and SIG_IGN, but it breaks if anything has set a C level signal handler.

The opaque wrapper object is a solution that had occurred to me too. Another option would be a context manager implemented in C (I assume context managers can be written in C) which can set one or more signal handlers on entry, and restore them on exit.

----------
nosy: +takluyver

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue13285>
_______________________________________

From report at bugs.python.org  Thu Jan 19 13:55:38 2017
From: report at bugs.python.org (=?utf-8?q?Pekka_Kl=C3=A4rck?=)
Date: Thu, 19 Jan 2017 18:55:38 +0000
Subject: [issue29329] Incorrect documentation for custom `hex()` support on
 Python 2
Message-ID: <1484852138.82.0.311515909069.issue29329@psf.upfronthosting.co.za>


New submission from Pekka Kl?rck:

Documentation of `hex()` on Python 2 says that custom objects need to implement `__index__` to support it. Based on my tests that doesn't work but `__hex__` is needed instead. Docs are at 
https://docs.python.org/2/library/functions.html?highlight=hex#hex and here's an example session:

Python 2.7.6 (default, Oct 26 2016, 20:30:19) 
[GCC 4.8.4] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> class Hex(object):
...     def __index__(self):
...         return 255
... 
>>> hex(Hex())
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: hex() argument can't be converted to hex
>>> 
>>> class Hex(object):
...     def __hex__(self):
...         return hex(255)
... 
>>> hex(Hex())
'0xff'


 
Assuming this is fixed, should probably note that with Python 3 you actually *need* to implement `__index__` and `__hex__` has no effect.

----------
messages: 285832
nosy: pekka.klarck
priority: normal
severity: normal
status: open
title: Incorrect documentation for custom `hex()` support on Python 2

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29329>
_______________________________________

From report at bugs.python.org  Thu Jan 19 14:04:59 2017
From: report at bugs.python.org (Ethan Furman)
Date: Thu, 19 Jan 2017 19:04:59 +0000
Subject: [issue29328] struct module should support variable-length strings
In-Reply-To: <1484850272.25.0.472309940141.issue29328@psf.upfronthosting.co.za>
Message-ID: <1484852699.44.0.73257543307.issue29328@psf.upfronthosting.co.za>


Changes by Ethan Furman <ethan at stoneleaf.us>:


----------
nosy: +ethan.furman

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29328>
_______________________________________

From report at bugs.python.org  Thu Jan 19 14:16:17 2017
From: report at bugs.python.org (Tal Einat)
Date: Thu, 19 Jan 2017 19:16:17 +0000
Subject: [issue20186] Derby #18: Convert 31 sites to Argument Clinic across 23
 files
In-Reply-To: <1389140586.13.0.921060348978.issue20186@psf.upfronthosting.co.za>
Message-ID: <1484853377.67.0.0336140259835.issue20186@psf.upfronthosting.co.za>


Tal Einat added the comment:

Serhiy, no apology is required. On the contrary, thank you for the taking the time to review this and commit, I don't have time available for this these days.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue20186>
_______________________________________

From report at bugs.python.org  Thu Jan 19 14:41:53 2017
From: report at bugs.python.org (Mark Dickinson)
Date: Thu, 19 Jan 2017 19:41:53 +0000
Subject: [issue29328] struct module should support variable-length strings
In-Reply-To: <1484850272.25.0.472309940141.issue29328@psf.upfronthosting.co.za>
Message-ID: <1484854913.11.0.906796993858.issue29328@psf.upfronthosting.co.za>


Mark Dickinson added the comment:

IMO, as one of the previous maintainers of the struct module, this feature request isn't compatible with the current design and purpose of the struct module. I agree that there's an important problem to solve (and one I've had to solve many times for various formats in consulting work); it's simply that the struct module isn't the right place to solve it.

----------
nosy: +mark.dickinson

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29328>
_______________________________________

From report at bugs.python.org  Thu Jan 19 14:44:51 2017
From: report at bugs.python.org (Eryk Sun)
Date: Thu, 19 Jan 2017 19:44:51 +0000
Subject: [issue29329] Incorrect documentation for custom `hex()` support on
 Python 2
In-Reply-To: <1484852138.82.0.311515909069.issue29329@psf.upfronthosting.co.za>
Message-ID: <1484855091.05.0.350299971111.issue29329@psf.upfronthosting.co.za>


Eryk Sun added the comment:

Python 3 uses __index__ for bin(), oct(), and hex(), but Python 2 only uses __index__ for bin() and otherwise uses __oct__ and __hex__. Antoine overlooked this when updating the 2.7 docs for hex() in issue 16665.

----------
nosy: +eryksun

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29329>
_______________________________________

From report at bugs.python.org  Thu Jan 19 14:44:53 2017
From: report at bugs.python.org (Mark Dickinson)
Date: Thu, 19 Jan 2017 19:44:53 +0000
Subject: [issue29282] Fused multiply-add: proposal to add math.fma()
In-Reply-To: <1484560897.19.0.0955404368118.issue29282@psf.upfronthosting.co.za>
Message-ID: <1484855093.43.0.0488418505997.issue29282@psf.upfronthosting.co.za>


Mark Dickinson added the comment:

Serhiy, Victor: thanks for the reviews. Here's a new patch. Differences w.r.t. the old one:

- Converted to argument clinic.
- Updated docstring to talk about special cases.
- More tests, as suggested by Serhiy.
- whatsnew entry and ..versionadded in docs.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29282>
_______________________________________

From report at bugs.python.org  Thu Jan 19 14:46:16 2017
From: report at bugs.python.org (Mark Dickinson)
Date: Thu, 19 Jan 2017 19:46:16 +0000
Subject: [issue29282] Fused multiply-add: proposal to add math.fma()
In-Reply-To: <1484560897.19.0.0955404368118.issue29282@psf.upfronthosting.co.za>
Message-ID: <1484855176.2.0.35262989149.issue29282@psf.upfronthosting.co.za>


Mark Dickinson added the comment:

Whoops; looks like I failed to attach the updated patch. Here it is.

----------
Added file: http://bugs.python.org/file46345/math_fma2.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29282>
_______________________________________

From report at bugs.python.org  Thu Jan 19 14:46:59 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Thu, 19 Jan 2017 19:46:59 +0000
Subject: [issue29329] Incorrect documentation for custom `hex()` support on
 Python 2
In-Reply-To: <1484852138.82.0.311515909069.issue29329@psf.upfronthosting.co.za>
Message-ID: <1484855219.99.0.857018256284.issue29329@psf.upfronthosting.co.za>


Changes by Serhiy Storchaka <storchaka+cpython at gmail.com>:


----------
assignee:  -> docs at python
components: +Documentation
keywords: +easy
nosy: +docs at python
stage:  -> needs patch
versions: +Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29329>
_______________________________________

From report at bugs.python.org  Thu Jan 19 14:48:04 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Thu, 19 Jan 2017 19:48:04 +0000
Subject: [issue29282] Fused multiply-add: proposal to add math.fma()
In-Reply-To: <1484560897.19.0.0955404368118.issue29282@psf.upfronthosting.co.za>
Message-ID: <1484855284.47.0.184379871706.issue29282@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

Have you missed the patch?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29282>
_______________________________________

From report at bugs.python.org  Thu Jan 19 14:48:43 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Thu, 19 Jan 2017 19:48:43 +0000
Subject: [issue29282] Fused multiply-add: proposal to add math.fma()
In-Reply-To: <1484560897.19.0.0955404368118.issue29282@psf.upfronthosting.co.za>
Message-ID: <1484855323.65.0.289784369281.issue29282@psf.upfronthosting.co.za>


Changes by Serhiy Storchaka <storchaka+cpython at gmail.com>:


----------
Removed message: http://bugs.python.org/msg285838

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29282>
_______________________________________

From report at bugs.python.org  Thu Jan 19 15:00:44 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Thu, 19 Jan 2017 20:00:44 +0000
Subject: [issue29282] Fused multiply-add: proposal to add math.fma()
In-Reply-To: <1484560897.19.0.0955404368118.issue29282@psf.upfronthosting.co.za>
Message-ID: <1484856044.15.0.0403960654998.issue29282@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

LGTM except that lines in What's New are too long.

----------
assignee:  -> mark.dickinson
stage: patch review -> commit review

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29282>
_______________________________________

From report at bugs.python.org  Thu Jan 19 15:03:34 2017
From: report at bugs.python.org (Mark Dickinson)
Date: Thu, 19 Jan 2017 20:03:34 +0000
Subject: [issue29282] Fused multiply-add: proposal to add math.fma()
In-Reply-To: <1484560897.19.0.0955404368118.issue29282@psf.upfronthosting.co.za>
Message-ID: <1484856214.03.0.393156018691.issue29282@psf.upfronthosting.co.za>


Mark Dickinson added the comment:

> lines in What's New are too long.

Thanks. Fixed (I think). I'm not sure what the limit is, but the lines are now all <= 79 characters long.

----------
Added file: http://bugs.python.org/file46346/math_fma3.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29282>
_______________________________________

From report at bugs.python.org  Thu Jan 19 15:04:38 2017
From: report at bugs.python.org (Mark Dickinson)
Date: Thu, 19 Jan 2017 20:04:38 +0000
Subject: [issue29282] Fused multiply-add: proposal to add math.fma()
In-Reply-To: <1484560897.19.0.0955404368118.issue29282@psf.upfronthosting.co.za>
Message-ID: <1484856278.25.0.20477125464.issue29282@psf.upfronthosting.co.za>


Mark Dickinson added the comment:

Ah, the dev guide says 80 characters. (https://docs.python.org/devguide/documenting.html)

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29282>
_______________________________________

From report at bugs.python.org  Thu Jan 19 15:12:05 2017
From: report at bugs.python.org (R. David Murray)
Date: Thu, 19 Jan 2017 20:12:05 +0000
Subject: [issue13285] signal module ignores external signal changes
In-Reply-To: <1319801278.56.0.946220269602.issue13285@psf.upfronthosting.co.za>
Message-ID: <1484856725.57.0.313098155166.issue13285@psf.upfronthosting.co.za>


R. David Murray added the comment:

IMO the signal handler context manager would be useful (I have existing code where I wrote one that didn't quite work right :).  I suggest you propose this on python-ideas.

----------
nosy: +r.david.murray

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue13285>
_______________________________________

From report at bugs.python.org  Thu Jan 19 15:13:18 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Thu, 19 Jan 2017 20:13:18 +0000
Subject: [issue29282] Fused multiply-add: proposal to add math.fma()
In-Reply-To: <1484560897.19.0.0955404368118.issue29282@psf.upfronthosting.co.za>
Message-ID: <1484856798.51.0.150828179018.issue29282@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

Then LGTM unconditionally.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29282>
_______________________________________

From report at bugs.python.org  Thu Jan 19 15:23:13 2017
From: report at bugs.python.org (Mark Dickinson)
Date: Thu, 19 Jan 2017 20:23:13 +0000
Subject: [issue29282] Fused multiply-add: proposal to add math.fma()
In-Reply-To: <1484560897.19.0.0955404368118.issue29282@psf.upfronthosting.co.za>
Message-ID: <1484857393.71.0.741165752579.issue29282@psf.upfronthosting.co.za>


Mark Dickinson added the comment:

Thanks, Serhiy. I'm going to hold off committing this for 24 hours or so, because I want to follow the buildbots when I do (and I don't have time for that right now). I wouldn't be at all surprised to see platform-specific test failures.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29282>
_______________________________________

From report at bugs.python.org  Thu Jan 19 16:10:46 2017
From: report at bugs.python.org (Martin Panter)
Date: Thu, 19 Jan 2017 21:10:46 +0000
Subject: [issue29311] Argument Clinic: convert dict methods
In-Reply-To: <1484758450.31.0.717694851278.issue29311@psf.upfronthosting.co.za>
Message-ID: <1484860246.54.0.426160578362.issue29311@psf.upfronthosting.co.za>


Martin Panter added the comment:

D.get(key[, default]) -> D[key] if key in D, else default.

There is no big problem with that. D is defined at the start. The only thing I would have suggested is avoid using square brackets to mean two things in the one expression. Since it is no longer the signature, calling with both parameters should be okay:

'''
get($self, key, default=None, /)
--
D.get(key, default) -> D[key] if key in D, else default.
'''

However the other method no longer defines D:

'''
setdefault($self, key, default=None, /)
--
D.get(key,default), also set D[key]=default if key not in D.
'''

You could restore the initial text as ?D.setdefault(key,default) ->?, or maybe rewrite it like

?Like get(), but also insert the default value if it is missing.?

----------
nosy: +martin.panter

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29311>
_______________________________________

From report at bugs.python.org  Thu Jan 19 16:15:08 2017
From: report at bugs.python.org (Ethan Furman)
Date: Thu, 19 Jan 2017 21:15:08 +0000
Subject: [issue29328] struct module should support variable-length strings
In-Reply-To: <1484850272.25.0.472309940141.issue29328@psf.upfronthosting.co.za>
Message-ID: <1484860508.95.0.65188263131.issue29328@psf.upfronthosting.co.za>


Ethan Furman added the comment:

>From Yury Selivanov:
-------------------
This is a neat idea, but this will only work for parsing framed
binary protocols.  For example, if you protocol prefixes all packets
with a length field, you can write an efficient read buffer and
use your proposal to decode all of message's fields in one shot.
Which is good.

Not all protocols use framing though.  For instance, your proposal
won't help to write Thrift or Postgres protocols parsers.

Overall, I'm not sure that this is worth the hassle.  With proposal:

   data, = struct.unpack('!H$', buf)
   buf = buf[2+len(data):]

with the current struct module:

   len, = struct.unpack('!H', buf)
   data = buf[2:2+len]
   buf = buf[2+len:]

Another thing: struct.calcsize won't work with structs that use
variable length fields.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29328>
_______________________________________

From report at bugs.python.org  Thu Jan 19 16:46:34 2017
From: report at bugs.python.org (STINNER Victor)
Date: Thu, 19 Jan 2017 21:46:34 +0000
Subject: [issue29296] convert print() to METH_FASTCALL
In-Reply-To: <1484827998.47.0.746428626193.issue29296@psf.upfronthosting.co.za>
Message-ID: <CAMpsgwbcz21CGj7z5sbgpWtJFwos10ctjGZdMmxz3iz5kW+o_Q@mail.gmail.com>


STINNER Victor added the comment:

Serhiy Storchaka:
> When the fastcall protocol be changed, this would require changing more handwritten code.

I don't plan to modify the fastcall protocol. I'm not sure that it's
really possible to modify it anymore. It would probably be simpler to
add a new protocol. But it may become a little bit annoying to have to
support so many calling convention :-) So someone should come up with
a serious rationale to add another one :-)

I tried to make fastcall as private as possible, but recently I
noticed that METH_FASTCALL is now public in Python 3.6 API (but
hopefully excluded from the stable ABI). I know that it's already used
by Cython since 0.25:
https://mail.python.org/pipermail/cython-devel/2016-October/004959.html

About the maintenance burden, I'm ok to handle it :-) My final goal is
to use Argument Clinic everywhere. While converting existing code to
AC is a slow process, I consider (from my recent experiences with AC)
that the new code is easier to maintain! It's a slow process beause AC
still has limitaitons, but also because it's used an opportunity to
review (and enhance!) the existing docstrings.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29296>
_______________________________________

From report at bugs.python.org  Thu Jan 19 16:55:08 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Thu, 19 Jan 2017 21:55:08 +0000
Subject: [issue29327] SystemError or crash in sorted(iterable=[])
In-Reply-To: <1484847068.03.0.460461228187.issue29327@psf.upfronthosting.co.za>
Message-ID: <1484862908.52.0.0911566786715.issue29327@psf.upfronthosting.co.za>


Raymond Hettinger added the comment:

Please be careful with all of these AC changes.  They need to have tests.  They need to not change APIs.  They need to not introduce bugs.  The whole effort seems to be being pushed forward in a cavalier and aggressive manner.

In this case, there was an API change and bugs introduced for basically zero benefit.

----------
nosy: +rhettinger
priority: normal -> high

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29327>
_______________________________________

From report at bugs.python.org  Thu Jan 19 17:20:43 2017
From: report at bugs.python.org (STINNER Victor)
Date: Thu, 19 Jan 2017 22:20:43 +0000
Subject: [issue29327] SystemError or crash in sorted(iterable=[])
In-Reply-To: <1484847068.03.0.460461228187.issue29327@psf.upfronthosting.co.za>
Message-ID: <1484864443.52.0.664068494114.issue29327@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Oh, this issue is very subtle.

Since the list.sorted() class method became a builtin sorted() method (Python 2.4.0, change c06b570adf12 by Raymond Hettinger), the sorted() function accepts an iterable as a keyword argument, whereas list.sort() doesn't. sorted(iterable=[]) fails on calling internally list.sort(iterable=[]), not when sorted() parses its arguments.

The change 6219aa966a5f (issue #20184) converted sorted() to Argument Clinic. This change was released in Python 3.5.3 and 3.6.0... but it didn't introduce the bug. It's not the fault of Argument Clinic!

The change b34d2ef5c412 (issue #27809) replacing METH_VARARGS|METH_KEYWORDS with METH_FASTCALL didn't introduce the bug neither.

It's the change 15eab21bf934 (issue #27809) which replaced PyEval_CallObjectWithKeywords() with _PyObject_FastCallDict(). This change also replaces PyTuple_GetSlice(args, 1, argc) with &PyTuple_GET_ITEM(args, 1) which introduced the bug. I didn't notice that args can be an empty tuple. I never tried to call sorted(iterable=seq), I didn't know the name of the first parameter :-)

I also replaced PyTuple_GetSlice(args, 1, argc) with &PyTuple_GET_ITEM(args, 1) in methoddescr_call(), but this function make sure that we have at least one positional argument and so doesn't have this bug. Moreover, this method doesn't use Argument Clinic (yet? ;-)). I'm quite sure that I didn't replace PyTuple_GetSlice() in other functions.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29327>
_______________________________________

From report at bugs.python.org  Thu Jan 19 17:30:40 2017
From: report at bugs.python.org (saumitra paul)
Date: Thu, 19 Jan 2017 22:30:40 +0000
Subject: [issue29330] __slots__ needs documentation
Message-ID: <1484865040.1.0.923136956691.issue29330@psf.upfronthosting.co.za>


New submission from saumitra paul:

As you said..assiging it to you :)

----------
messages: 285850
nosy: rhettinger, saumitra1978
priority: normal
severity: normal
status: open
title: __slots__ needs documentation
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29330>
_______________________________________

From report at bugs.python.org  Thu Jan 19 17:34:58 2017
From: report at bugs.python.org (STINNER Victor)
Date: Thu, 19 Jan 2017 22:34:58 +0000
Subject: [issue29327] SystemError or crash in sorted(iterable=[])
In-Reply-To: <1484847068.03.0.460461228187.issue29327@psf.upfronthosting.co.za>
Message-ID: <1484865298.19.0.384278619782.issue29327@psf.upfronthosting.co.za>


STINNER Victor added the comment:

While Python 3.5 doesn't crash, I consider that it has also the bug. So I added Python 3.5 in Versions.

sorted.diff LGTM. And thank you for the unit test!

----------
versions: +Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29327>
_______________________________________

From report at bugs.python.org  Thu Jan 19 17:36:08 2017
From: report at bugs.python.org (Martin Panter)
Date: Thu, 19 Jan 2017 22:36:08 +0000
Subject: [issue29323] Wrong documentation (Library) for unicode and str
 comparison
In-Reply-To: <1484836831.64.0.870722472975.issue29323@psf.upfronthosting.co.za>
Message-ID: <1484865368.31.0.826436488155.issue29323@psf.upfronthosting.co.za>


Martin Panter added the comment:

If you read the whole paragraph carefully, I don't think it is too misleading. "In particular, tuples and lists . . ." suggests the author was just trying to say that a tuple never compares equal to a list. Maybe we just need to make that more obvious?

However there are other problems in this part of the reference about comparing different types. See Issue 22000, about the earlier section on Comparisons of built-in types.

----------
nosy: +martin.panter

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29323>
_______________________________________

From report at bugs.python.org  Thu Jan 19 17:47:23 2017
From: report at bugs.python.org (Maxime Belanger)
Date: Thu, 19 Jan 2017 22:47:23 +0000
Subject: [issue27596] Build failure with Xcode 8 beta on OSX 10.11
In-Reply-To: <1469270983.6.0.702583698233.issue27596@psf.upfronthosting.co.za>
Message-ID: <1484866043.66.0.798291197096.issue27596@psf.upfronthosting.co.za>


Maxime Belanger added the comment:

We're hitting this issue with Python 2.7 (which we deploy on Mac OS X 10.6-10.12). We've worked around it by manually patching `pyconfig.h` to un-define `HAVE_GETENTROPY` before `make`ing. Is a patch is in the works to support weak-linking this symbol (and the other new Sierra additions)?

----------
nosy: +Maxime Belanger

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue27596>
_______________________________________

From report at bugs.python.org  Thu Jan 19 17:48:05 2017
From: report at bugs.python.org (Martin Panter)
Date: Thu, 19 Jan 2017 22:48:05 +0000
Subject: [issue29330] __slots__ needs documentation
In-Reply-To: <1484865040.1.0.923136956691.issue29330@psf.upfronthosting.co.za>
Message-ID: <1484866085.3.0.686201795195.issue29330@psf.upfronthosting.co.za>


Martin Panter added the comment:

Have you seen <https://docs.python.org/2.7/reference/datamodel.html#slots>? There is also <https://docs.python.org/2.7/glossary.html#term-slots>.

----------
assignee:  -> docs at python
components: +Documentation
nosy: +docs at python, martin.panter

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29330>
_______________________________________

From report at bugs.python.org  Thu Jan 19 17:50:38 2017
From: report at bugs.python.org (STINNER Victor)
Date: Thu, 19 Jan 2017 22:50:38 +0000
Subject: [issue29327] SystemError or crash in sorted(iterable=[])
In-Reply-To: <1484847068.03.0.460461228187.issue29327@psf.upfronthosting.co.za>
Message-ID: <1484866238.12.0.125124474938.issue29327@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Raymond: "Please be careful with all of these AC changes.  They need to have tests.  They need to not change APIs.  They need to not introduce bugs."

The bug was not introduced by an Argument Clinic change.

I agree that switching to AC must not change the API.

I didn't look much at the "recent" AC changes (it started somethings like 2 years ago, no?), but it seems like the most common trap are default values of optional positional arguments. Sometimes, there are inconsistencies between .rst doc (in Doc/ directory), the docstring and the C implementation. The trap is when the default is documented as None, the C code uses NULL and passing None behaves differently...


Raymond: "The whole effort seems to be being pushed forward in a cavalier and aggressive manner. In this case, there was an API change and bugs introduced for basically zero benefit."

I converted a few functions to AC. I used regular reviews for that, since I noticed that it's easy to make mistakes. My hope is that the AC conversion will also help to fix inconsistencies.

The benefit of switching to AC is a better docstring and a correct signature for inspect.signature(func). Python 3.5:
---
sorted(...)
    sorted(iterable, key=None, reverse=False) --> new sorted list
---
versus Python 3.7
---
sorted(iterable, key=None, reverse=False)
    Return a new list containing all items from the iterable in ascending order.
    
    A custom key function can be supplied to customize the sort order, and the
    reverse flag can be set to request the result in descending order.
---

IHMO Python 3.7 docstring looks better. (Sadly, sorted() doesn't use AC yet, the "AC code" was genereted manually. AC should support **kwargs, issue #20291.)



I guess "cavalier" and "aggressive" is more related to my FASTCALL work. I'm waiting for reviews for major API changes. I agree that I pushed a lot of code without reviews when I considered that the change was safe and simple. It became hard to get a review, there are too few reviewers, especially on the C code.

The benefit of FASTCALL are better performances. I'm trying to provide benchmarks whenever possible, but since I modified dozens of functions, sometimes I didn't publish all benchmark results (sometimes even to not spam an issue).


Microbenchmark on sorted() on Python 3.7 compared to 3.5 (before FASTCALL):
---
haypo at smithers$ ./python -m perf timeit 'seq=list(range(10))' 'sorted(seq)' --compare-to=../3.5/python -v
Median +- std dev: [3.5] 1.07 us +- 0.06 us -> [3.7] 958 ns +- 15 ns: 1.12x faster (-11%)

haypo at smithers$ ./python -m perf timeit 'seq=list(range(10)); k=lambda x:x' 'sorted(seq, key=k)' --compare-to=../3.5/python -v
Median +- std dev: [3.5] 3.34 us +- 0.07 us -> [3.7] 2.66 us +- 0.05 us: 1.26x faster (-21%)
---

It's not easy nor interesting to measure the speedup of the changes limited to sorted(). Sadly, FASTCALL requires to modify a lot of code to show its full power. Otherwise, the gain is much smaller.


The latest major "API change" was made by you 14 years ago. Yes, I introduced a bug, and I'm sorry about that. Shit happens.

Let's be more constructive: to avoid bugs, the best is to get reviews. I have dozens of patches waiting for your review, so please come to help me to spot bugs before releases ;-)

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29327>
_______________________________________

From report at bugs.python.org  Thu Jan 19 17:54:14 2017
From: report at bugs.python.org (saumitra paul)
Date: Thu, 19 Jan 2017 22:54:14 +0000
Subject: [issue29330] __slots__ needs documentation
In-Reply-To: <1484866085.3.0.686201795195.issue29330@psf.upfronthosting.co.za>
Message-ID: <CANjQuE9VMj746h2g9jdM6NH-PmZkKwnv+Ura2xr7jfEeFFHzmA@mail.gmail.com>


saumitra paul added the comment:

Python 2.7.10 (default, Oct 23 2015, 19:19:21)

[GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.0.59.5)] on darwin

Type "help", "copyright", "credits" or "license" for more information.

>>> help(__slots__)

Traceback (most recent call last):

  File "<stdin>", line 1, in <module>

NameError: name '__slots__' is not defined

>>> help('__slots__')

no Python documentation found for '__slots__'

>>>

On Thu, Jan 19, 2017 at 2:48 PM, Martin Panter <report at bugs.python.org>
wrote:

>
> Martin Panter added the comment:
>
> Have you seen <https://docs.python.org/2.7/reference/datamodel.html#slots>?
> There is also <https://docs.python.org/2.7/glossary.html#term-slots>.
>
> ----------
> assignee:  -> docs at python
> components: +Documentation
> nosy: +docs at python, martin.panter
>
> _______________________________________
> Python tracker <report at bugs.python.org>
> <http://bugs.python.org/issue29330>
> _______________________________________
>

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29330>
_______________________________________

From report at bugs.python.org  Thu Jan 19 18:02:30 2017
From: report at bugs.python.org (paul j3)
Date: Thu, 19 Jan 2017 23:02:30 +0000
Subject: [issue9625] argparse: Problem with defaults for variable nargs when
 using choices
In-Reply-To: <1282036744.37.0.964764969496.issue9625@psf.upfronthosting.co.za>
Message-ID: <1484866950.94.0.251534693589.issue9625@psf.upfronthosting.co.za>


Changes by paul j3 <ajipanca at gmail.com>:


----------
priority: normal -> high

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue9625>
_______________________________________

From report at bugs.python.org  Thu Jan 19 18:03:18 2017
From: report at bugs.python.org (paul j3)
Date: Thu, 19 Jan 2017 23:03:18 +0000
Subject: [issue9338] argparse optionals with nargs='?',
 '*' or '+' can't be followed by positionals
In-Reply-To: <1279881989.28.0.725806934086.issue9338@psf.upfronthosting.co.za>
Message-ID: <1484866998.72.0.156335355903.issue9338@psf.upfronthosting.co.za>


Changes by paul j3 <ajipanca at gmail.com>:


----------
priority: normal -> high

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue9338>
_______________________________________

From report at bugs.python.org  Thu Jan 19 18:10:31 2017
From: report at bugs.python.org (Ned Deily)
Date: Thu, 19 Jan 2017 23:10:31 +0000
Subject: [issue29057] Compiler failure on Mac OS X - sys/random.h
In-Reply-To: <1482514649.72.0.460227366509.issue29057@psf.upfronthosting.co.za>
Message-ID: <1484867431.49.0.266992342531.issue29057@psf.upfronthosting.co.za>


Changes by Ned Deily <nad at python.org>:


----------
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29057>
_______________________________________

From report at bugs.python.org  Thu Jan 19 18:12:35 2017
From: report at bugs.python.org (Ned Deily)
Date: Thu, 19 Jan 2017 23:12:35 +0000
Subject: [issue27596] Build failure with Xcode 8 beta on OSX 10.11
In-Reply-To: <1469270983.6.0.702583698233.issue27596@psf.upfronthosting.co.za>
Message-ID: <1484867555.03.0.00929828653928.issue27596@psf.upfronthosting.co.za>


Ned Deily added the comment:

Maxime, there have been a couple of attempts to address this problem.  We believe it now finally fixed for all versions of macOS from 10.6 through 10.12 by the changes in Issue29057.  Unfortunately, for 2.7,x the last set of fixes were after the release of 2.7.13 in December so, until 2.7.14 is released, you may need to manually apply changeset 74eb71b91112.  (Closing this issue as a duplicate of Issue29057)

----------
resolution:  -> duplicate
stage: needs patch -> resolved
status: open -> closed
superseder:  -> Compiler failure on Mac OS X - sys/random.h

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue27596>
_______________________________________

From report at bugs.python.org  Thu Jan 19 18:18:06 2017
From: report at bugs.python.org (Yury Selivanov)
Date: Thu, 19 Jan 2017 23:18:06 +0000
Subject: [issue29328] struct module should support variable-length strings
In-Reply-To: <1484850272.25.0.472309940141.issue29328@psf.upfronthosting.co.za>
Message-ID: <1484867886.27.0.677367281403.issue29328@psf.upfronthosting.co.za>


Yury Selivanov added the comment:

Ethan, thanks for moving my reply on the list to here.  Also +1 to what Mark said.

----------
nosy: +yselivanov

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29328>
_______________________________________

From report at bugs.python.org  Thu Jan 19 18:20:37 2017
From: report at bugs.python.org (Maxime Belanger)
Date: Thu, 19 Jan 2017 23:20:37 +0000
Subject: [issue27596] Build failure with Xcode 8 beta on OSX 10.11
In-Reply-To: <1469270983.6.0.702583698233.issue27596@psf.upfronthosting.co.za>
Message-ID: <1484868037.67.0.71864708641.issue27596@psf.upfronthosting.co.za>


Maxime Belanger added the comment:

Glad to hear it! Thanks for the quick followup :).

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue27596>
_______________________________________

From report at bugs.python.org  Thu Jan 19 18:31:25 2017
From: report at bugs.python.org (Eryk Sun)
Date: Thu, 19 Jan 2017 23:31:25 +0000
Subject: [issue29330] __slots__ needs documentation
In-Reply-To: <1484865040.1.0.923136956691.issue29330@psf.upfronthosting.co.za>
Message-ID: <1484868685.03.0.173168413824.issue29330@psf.upfronthosting.co.za>


Eryk Sun added the comment:

Are you suggesting that the Helper class in Lib/pydoc.py should index [sub-]topics by special names such as "__slots__"? Currently you can see the __slots__ documentation via the topics "ATTRIBUTEMETHODS" and "SPECIALMETHODS". The list of topics can be printed via help('topics'). Also when you exit out of help('__') it refers you to the SPECIALMETHODS topic.

----------
nosy: +eryksun

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29330>
_______________________________________

From report at bugs.python.org  Thu Jan 19 18:45:11 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Thu, 19 Jan 2017 23:45:11 +0000
Subject: [issue29330] __slots__ needs documentation
In-Reply-To: <1484865040.1.0.923136956691.issue29330@psf.upfronthosting.co.za>
Message-ID: <1484869511.36.0.257492555205.issue29330@psf.upfronthosting.co.za>


Changes by Raymond Hettinger <raymond.hettinger at gmail.com>:


----------
assignee: docs at python -> rhettinger

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29330>
_______________________________________

From report at bugs.python.org  Thu Jan 19 18:51:30 2017
From: report at bugs.python.org (Ned Deily)
Date: Thu, 19 Jan 2017 23:51:30 +0000
Subject: [issue29316] Can we keep typing.py provisional for the duration of
 the Python 3.6 release cycle?
In-Reply-To: <1484774407.47.0.689245055951.issue29316@psf.upfronthosting.co.za>
Message-ID: <1484869890.69.0.394705877071.issue29316@psf.upfronthosting.co.za>


Ned Deily added the comment:

As far as I can tell, the only places that typing was declared as provisional were in PEP 484 and twice in the 3.5 What's New document (https://docs.python.org/3.5/whatsnew/3.5.html); it doesn't appear to have been mentioned in the library reference (https://docs.python.org/3.5/library/typing.html) and probably should have been.  FTR, it looks like there was some discussion of removing typing's provisional status in https://github.com/python/typing/issues/278 and https://github.com/python/typing/issues/247.  While it seems somewhat unusual to reinstate a feature's provisional status, I don't have an objection as 3.6 release manager since it is clear that typing is still a new and evolving module.  To make the change in status, the 3.6 What's New document should be updated, a Misc/NEWS entry added for 3.6.1, and the provisional status added to Docs/library/typing.rst.

----------
stage:  -> needs patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29316>
_______________________________________

From report at bugs.python.org  Thu Jan 19 18:53:21 2017
From: report at bugs.python.org (R. David Murray)
Date: Thu, 19 Jan 2017 23:53:21 +0000
Subject: [issue29323] Wrong documentation (Library) for unicode and str
 comparison
In-Reply-To: <1484836831.64.0.870722472975.issue29323@psf.upfronthosting.co.za>
Message-ID: <1484870001.25.0.778399962666.issue29323@psf.upfronthosting.co.za>


R. David Murray added the comment:

That's a good point, I think that is exactly the issue with that paragraph.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29323>
_______________________________________

From report at bugs.python.org  Thu Jan 19 19:08:21 2017
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 20 Jan 2017 00:08:21 +0000
Subject: [issue29259] Add tp_fastcall to PyTypeObject: support FASTCALL
 calling convention for all callable objects
In-Reply-To: <1484310274.99.0.832491900546.issue29259@psf.upfronthosting.co.za>
Message-ID: <1484870901.9.0.511562547031.issue29259@psf.upfronthosting.co.za>


STINNER Victor added the comment:

tp_fastcall-5.patch: Rebased patch. I already tried to push changes not directly related to tp_fastcall, so the new patch should be a *little bit* shorter.

----------
Added file: http://bugs.python.org/file46347/tp_fastcall-5.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29259>
_______________________________________

From report at bugs.python.org  Thu Jan 19 19:14:48 2017
From: report at bugs.python.org (Guido van Rossum)
Date: Fri, 20 Jan 2017 00:14:48 +0000
Subject: [issue29316] Keep typing.py provisional for the duration of the
 Python 3.6 release cycle
In-Reply-To: <1484774407.47.0.689245055951.issue29316@psf.upfronthosting.co.za>
Message-ID: <1484871288.68.0.865319121835.issue29316@psf.upfronthosting.co.za>


Guido van Rossum added the comment:

Thanks everybody! We'll make typing.py provisional again for 3.6. @levkivskyi do you have the stomach to upload a patch here?

----------
title: Can we keep typing.py provisional for the duration of the Python 3.6 release cycle? -> Keep typing.py provisional for the duration of the Python 3.6 release cycle

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29316>
_______________________________________

From report at bugs.python.org  Thu Jan 19 20:28:04 2017
From: report at bugs.python.org (paul j3)
Date: Fri, 20 Jan 2017 01:28:04 +0000
Subject: [issue9625] argparse: Problem with defaults for variable nargs when
 using choices
In-Reply-To: <1282036744.37.0.964764969496.issue9625@psf.upfronthosting.co.za>
Message-ID: <1484875684.09.0.473436556538.issue9625@psf.upfronthosting.co.za>


paul j3 added the comment:

Recent StackOverFlow question related to this issue

http://stackoverflow.com/questions/41750896/python-argparse-type-inconsistencies-when-combining-choices-nargs-and-def/41751730#41751730

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue9625>
_______________________________________

From report at bugs.python.org  Thu Jan 19 20:30:01 2017
From: report at bugs.python.org (paul j3)
Date: Fri, 20 Jan 2017 01:30:01 +0000
Subject: [issue9338] argparse optionals with nargs='?',
 '*' or '+' can't be followed by positionals
In-Reply-To: <1279881989.28.0.725806934086.issue9338@psf.upfronthosting.co.za>
Message-ID: <1484875801.01.0.708281840563.issue9338@psf.upfronthosting.co.za>


paul j3 added the comment:

Recent StackOverFlow question related to this issue - where the following positional is a subparsers.


http://stackoverflow.com/questions/41742205/how-to-argparse-with-nargs-and-subcommands

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue9338>
_______________________________________

From report at bugs.python.org  Thu Jan 19 21:43:08 2017
From: report at bugs.python.org (Ammar Askar)
Date: Fri, 20 Jan 2017 02:43:08 +0000
Subject: [issue29292] Missing a parameter in PyEval_EvalCodeEx doc
In-Reply-To: <1484640380.78.0.065326592654.issue29292@psf.upfronthosting.co.za>
Message-ID: <1484880188.16.0.29153849381.issue29292@psf.upfronthosting.co.za>


Ammar Askar added the comment:

Updated patch for review comments

----------
Added file: http://bugs.python.org/file46348/kwdefs_docs.diff3

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29292>
_______________________________________

From report at bugs.python.org  Thu Jan 19 21:54:54 2017
From: report at bugs.python.org (Ammar Askar)
Date: Fri, 20 Jan 2017 02:54:54 +0000
Subject: [issue29292] Missing a parameter in PyEval_EvalCodeEx doc
In-Reply-To: <1484640380.78.0.065326592654.issue29292@psf.upfronthosting.co.za>
Message-ID: <1484880894.12.0.222744598216.issue29292@psf.upfronthosting.co.za>


Ammar Askar added the comment:

Updated patch to include a cross reference to keyword-only arguments since I think not everyone will know about this rather new feature.

----------
Added file: http://bugs.python.org/file46349/kwdefs_docs.diff4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29292>
_______________________________________

From report at bugs.python.org  Thu Jan 19 22:06:41 2017
From: report at bugs.python.org (Ammar Askar)
Date: Fri, 20 Jan 2017 03:06:41 +0000
Subject: [issue29281] json.loads documentation missing "versionchanged"
 statement
In-Reply-To: <1484558399.93.0.792457597374.issue29281@psf.upfronthosting.co.za>
Message-ID: <1484881601.16.0.770985930771.issue29281@psf.upfronthosting.co.za>


Ammar Askar added the comment:

Attached patch adds a versionchanged block to specify that bytes and bytesarray can now be used and the types of encodings it supports (as taken from the whatsnew changes here https://hg.python.org/cpython/rev/e9e1bf9ec2ac#l2.7)

----------
keywords: +patch
Added file: http://bugs.python.org/file46350/json_loads_bytes.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29281>
_______________________________________

From report at bugs.python.org  Thu Jan 19 22:34:46 2017
From: report at bugs.python.org (Roundup Robot)
Date: Fri, 20 Jan 2017 03:34:46 +0000
Subject: [issue29292] Missing a parameter in PyEval_EvalCodeEx doc
In-Reply-To: <1484640380.78.0.065326592654.issue29292@psf.upfronthosting.co.za>
Message-ID: <20170120033443.62768.65916.07AD64DB@psf.io>


Roundup Robot added the comment:

New changeset bb76ea32a32f by Xiang Zhang in branch '3.5':
Issue #29292: Update outdated doc of PyEval_EvalCodeEx.
https://hg.python.org/cpython/rev/bb76ea32a32f

New changeset bd121b7517ee by Xiang Zhang in branch '3.6':
Issue #29292: Merge 3.5.
https://hg.python.org/cpython/rev/bd121b7517ee

New changeset ef1146c95860 by Xiang Zhang in branch 'default':
Issue #29292: Merge 3.6.
https://hg.python.org/cpython/rev/ef1146c95860

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29292>
_______________________________________

From report at bugs.python.org  Thu Jan 19 22:35:38 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Fri, 20 Jan 2017 03:35:38 +0000
Subject: [issue29292] Missing a parameter in PyEval_EvalCodeEx doc
In-Reply-To: <1484640380.78.0.065326592654.issue29292@psf.upfronthosting.co.za>
Message-ID: <1484883338.0.0.608659238103.issue29292@psf.upfronthosting.co.za>


Xiang Zhang added the comment:

Thanks Ammar. :-)

----------
resolution:  -> fixed
stage: needs patch -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29292>
_______________________________________

From report at bugs.python.org  Thu Jan 19 22:42:24 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Fri, 20 Jan 2017 03:42:24 +0000
Subject: [issue29325] pysqlite: Evaluate removal of sqlite3_stmt_readonly
In-Reply-To: <1484843450.51.0.0079411502064.issue29325@psf.upfronthosting.co.za>
Message-ID: <1484883744.66.0.507524728064.issue29325@psf.upfronthosting.co.za>


Xiang Zhang added the comment:

pysqlite is not part of Python stdlib so here is not the right place. :-(
Maybe you could go to https://github.com/ghaering/pysqlite for help.

----------
nosy: +xiang.zhang
resolution:  -> third party
stage:  -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29325>
_______________________________________

From report at bugs.python.org  Thu Jan 19 22:50:50 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Fri, 20 Jan 2017 03:50:50 +0000
Subject: [issue29327] SystemError or crash in sorted(iterable=[])
In-Reply-To: <1484847068.03.0.460461228187.issue29327@psf.upfronthosting.co.za>
Message-ID: <1484884250.46.0.916723772266.issue29327@psf.upfronthosting.co.za>


Raymond Hettinger added the comment:

A few random thoughts that may or may not be helpful:

* We now have two seasoned developers and one new core developer that collectively are creating many non-trivial patches to core parts of Python at an unprecedented rate of change.  The patches are coming in much faster than they can reasonably be reviewed and carefully considered, especially by devs such as myself who have very limited time available.  IMO, taken as whole, these changes are destabilizing the language.  Python is so successful and widely adopted that we can't afford a "shit happens" attitude.  Perhaps that works in corners of the language, infrequently used modules, but it makes less sense when touching the critical paths that have had slow and careful evolution over 26 years.

* Besides the volume of patches, one other reason that reviews are hard to come by is that they apply new APIs that I don't fully understand yet.  There are perhaps two people on the planet who could currently give thoughtful, correct, and critical evaluation of all those patches.  Everyone else is just watching them all fly by and hoping that something good is happening.

* One other reason for the lack of review comments in the enthusiasm and fervor surrounding the patches.  I feel like there is a cost of questioning whether the patches should be done or how they are done, like I am burning little karma every time.  Sometimes it feels safest and most cordial to just say nothing and let you make hundreds of semi-reviewed changes to just about every critical part of the language.

* Historically, if there was creator or maintainer of the code who was still active, that person would always be consulted and have a final say on whether a change should be applied.  Now, we have code constantly being changed without consulting the original author (for example, the recent and catastrophic random initialization bug was due to application of a patch without consulting the author of _randommodule.c and the maintainer of random.py, or this change to sorted(), or the changes to decimal, etc).

* In general, Guido has been opposed to sweeping changes across the code base for only tiny benefits.  Of late, that rule seems to have been lost.

* The benefits of FASTCALL mainly apply to fine grained functions which only do a little work and tend to be called frequently in loops.  For functions such as sorted(), the calling overhead is dominated by the cost of actually doing the sort.  For sorted(), FASTCALL is truly irrelevant and likely wasn't worth the complexity, or the actual bug, or any of the time we've now put in it.  There was no actual problem being solved, just a desire to broadly apply new optimizations.

* Historically, we've relied on core developers showing restraint.  Not every idea that pops into their head is immediately turned into a patch accompanied by pressure to apply it.  Devs tended to restrict themselves to parts of the code they knew best through long and careful study rather sweeping through modules and altering other people's carefully crafted code.

* FWIW, I applaud your efforts to reduce call overhead -- that has long been a sore spot for the language.

* Guido has long opposed optimizations that increase risk of bugs, introduce complexity, or that affect long-term maintainability.   In some places, it looks like FASTCALL is increasing the complexity (replacing something simple and well-understood with a wordier, more intricate API that I don't yet fully understand and will affect my ability to maintain the surrounding code).

* It was no long ago that you fought tooth-and-nail against a single line patch optimization I submitted.  The code was clearly correct and had a simple disassembly to prove its benefit.  Your opposition was based on "it increases the complexity of the code, introduces a maintenance cost, and increases the risk of bugs".  In the end, your opposition killed the patch.  But now, the AC and FASTCALL patches don't seem to mind any of these considerations.

* AC is supposed to be a CPython-only concept.  But along the way APIs are being changed without discussion.  I don't mind that sorted() now exposes *iterable* as a keyword argument, but it was originally left out on purpose (Tim opined that code would look worse with iterable as a keyword argument).  That decision was reversed unilaterally without consulting the author and without a test.  Also as AC is being applied, the variable names are being changed.  I never really liked the "mp" that used in dicts and prefer the use of "self" better, but it is a gratuitous change that unilaterally reverses the decisions of the authors and makes the code not match any of the surrounding code that uses the prior conventions.

* FWIW, the claim that the help is much better is specious.  AFAICT, there has never been the slightest problem with "sorted(iterable, key=None, reverse=False) --> new sorted list" which has been clear since the day it was released.   It is some of the new strings the are causing problems with users (my students frequently are tripped-up by the / notation for example; no one seems to be able to intuit what it means without it being explained first).

* FWIW, I'm trying to be constructive and contribute where I can, but frankly I can't keep up with the volume of churn.   Having seen bugs being introduced, it is not inappropriate to ask another dev to please be careful, especially when that dev has been prolific to an unprecedented degree and altering core parts of the language for function calls, to new opcodes, the memory allocators, etc.  Very few people on the planet are competent to review these changes, make reasonable assessments about whether the complexity and churn are worth it.  An fewer still have the time to keep up with the volume of changes.

* Please do continue your efforts to improve the language, but also please moderate the rate of change, mitigate the addition complexity, value stability over micro-optimizations, consult the authors and maintainers of code, take special care without code that hasn't been reviewed because that lacks a safety net, and remember that newer devs may be taking cues from you (do you want them making extensive changes to long existing stable code without consulting the authors and with weak LGTM reviews?)

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29327>
_______________________________________

From report at bugs.python.org  Thu Jan 19 23:35:55 2017
From: report at bugs.python.org (Ma Lin)
Date: Fri, 20 Jan 2017 04:35:55 +0000
Subject: [issue29325] pysqlite: Evaluate removal of sqlite3_stmt_readonly
In-Reply-To: <1484843450.51.0.0079411502064.issue29325@psf.upfronthosting.co.za>
Message-ID: <1484886955.83.0.853848539322.issue29325@psf.upfronthosting.co.za>


Ma Lin added the comment:

sqlite3 module in CPython 3.6.0 also uses sqlite3_stmt_readonly(). see 284676cf2ac8.
sqlite3_stmt_readonly() was used twice [1][2], but it seems that we can avoid using this function in both of them.

[1] https://hg.python.org/cpython/file/3.6/Modules/_sqlite/cursor.c#l517
In palaviv's patch of issue28518, we can eliminate sqlite3_stmt_readonly() in here.

[2] https://hg.python.org/cpython/file/3.6/Modules/_sqlite/cursor.c#l612
We can using the old way, so that we can get around it in here.

----------
nosy: +Ma Lin

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29325>
_______________________________________

From report at bugs.python.org  Thu Jan 19 23:45:42 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Fri, 20 Jan 2017 04:45:42 +0000
Subject: [issue29325] pysqlite: Evaluate removal of sqlite3_stmt_readonly
In-Reply-To: <1484843450.51.0.0079411502064.issue29325@psf.upfronthosting.co.za>
Message-ID: <1484887542.19.0.351986177259.issue29325@psf.upfronthosting.co.za>


Xiang Zhang added the comment:

Hi Ma Lin. If you find this is also a problem in sqlite3 module, feel free to open another issue against sqlite3. :-)

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29325>
_______________________________________

From report at bugs.python.org  Fri Jan 20 00:13:38 2017
From: report at bugs.python.org (Ma Lin)
Date: Fri, 20 Jan 2017 05:13:38 +0000
Subject: [issue29325] pysqlite: Evaluate removal of sqlite3_stmt_readonly
In-Reply-To: <1484843450.51.0.0079411502064.issue29325@psf.upfronthosting.co.za>
Message-ID: <1484889218.87.0.891552961026.issue29325@psf.upfronthosting.co.za>


Ma Lin added the comment:

RHEL6's life-cycle is around 10 years [1], it will be alive until 2020.
https://access.redhat.com/support/policy/updates/errata/

Let's wait issue28518, IMO, that issue will decide whether we can get around sqlite3_stmt_readonly().

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29325>
_______________________________________

From report at bugs.python.org  Fri Jan 20 00:27:13 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Fri, 20 Jan 2017 05:27:13 +0000
Subject: [issue29281] json.loads documentation missing "versionchanged"
 statement
In-Reply-To: <1484558399.93.0.792457597374.issue29281@psf.upfronthosting.co.za>
Message-ID: <1484890033.39.0.545310198734.issue29281@psf.upfronthosting.co.za>


Changes by Raymond Hettinger <raymond.hettinger at gmail.com>:


----------
assignee: docs at python -> rhettinger
nosy: +rhettinger

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29281>
_______________________________________

From report at bugs.python.org  Fri Jan 20 00:38:54 2017
From: report at bugs.python.org (paul j3)
Date: Fri, 20 Jan 2017 05:38:54 +0000
Subject: [issue29298] argparse fails with required subparsers, un-named dest, 
 and empty argv
In-Reply-To: <1484664769.91.0.625725054847.issue29298@psf.upfronthosting.co.za>
Message-ID: <1484890734.49.0.690486182916.issue29298@psf.upfronthosting.co.za>


paul j3 added the comment:

http://bugs.python.org/issue9253  argparse: optional subparsers

Initially this bug/issue was a request to allow subparsers to be optional.  But with the change in how required actions are handled, subparsers are now optional by default.

As you learned from the SO question you now have to specify

subparsers.required = True

This is also discussed in my post (and following ones)

http://bugs.python.org/issue9253#msg186387

The default 'dest' is SUPPRESS.  The error you report occurs because the 'required' error mechanism cannot handle that value. The suggest fix is to assign `dest`, even if it is not needed in the Namespace.  For now it is needed for error reporting.

Reviewing my suggested patches, it looks like I generate a 'dest' substitute from the subparser names.  So the 'required' error would look like

python: error: the following arguments are required: {cmd1, cmd2}

I think this issue can be closed with a reference to 9253. Or maybe that issue is too old, long and confusing, and we need a new bug/issue.

----------
nosy: +paul.j3

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29298>
_______________________________________

From report at bugs.python.org  Fri Jan 20 00:40:52 2017
From: report at bugs.python.org (Roundup Robot)
Date: Fri, 20 Jan 2017 05:40:52 +0000
Subject: [issue29281] json.loads documentation missing "versionchanged"
 statement
In-Reply-To: <1484558399.93.0.792457597374.issue29281@psf.upfronthosting.co.za>
Message-ID: <20170120054008.62405.61681.6B846C05@psf.io>


Roundup Robot added the comment:

New changeset 3452ff9e8f0a by Raymond Hettinger in branch '3.6':
Issue #29281:  Fill-in a missing versionchanged entry
https://hg.python.org/cpython/rev/3452ff9e8f0a

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29281>
_______________________________________

From report at bugs.python.org  Fri Jan 20 00:41:14 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Fri, 20 Jan 2017 05:41:14 +0000
Subject: [issue29281] json.loads documentation missing "versionchanged"
 statement
In-Reply-To: <1484558399.93.0.792457597374.issue29281@psf.upfronthosting.co.za>
Message-ID: <1484890874.43.0.991524987995.issue29281@psf.upfronthosting.co.za>


Raymond Hettinger added the comment:

Thanks for the patch.

----------
resolution:  -> fixed
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29281>
_______________________________________

From report at bugs.python.org  Fri Jan 20 01:04:20 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Fri, 20 Jan 2017 06:04:20 +0000
Subject: [issue29327] SystemError or crash in sorted(iterable=
In-Reply-To: <1484865298.19.0.384278619782.issue29327@psf.upfronthosting.co.za>
Message-ID: <10609079.RnFeg9HAVi@raxxla>


Serhiy Storchaka added the comment:

> While Python 3.5 doesn't crash, I consider that it has also the bug. So I
> added Python 3.5 in Versions.

The patch uses new feature of 3.6 -- supporting positional-only parameters in 
PyArg_ParseTupleAndKeywords()  (see issue26282). Since passing an iterable as 
a keyword argument never worked, it is safe to make the first parameter 
positional-only.

Actually I think that argument parsing code of sorted() and list.sort() can be 
simplified, but this is separate issue. I tried to make the bugfix patch simple.

----------
title: SystemError or crash in sorted(iterable=[]) -> SystemError or crash in sorted(iterable=

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29327>
_______________________________________

From report at bugs.python.org  Fri Jan 20 01:09:34 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Fri, 20 Jan 2017 06:09:34 +0000
Subject: [issue29327] SystemError or crash in sorted(iterable=[])
In-Reply-To: <1484847068.03.0.460461228187.issue29327@psf.upfronthosting.co.za>
Message-ID: <1484892574.94.0.527955195269.issue29327@psf.upfronthosting.co.za>


Changes by Serhiy Storchaka <storchaka+cpython at gmail.com>:


----------
title: SystemError or crash in sorted(iterable= -> SystemError or crash in sorted(iterable=[])

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29327>
_______________________________________

From report at bugs.python.org  Fri Jan 20 01:16:22 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Fri, 20 Jan 2017 06:16:22 +0000
Subject: [issue29327] SystemError or crash in sorted(iterable=[])
In-Reply-To: <1484847068.03.0.460461228187.issue29327@psf.upfronthosting.co.za>
Message-ID: <1484892982.37.0.852275334426.issue29327@psf.upfronthosting.co.za>


Raymond Hettinger added the comment:

Serhiy, this patch passes tests and looks fine.  I think you can go ahead with this patch.

----------
assignee:  -> serhiy.storchaka

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29327>
_______________________________________

From report at bugs.python.org  Fri Jan 20 01:19:21 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Fri, 20 Jan 2017 06:19:21 +0000
Subject: [issue29328] struct module should support variable-length strings
In-Reply-To: <1484850272.25.0.472309940141.issue29328@psf.upfronthosting.co.za>
Message-ID: <1484893161.71.0.264333074116.issue29328@psf.upfronthosting.co.za>


Raymond Hettinger added the comment:

FWIW, I concur with Mark and Yuri that the feature request isn't compatible with the design and purpose of the struct module

----------
nosy: +rhettinger

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29328>
_______________________________________

From report at bugs.python.org  Fri Jan 20 01:24:59 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Fri, 20 Jan 2017 06:24:59 +0000
Subject: [issue29327] SystemError or crash in sorted(iterable=
In-Reply-To: <1484892982.37.0.852275334426.issue29327@psf.upfronthosting.co.za>
Message-ID: <8029157.harqS1mbvJ@raxxla>


Serhiy Storchaka added the comment:

> Serhiy, this patch passes tests and looks fine.  I think you can go ahead
> with this patch.

Right now I'm building 3.6 with applied patch before final testing and 
committing. My netbook is very slow. :(

----------
title: SystemError or crash in sorted(iterable=[]) -> SystemError or crash in sorted(iterable=

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29327>
_______________________________________

From report at bugs.python.org  Fri Jan 20 01:29:27 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Fri, 20 Jan 2017 06:29:27 +0000
Subject: [issue29327] SystemError or crash in sorted(iterable=[])
In-Reply-To: <1484847068.03.0.460461228187.issue29327@psf.upfronthosting.co.za>
Message-ID: <1484893767.26.0.93087422747.issue29327@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

The title is spoiled when reply by email. I suppose [] has special meaning in email subject.

----------
title: SystemError or crash in sorted(iterable= -> SystemError or crash in sorted(iterable=[])

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29327>
_______________________________________

From report at bugs.python.org  Fri Jan 20 01:35:37 2017
From: report at bugs.python.org (Roundup Robot)
Date: Fri, 20 Jan 2017 06:35:37 +0000
Subject: [issue29327] SystemError or crash in sorted(iterable=[])
In-Reply-To: <1484847068.03.0.460461228187.issue29327@psf.upfronthosting.co.za>
Message-ID: <20170120063534.111547.61042.A9FBDF81@psf.io>


Roundup Robot added the comment:

New changeset 1827b64cfce8 by Serhiy Storchaka in branch '3.6':
Issue #29327: Fixed a crash when pass the iterable keyword argument to sorted().
https://hg.python.org/cpython/rev/1827b64cfce8

New changeset f44f44b14dfc by Serhiy Storchaka in branch 'default':
Issue #29327: Fixed a crash when pass the iterable keyword argument to sorted().
https://hg.python.org/cpython/rev/f44f44b14dfc

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29327>
_______________________________________

From report at bugs.python.org  Fri Jan 20 01:36:39 2017
From: report at bugs.python.org (INADA Naoki)
Date: Fri, 20 Jan 2017 06:36:39 +0000
Subject: [issue29296] convert print() to METH_FASTCALL
In-Reply-To: <1484652871.69.0.785445814249.issue29296@psf.upfronthosting.co.za>
Message-ID: <1484894199.66.0.737551928179.issue29296@psf.upfronthosting.co.za>


INADA Naoki added the comment:

> Can we close the issue, or do you prefer to keep it open?

I don't care.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29296>
_______________________________________

From report at bugs.python.org  Fri Jan 20 01:59:39 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Fri, 20 Jan 2017 06:59:39 +0000
Subject: [issue29331] Simplify argument parsing in sorted() and list.sort()
Message-ID: <1484895579.74.0.0168401872287.issue29331@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka:

Proposed patch simplifies argument parsing in sorted() and list.sort().

list.sort() now uses the support of keyword-only parameters in PyArg_ParseTupleAndKeywords().

Keyword arguments now are passed as is to list.sorted() in sorted().

----------
components: Interpreter Core
files: sort-arg-parse.patch
keywords: patch
messages: 285887
nosy: haypo, rhettinger, serhiy.storchaka
priority: normal
severity: normal
stage: patch review
status: open
title: Simplify argument parsing in sorted() and list.sort()
type: enhancement
versions: Python 3.7
Added file: http://bugs.python.org/file46351/sort-arg-parse.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29331>
_______________________________________

From report at bugs.python.org  Fri Jan 20 02:14:19 2017
From: report at bugs.python.org (JGoutin)
Date: Fri, 20 Jan 2017 07:14:19 +0000
Subject: [issue29241] sys._enablelegacywindowsfsencoding() don't apply to
 os.fsencode and os.fsdecode
In-Reply-To: <1484137681.59.0.0301192140292.issue29241@psf.upfronthosting.co.za>
Message-ID: <1484896459.48.0.320114938992.issue29241@psf.upfronthosting.co.za>


JGoutin added the comment:

A little encoding cache benchmark.


Current Code:
=============

import sys

def _fscodec():
    encoding = sys.getfilesystemencoding()
    errors = sys.getfilesystemencodeerrors()

    def fsencode(filename):
        filename = fspath(filename)  # Does type-checking of `filename`.
        if isinstance(filename, str):
            return filename.encode(encoding, errors)
        else:
            return filename

    def fsdecode(filename):
        filename = fspath(filename)  # Does type-checking of `filename`.
        if isinstance(filename, bytes):
            return filename.decode(encoding, errors)
        else:
            return filename

    return fsencode, fsdecode

fsencode, fsdecode = _fscodec()
del _fscodec

---------

import os

%timeit os.fsdecode(b'\xc3\xa9')
The slowest run took 21.59 times longer than the fastest. This could mean that an intermediate result is being cached.
1000000 loops, best of 3: 449 ns per loop

%timeit os.fsencode('?')
The slowest run took 24.20 times longer than the fastest. This could mean that an intermediate result is being cached.
1000000 loops, best of 3: 412 ns per loop


Modified Code:
==============

from sys import getfilesystemencoding, getfilesystemencodeerrors

def fsencode(filename):
    filename = fspath(filename)  # Does type-checking of `filename`.
    if isinstance(filename, str):
        return filename.encode(getfilesystemencoding(),
                               getfilesystemencodeerrors())
    else:
        return filename

def fsdecode(filename):
    filename = fspath(filename)  # Does type-checking of `filename`.
    if isinstance(filename, bytes):
        return filename.decode(getfilesystemencoding(),
                               getfilesystemencodeerrors())
    else:
        return filename

---------

import os

%timeit os.fsdecode(b'\xc3\xa9')
The slowest run took 15.88 times longer than the fastest. This could mean that an intermediate result is being cached.
1000000 loops, best of 3: 541 ns per loop

%timeit os.fsencode('?')
The slowest run took 19.32 times longer than the fastest. This could mean that an intermediate result is being cached.
1000000 loops, best of 3: 502 ns per loop


Result:
=======

Cache is a 17% speed up optimization.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29241>
_______________________________________

From report at bugs.python.org  Fri Jan 20 02:49:46 2017
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 20 Jan 2017 07:49:46 +0000
Subject: [issue29327] SystemError or crash in sorted(iterable=
In-Reply-To: <8029157.harqS1mbvJ@raxxla>
Message-ID: <CAMpsgwaj_7BHzXytbV4_=eXMTz3HsKsQR1=+g1PExhuVjtPB=A@mail.gmail.com>


STINNER Victor added the comment:

"The patch uses new feature of 3.6 -- supporting positional-only
parameters in PyArg_ParseTupleAndKeywords()  (see issue26282)."

Oh, I didn't recall that it's a new feature. It's a nice feature by
the way, thanks for implementing it :-)

In that case, yeah it's ok to leave Python 3.5 unchanged. I proposed
to fix 3.5 because I expected that you could use exactly the same
patch.

----------
title: SystemError or crash in sorted(iterable=[]) -> SystemError or crash in sorted(iterable=

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29327>
_______________________________________

From report at bugs.python.org  Fri Jan 20 02:51:29 2017
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 20 Jan 2017 07:51:29 +0000
Subject: [issue29296] convert print() to METH_FASTCALL
In-Reply-To: <1484652871.69.0.785445814249.issue29296@psf.upfronthosting.co.za>
Message-ID: <1484898689.3.0.72275093807.issue29296@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Since the title of the issue is "convert print() to METH_FASTCALL", I consider that the issue is now done. As I wrote, as soon as AC will support **kwargs, we will be able to print() to use AC.

Thanks Naoki for the change.

----------
dependencies:  -Argument Clinic should understand *args and **kwargs parameters
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29296>
_______________________________________

From report at bugs.python.org  Fri Jan 20 03:29:05 2017
From: report at bugs.python.org (Wen Adam)
Date: Fri, 20 Jan 2017 08:29:05 +0000
Subject: [issue29332] Uniform SelectSelector._select behavior
Message-ID: <1484900945.96.0.01738502245.issue29332@psf.upfronthosting.co.za>


New submission from Wen Adam:

SelectSelector._select is differently on different platforms.

On win32, SelectSelector._select is a unbound/bound method and pass instance as the first argument, but on *nix, SelectSelector._select is a builtin_function_or_method(Although self.select still works)

If someone want to replace(or patch) select, he must implement two select function(one is a function, the other is a instance method), 
Otherwise you will get?TypeError: select() takes at most 4 arguments (5 given)?.

And I think make built-in funtion(like len, select.select) as a class attr/method is a black magic(it's still a function) and hard to maintain

----------
components: Library (Lib)
files: uniform_select_behaviour.patch
keywords: patch
messages: 285891
nosy: Wen Adam, neologix
priority: normal
severity: normal
status: open
title: Uniform SelectSelector._select behavior
type: behavior
versions: Python 3.4, Python 3.5, Python 3.6, Python 3.7
Added file: http://bugs.python.org/file46352/uniform_select_behaviour.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29332>
_______________________________________

From report at bugs.python.org  Fri Jan 20 03:53:35 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Fri, 20 Jan 2017 08:53:35 +0000
Subject: [issue29281] json.loads documentation missing "versionchanged"
 statement
In-Reply-To: <1484558399.93.0.792457597374.issue29281@psf.upfronthosting.co.za>
Message-ID: <1484902415.25.0.284903257798.issue29281@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

The wording looks little misleading. There is the encoding parameter in json.loads(). It is deprecated and ignored.

----------
nosy: +serhiy.storchaka

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29281>
_______________________________________

From report at bugs.python.org  Fri Jan 20 04:03:24 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Fri, 20 Jan 2017 09:03:24 +0000
Subject: [issue29331] Simplify argument parsing in sorted() and list.sort()
In-Reply-To: <1484895579.74.0.0168401872287.issue29331@psf.upfronthosting.co.za>
Message-ID: <1484903004.77.0.466679102297.issue29331@psf.upfronthosting.co.za>


Changes by Serhiy Storchaka <storchaka+cpython at gmail.com>:


Added file: http://bugs.python.org/file46353/sort-arg-parse-2.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29331>
_______________________________________

From report at bugs.python.org  Fri Jan 20 04:10:53 2017
From: report at bugs.python.org (Aviv Palivoda)
Date: Fri, 20 Jan 2017 09:10:53 +0000
Subject: [issue28518] execute("begin immediate") throwing OperationalError
In-Reply-To: <1477307118.31.0.70785963389.issue28518@psf.upfronthosting.co.za>
Message-ID: <1484903453.64.0.012584490699.issue28518@psf.upfronthosting.co.za>


Aviv Palivoda added the comment:

Uploading a new patch with fixes from Ma Lin comments.
Two points:
1. Should we add the VACUUM with a explicit commit? Maybe there should be an implicit commit before VACUUM?
2. Should a SELECT start a transaction? I think it should according to PEP 249. There is a open issue on the case (#9924). Should we just change this on this patch?

----------
Added file: http://bugs.python.org/file46354/sqlite-ddl-dml-2.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28518>
_______________________________________

From report at bugs.python.org  Fri Jan 20 04:11:22 2017
From: report at bugs.python.org (Aviv Palivoda)
Date: Fri, 20 Jan 2017 09:11:22 +0000
Subject: [issue9924] sqlite3 SELECT does not BEGIN a transaction,
 but should according to spec
In-Reply-To: <1285214166.49.0.269440190097.issue9924@psf.upfronthosting.co.za>
Message-ID: <1484903482.59.0.136001266567.issue9924@psf.upfronthosting.co.za>


Changes by Aviv Palivoda <palaviv at gmail.com>:


----------
nosy: +palaviv

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue9924>
_______________________________________

From report at bugs.python.org  Fri Jan 20 04:22:36 2017
From: report at bugs.python.org (Mark Dickinson)
Date: Fri, 20 Jan 2017 09:22:36 +0000
Subject: [issue29160] pow with three int arguments works like it had two
 arguments
In-Reply-To: <1483569297.56.0.246813655497.issue29160@psf.upfronthosting.co.za>
Message-ID: <1484904156.66.0.83737940656.issue29160@psf.upfronthosting.co.za>


Changes by Mark Dickinson <dickinsm at gmail.com>:


----------
resolution: not a bug -> third party

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29160>
_______________________________________

From report at bugs.python.org  Fri Jan 20 04:24:40 2017
From: report at bugs.python.org (Antoine Pitrou)
Date: Fri, 20 Jan 2017 09:24:40 +0000
Subject: [issue29328] struct module should support variable-length strings
In-Reply-To: <1484850272.25.0.472309940141.issue29328@psf.upfronthosting.co.za>
Message-ID: <1484904280.04.0.0382511539738.issue29328@psf.upfronthosting.co.za>


Antoine Pitrou added the comment:

To add a bit to what Yury said, even framing isn't always compatible with this proposal.  For example, in dask/distributed, we first have a word for the number of frames, then one word per frame to indicate each frame's length, then the frame bodies.

----------
nosy: +pitrou

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29328>
_______________________________________

From report at bugs.python.org  Fri Jan 20 05:14:43 2017
From: report at bugs.python.org (Christian Heimes)
Date: Fri, 20 Jan 2017 10:14:43 +0000
Subject: [issue9216] FIPS support for hashlib
In-Reply-To: <1278721335.16.0.522410247151.issue9216@psf.upfronthosting.co.za>
Message-ID: <1484907283.72.0.421884906004.issue9216@psf.upfronthosting.co.za>


Christian Heimes added the comment:

Objection from hashlib maintainer: I will reject a used_for_security flag with default of False. I'm slowly moving Python to a secure-by-default policy. Therefore used_for_security must be an explicit opt-out.

I'm aware that the policy will require modifications to all software that uses MD5. To be honest that's my goal. If you care about FIPS, then any use of MD5 must be a concious and careful decision. I want developers to move away from MD5 and replace it with SipHash24, Blake2 or SHA-2. MD5 should *only* remain when backwards incompatibility prevent migration.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue9216>
_______________________________________

From report at bugs.python.org  Fri Jan 20 05:20:19 2017
From: report at bugs.python.org (Mark Shannon)
Date: Fri, 20 Jan 2017 10:20:19 +0000
Subject: [issue29051] Improve error reporting involving f-strings (PEP 498)
In-Reply-To: <1482494461.91.0.191852219403.issue29051@psf.upfronthosting.co.za>
Message-ID: <1484907619.13.0.277166701964.issue29051@psf.upfronthosting.co.za>


Mark Shannon added the comment:

This problem is the parsing of f-strings. 

The expressions in an f-string are not "eval"ed in the sense of the eval() function. They are evaluated exactly the same as any other Python expression. However the parsing of f-strings does not provide correct line numbers.

This problem also manifests itself in the ast and tokenize modules.

>>> m = ast.parse("""f'''
... {
... FOO
... }
... '''
... """)

>>> m.body[0].value.values[1].value.id
'FOO'
>>> m.body[0].value.values[1].value.lineno
2

That 2 should be a 3, and yet
eval(compile(m, "test2", "exec"))
  File "<stdin>", line 1, in <module>
  File "test2", line 5, in <module>
NameError: name 'FOO' is not defined

gives line 5 for the error, so not only are the line numbers wrong they are inconsistent.

The problem is that the internals of the f-string are not tokenized and parsed using the normal mechanism, but in an ad-hoc fashion in Python-ast.c as demonstrated when tokenizing the source

$ python3.6 -m tokenize test2
0,0-0,0:            ENCODING       'utf-8'        
1,0-5,3:            STRING         "f'''\n{\nFOO\n}\n'''"
5,3-5,4:            NEWLINE        '\n'           
6,0-6,0:            ENDMARKER      ''

The f-string could should be tokenized as something like:
FSTRING_START f'''
STRING_PART \n 
LEFT_BRACE {
NEWLINE
IDENTIFIER FOO
NEWLINE
RIGHT_BRACE }
STRING_PART \n
FSTRING_END '''

Although this would complicate the tokenizer, it would mean that the internals of f-strings could be made explicit in the grammar, and that the compiler could generate correct offsets.

----------
nosy: +Mark.Shannon

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29051>
_______________________________________

From report at bugs.python.org  Fri Jan 20 05:20:58 2017
From: report at bugs.python.org (Christian Heimes)
Date: Fri, 20 Jan 2017 10:20:58 +0000
Subject: [issue9216] FIPS support for hashlib
In-Reply-To: <1278721335.16.0.522410247151.issue9216@psf.upfronthosting.co.za>
Message-ID: <1484907658.46.0.95187087297.issue9216@psf.upfronthosting.co.za>


Christian Heimes added the comment:

PS: I'm also against a hashlib.fips module in stdlib. FIPS mode is irrelevant for majority of users and countries. I neither want to confuse people nor introduce more maintenance and documentation burden than necessary. Antoine gave another good reason against a fips module, too.

I'm fine with a used_for_security flag and functions to get/set FIPS state. Something like hashlib.get_fips_mode() is useful for testing.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue9216>
_______________________________________

From report at bugs.python.org  Fri Jan 20 05:23:25 2017
From: report at bugs.python.org (Mark Shannon)
Date: Fri, 20 Jan 2017 10:23:25 +0000
Subject: [issue29051] Improve error reporting involving f-strings (PEP 498)
In-Reply-To: <1482494461.91.0.191852219403.issue29051@psf.upfronthosting.co.za>
Message-ID: <1484907805.79.0.561847517196.issue29051@psf.upfronthosting.co.za>


Mark Shannon added the comment:

It is also worth mentioning that incorrect line numbers means that tools like pyflakes, pylint, mypy, lgtm, etc, need to reimplement parsing of f-strings.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29051>
_______________________________________

From report at bugs.python.org  Fri Jan 20 05:57:37 2017
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 20 Jan 2017 10:57:37 +0000
Subject: [issue29327] SystemError or crash in sorted(iterable=
In-Reply-To: <1484847068.03.0.460461228187.issue29327@psf.upfronthosting.co.za>
Message-ID: <1484909857.77.0.564872013295.issue29327@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Raymond Hettinger: "A few random thoughts that may or may not be helpful: (...)"

I replied on the python-committers mailing list:
https://mail.python.org/pipermail/python-committers/2017-January/004129.html

I'm not sure that this specific issue is the best place to discuss, I propose to continue the discussion there.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29327>
_______________________________________

From report at bugs.python.org  Fri Jan 20 06:01:32 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Fri, 20 Jan 2017 11:01:32 +0000
Subject: [issue29327] SystemError or crash in sorted(iterable=
In-Reply-To: <1484847068.03.0.460461228187.issue29327@psf.upfronthosting.co.za>
Message-ID: <1484910092.46.0.301316307623.issue29327@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

Thank you for your reviews Victor and Raymond.

As for more general Raymond comments, I agreed with many of them in principle, but this isn't directly related to this issue. Victor opened a topic on the python-committers mailing list:

https://mail.python.org/pipermail/python-committers/2017-January/004129.html

----------
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed
versions:  -Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29327>
_______________________________________

From report at bugs.python.org  Fri Jan 20 06:02:15 2017
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 20 Jan 2017 11:02:15 +0000
Subject: [issue29327] SystemError or crash in sorted(iterable=[])
In-Reply-To: <1484847068.03.0.460461228187.issue29327@psf.upfronthosting.co.za>
Message-ID: <1484910135.51.0.74329453111.issue29327@psf.upfronthosting.co.za>


Changes by STINNER Victor <victor.stinner at gmail.com>:


----------
title: SystemError or crash in sorted(iterable= -> SystemError or crash in sorted(iterable=[])

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29327>
_______________________________________

From report at bugs.python.org  Fri Jan 20 06:19:20 2017
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 20 Jan 2017 11:19:20 +0000
Subject: [issue29331] Simplify argument parsing in sorted() and list.sort()
In-Reply-To: <1484895579.74.0.0168401872287.issue29331@psf.upfronthosting.co.za>
Message-ID: <1484911160.44.0.704840575965.issue29331@psf.upfronthosting.co.za>


STINNER Victor added the comment:

sort-arg-parse-2.patch now LGTM with the comment, thanks :-)

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29331>
_______________________________________

From report at bugs.python.org  Fri Jan 20 06:50:36 2017
From: report at bugs.python.org (Gereon Kaiping)
Date: Fri, 20 Jan 2017 11:50:36 +0000
Subject: [issue29333] ConfigParser calls Interpolation.before_read after
 reading
Message-ID: <1484913036.08.0.781257093151.issue29333@psf.upfronthosting.co.za>


New submission from Gereon Kaiping:

In its current implementation, `ConfigParser` calls its Interpolation's `before_read` method in the very last step of `_read`, when all properties have already been overwritten by their new uninterpolated values.

I am developing a program with modular config files: It is possible to supply several configuration files on the command line, and they are all fed through the `.read` method.

Now it would be amazing to use `read` time interpolation instead of `get` time interpolation to construct things like

outputfilename = %(outputfilename)s_extension_from_this_ini_module

By looking at the `Interpolation` class, it seems that behaviour like this should be supported by supplying a `before_read` as follows.

    def before_read(self, parser, section, option, value):
        L = []
        interpolations = parser[section]
        self._interpolate_some(
            parser, option, L, value, section, interpolations, 1)
        return ''.join(L)

However, this is not possible, because `before_read` is only called *after* all values in the config file have been read and all old values in the ConfigParser object have been overridden.

The attached file contains a subclass of `BasicInterpolation` and a subclass of `ConfigParser` which in concert allow me to write recursive property definitions as given above.

The downside of this change is that (a) interpolation values can't be defined *after* they are used any more, and (b) it is not possible to hack this parser to accept multi-line option names as interpolations. (To me personally, both of these don't sound like useful features to begin with.)

----------
files: earlyinterpolation.py
messages: 285902
nosy: Anaphory
priority: normal
severity: normal
status: open
title: ConfigParser calls Interpolation.before_read after reading
Added file: http://bugs.python.org/file46355/earlyinterpolation.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29333>
_______________________________________

From report at bugs.python.org  Fri Jan 20 07:13:47 2017
From: report at bugs.python.org (Greg Stark)
Date: Fri, 20 Jan 2017 12:13:47 +0000
Subject: [issue29334] ssl.SSLObject method getpeercert() is buggy,
 do_handshake() is strange
Message-ID: <1484914427.68.0.380715465195.issue29334@psf.upfronthosting.co.za>


New submission from Greg Stark:

In my experiments with the relatively new class SSLObject from the ssl module I've noticed the following behavior(s) which I think can be described as bugs.

The getpeercert() method raises a ValueError exception "handshake not done" even after the handshake has successfully completed. If, however, I call the do_handshake() method *after* the handshake completes, then getpeercert() correctly runs and returns the peer's certificate. So now let's focus on do_handshake(). This method is basically undocumented, which I thought was ok because what it does should be obvious. It does seem to initiate a handshake if it's the first method call after the SSLObject is created. If called afterward, it doesn't outwardly appear to do anything, but as mentioned previously it does magically make the getpeercert() method start working.

----------
assignee: christian.heimes
components: SSL
messages: 285903
nosy: Greg Stark, christian.heimes
priority: normal
severity: normal
status: open
title: ssl.SSLObject method getpeercert() is buggy, do_handshake() is strange
type: behavior
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29334>
_______________________________________

From report at bugs.python.org  Fri Jan 20 07:17:06 2017
From: report at bugs.python.org (Nick Coghlan)
Date: Fri, 20 Jan 2017 12:17:06 +0000
Subject: [issue29319] Embedded 3.6.0 distribution cannot run pyz files
In-Reply-To: <1484827891.73.0.559570067048.issue29319@psf.upfronthosting.co.za>
Message-ID: <1484914626.53.0.0154400154744.issue29319@psf.upfronthosting.co.za>


Nick Coghlan added the comment:

I think this is just a straight-up interaction bug where we never accounted for the combination of -I with directory and zipfile execution back when -I was added.

>From a bug fix perspective, I think the right thing to do in all affected versions would be to add a check to RunMainFromImporter such that it does an insert rather than a set when -I is used.

For 3.7+, it may make sense to change the signature of runpy._run_module_as_main to accept the path entry as Steve suggests, as that could also resolve the quirk where we'll run a __main__ module from *anywhere* on sys.path, rather than only from the given path entry.

The core of the current approach was implemented back in the 2.6 time frame, and then tweaked a bit in 2.7, and hence predates all the niceties offered by the importlib API.

However, the latter change would be a separate RFE rather than a bug fix.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29319>
_______________________________________

From report at bugs.python.org  Fri Jan 20 07:29:39 2017
From: report at bugs.python.org (Christian Heimes)
Date: Fri, 20 Jan 2017 12:29:39 +0000
Subject: [issue29334] ssl.SSLObject method getpeercert() is buggy,
 do_handshake() is strange
In-Reply-To: <1484914427.68.0.380715465195.issue29334@psf.upfronthosting.co.za>
Message-ID: <1484915379.81.0.170908238538.issue29334@psf.upfronthosting.co.za>


Christian Heimes added the comment:

Hi Grek,

can you provide a script to reproduce the problem, please?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29334>
_______________________________________

From report at bugs.python.org  Fri Jan 20 07:31:34 2017
From: report at bugs.python.org (Greg Stark)
Date: Fri, 20 Jan 2017 12:31:34 +0000
Subject: [issue29334] ssl.SSLObject method getpeercert() is buggy,
 do_handshake() is strange
In-Reply-To: <1484915379.81.0.170908238538.issue29334@psf.upfronthosting.co.za>
Message-ID: <CAJiBena2jVArQp_hXVvT-wY7bw17RU6u=FFOP90caeE3EAvs5w@mail.gmail.com>


Greg Stark added the comment:

Christian,

I will gladly do so a little later today. Thanks for your quick response.

--greg

On Fri, Jan 20, 2017 at 7:29 AM, Christian Heimes <report at bugs.python.org>
wrote:

>
> Christian Heimes added the comment:
>
> Hi Grek,
>
> can you provide a script to reproduce the problem, please?
>
> ----------
>
> _______________________________________
> Python tracker <report at bugs.python.org>
> <http://bugs.python.org/issue29334>
> _______________________________________
>

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29334>
_______________________________________

From report at bugs.python.org  Fri Jan 20 08:11:29 2017
From: report at bugs.python.org (R. David Murray)
Date: Fri, 20 Jan 2017 13:11:29 +0000
Subject: [issue29325] pysqlite: Evaluate removal of sqlite3_stmt_readonly
In-Reply-To: <1484843450.51.0.0079411502064.issue29325@psf.upfronthosting.co.za>
Message-ID: <1484917889.8.0.961475374223.issue29325@psf.upfronthosting.co.za>


R. David Murray added the comment:

Python ships with RH6, and includes a working sqlite3 module.  So if you are restricting yourself to software directly supported by RH6, you are fine.  If you are installing python3.6 on your rh6 box, then you can also install a more up to date sqlite in order to get it to compile the sqlite3 module.  So I don't see any actionable issue because of RH6 here, either for us or for pysqlite.  The other issue is a different story, of course.

----------
nosy: +r.david.murray

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29325>
_______________________________________

From report at bugs.python.org  Fri Jan 20 09:25:16 2017
From: report at bugs.python.org (Kushal Das)
Date: Fri, 20 Jan 2017 14:25:16 +0000
Subject: [issue28735] Mock is equal to ANY but MagicMock is not
In-Reply-To: <1479485234.58.0.222215885157.issue28735@psf.upfronthosting.co.za>
Message-ID: <1484922316.52.0.544277789515.issue28735@psf.upfronthosting.co.za>


Kushal Das added the comment:

>From a checkout in last week.

Python 3.7.0a0 (default:c163a76163f7, Jan 15 2017, 22:20:24) 
[GCC 6.3.1 20161221 (Red Hat 6.3.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from unittest import mock
>>> mock.ANY == mock.Mock() 
True
>>> mock.ANY == mock.MagicMock() 
True

----------
nosy: +kushal.das

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28735>
_______________________________________

From report at bugs.python.org  Fri Jan 20 09:49:07 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Fri, 20 Jan 2017 14:49:07 +0000
Subject: [issue28735] Mock is equal to ANY but MagicMock is not
In-Reply-To: <1479485234.58.0.222215885157.issue28735@psf.upfronthosting.co.za>
Message-ID: <1484923747.23.0.0374516874333.issue28735@psf.upfronthosting.co.za>


Xiang Zhang added the comment:

Kushal I think the OP is reporting `mock.MagicMock() == mock.ANY` is False while `mock.ANY == mock.MagicMock()` is True which is still the case in 3.7. A related issue is #25195. In that issue Serhiy has mentioned this behaviour(http://bugs.python.org/issue25195#msg259822).

----------
nosy: +berker.peksag, serhiy.storchaka, xiang.zhang

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28735>
_______________________________________

From report at bugs.python.org  Fri Jan 20 09:51:39 2017
From: report at bugs.python.org (Kushal Das)
Date: Fri, 20 Jan 2017 14:51:39 +0000
Subject: [issue28735] Mock is equal to ANY but MagicMock is not
In-Reply-To: <1479485234.58.0.222215885157.issue28735@psf.upfronthosting.co.za>
Message-ID: <1484923899.18.0.357504102452.issue28735@psf.upfronthosting.co.za>


Kushal Das added the comment:

Now I understood, thanks. I think I should have more coffee before commenting on bugs :)

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28735>
_______________________________________

From report at bugs.python.org  Fri Jan 20 09:56:22 2017
From: report at bugs.python.org (Ivan Levkivskyi)
Date: Fri, 20 Jan 2017 14:56:22 +0000
Subject: [issue29316] Keep typing.py provisional for the duration of the
 Python 3.6 release cycle
In-Reply-To: <1484774407.47.0.689245055951.issue29316@psf.upfronthosting.co.za>
Message-ID: <1484924182.29.0.928545626581.issue29316@psf.upfronthosting.co.za>


Ivan Levkivskyi added the comment:

Yes, I attached the patch.

----------
keywords: +patch
stage: needs patch -> patch review
Added file: http://bugs.python.org/file46356/provisional-typing-patch.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29316>
_______________________________________

From report at bugs.python.org  Fri Jan 20 10:17:30 2017
From: report at bugs.python.org (Roundup Robot)
Date: Fri, 20 Jan 2017 15:17:30 +0000
Subject: [issue29316] Keep typing.py provisional for the duration of the
 Python 3.6 release cycle
In-Reply-To: <1484774407.47.0.689245055951.issue29316@psf.upfronthosting.co.za>
Message-ID: <20170120151727.62176.25531.9FB90B18@psf.io>


Roundup Robot added the comment:

New changeset 72c64b6e5486 by Ned Deily in branch '3.6':
Issue #29316: Restore the provisional status of typing module and add
https://hg.python.org/cpython/rev/72c64b6e5486

New changeset ef416878495f by Ned Deily in branch 'default':
Issue #29316: merge 3.6
https://hg.python.org/cpython/rev/ef416878495f

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29316>
_______________________________________

From report at bugs.python.org  Fri Jan 20 10:18:24 2017
From: report at bugs.python.org (Ned Deily)
Date: Fri, 20 Jan 2017 15:18:24 +0000
Subject: [issue29316] Keep typing.py provisional for the duration of the
 Python 3.6 release cycle
In-Reply-To: <1484774407.47.0.689245055951.issue29316@psf.upfronthosting.co.za>
Message-ID: <1484925504.68.0.312991743853.issue29316@psf.upfronthosting.co.za>


Ned Deily added the comment:

Patch LGTM, thanks Ivan!

----------
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29316>
_______________________________________

From report at bugs.python.org  Fri Jan 20 10:59:08 2017
From: report at bugs.python.org (Ammar Askar)
Date: Fri, 20 Jan 2017 15:59:08 +0000
Subject: [issue29281] json.loads documentation missing "versionchanged"
 statement
In-Reply-To: <1484558399.93.0.792457597374.issue29281@psf.upfronthosting.co.za>
Message-ID: <1484927948.66.0.40538899925.issue29281@psf.upfronthosting.co.za>


Ammar Askar added the comment:

Which part is misleading, do you think the use of "encoding" could be confused with the argument encoding? 

There is a note right above the versionchanged block that says:

`The other arguments have the same meaning as in load(), except encoding which is ignored and deprecated.`


If that is the concern then I think changing the wording to be, "the encoding of the input string should be ..."

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29281>
_______________________________________

From report at bugs.python.org  Fri Jan 20 11:38:18 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Fri, 20 Jan 2017 16:38:18 +0000
Subject: [issue20186] Derby #18: Convert 31 sites to Argument Clinic across 23
 files
In-Reply-To: <1484853377.67.0.0336140259835.issue20186@psf.upfronthosting.co.za>
Message-ID: <2161034.UDzx9Zos4j@raxxla>


Serhiy Storchaka added the comment:

There are patches based on modules_issue20186.patch that convert to Argument 
Clinic 4 modules: _csv, _lsprof, _tracemalloc and symtable. They are 
synchronized with current sources and updated to current Argument Clinic. 
Other changes:

* Addressed Larry's comments.

* Used converter names instead of format units (object instead of 'O' etc).

* Removed unneeded self declarations in _lsprof.

* Removed changes for _csv.Dialect.__new__. Generated signature has None as 
default values, but this is not true. Actually parameters don't have default 
values. This function can't be converted.

* Parameters of _lsprof.Profiler.enable should have the int converter, not 
bool. Default values are -1 (meaning "don't change current value"), not True.

* Made the parameter of _tracemalloc._get_object_traceback() positional-only.

* Simplified code for creating a result in _traceback functions.

----------
Added file: http://bugs.python.org/file46357/csv_clinic.patch
Added file: http://bugs.python.org/file46358/lsprof_clinic.patch
Added file: http://bugs.python.org/file46359/tracemalloc_clinic.patch
Added file: http://bugs.python.org/file46360/symtable_clinic.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue20186>
_______________________________________
-------------- next part --------------
diff -r f44f44b14dfc Modules/_csv.c
--- a/Modules/_csv.c	Fri Jan 20 08:35:18 2017 +0200
+++ b/Modules/_csv.c	Fri Jan 20 18:17:14 2017 +0200
@@ -13,6 +13,12 @@ module instead.
 #include "Python.h"
 #include "structmember.h"
 
+#include "clinic/_csv.c.h"
+/*[clinic input]
+module _csv
+class _csv.Dialect "PyObject *" "&Dialect_Type"
+[clinic start generated code]*/
+/*[clinic end generated code: output=da39a3ee5e6b4b0d input=6f3170c88d1c5a7a]*/
 
 typedef struct {
     PyObject *error_obj;   /* CSV exception */
@@ -1421,12 +1427,21 @@ csv_writer(PyObject *module, PyObject *a
 /*
  * DIALECT REGISTRY
  */
+
+/*[clinic input]
+_csv.list_dialects
+
+Return a list of all know dialect names.
+[clinic start generated code]*/
+
 static PyObject *
-csv_list_dialects(PyObject *module, PyObject *args)
+_csv_list_dialects_impl(PyObject *module)
+/*[clinic end generated code: output=a5b92b215b006a6d input=a31ccb1071637d7c]*/
 {
     return PyDict_Keys(_csvstate_global->dialects);
 }
 
+
 static PyObject *
 csv_register_dialect(PyObject *module, PyObject *args, PyObject *kwargs)
 {
@@ -1454,36 +1469,69 @@ csv_register_dialect(PyObject *module, P
     return Py_None;
 }
 
+/*[clinic input]
+_csv.unregister_dialect
+
+    name: object
+    /
+
+Delete the name/dialect mapping associated with a string name.
+[clinic start generated code]*/
+
 static PyObject *
-csv_unregister_dialect(PyObject *module, PyObject *name_obj)
+_csv_unregister_dialect(PyObject *module, PyObject *name)
+/*[clinic end generated code: output=cf44401586cd32eb input=24474725768445c0]*/
 {
-    if (PyDict_DelItem(_csvstate_global->dialects, name_obj) < 0)
+    if (PyDict_DelItem(_csvstate_global->dialects, name) < 0)
         return PyErr_Format(_csvstate_global->error_obj, "unknown dialect");
-    Py_INCREF(Py_None);
-    return Py_None;
+    Py_RETURN_NONE;
 }
 
+
+/*[clinic input]
+_csv.get_dialect
+
+    name: object
+    /
+
+Return the dialect instance associated with name.
+[clinic start generated code]*/
+
 static PyObject *
-csv_get_dialect(PyObject *module, PyObject *name_obj)
+_csv_get_dialect(PyObject *module, PyObject *name)
+/*[clinic end generated code: output=8f66fc0a27b68e24 input=c97af3b62e402afa]*/
 {
-    return get_dialect_from_registry(name_obj);
+    return get_dialect_from_registry(name);
 }
 
+
+/*[clinic input]
+_csv.field_size_limit
+
+    [
+    limit: object
+    ]
+    /
+
+Sets an upper limit on parsed fields.
+
+Returns old limit. If limit is not given, no new limit is set and
+the old limit is returned.
+[clinic start generated code]*/
+
 static PyObject *
-csv_field_size_limit(PyObject *module, PyObject *args)
+_csv_field_size_limit(PyObject *module, int group_right_1, PyObject *limit)
+/*[clinic end generated code: output=49e0056f0b1e96f7 input=76b66600a7195ac4]*/
 {
-    PyObject *new_limit = NULL;
     long old_limit = _csvstate_global->field_limit;
 
-    if (!PyArg_UnpackTuple(args, "field_size_limit", 0, 1, &new_limit))
-        return NULL;
-    if (new_limit != NULL) {
-        if (!PyLong_CheckExact(new_limit)) {
+    if (group_right_1) {
+        if (!PyLong_CheckExact(limit)) {
             PyErr_Format(PyExc_TypeError,
                          "limit must be an integer");
             return NULL;
         }
-        _csvstate_global->field_limit = PyLong_AsLong(new_limit);
+        _csvstate_global->field_limit = PyLong_AsLong(limit);
         if (_csvstate_global->field_limit == -1 && PyErr_Occurred()) {
             _csvstate_global->field_limit = old_limit;
             return NULL;
@@ -1583,44 +1631,22 @@ PyDoc_STRVAR(csv_writer_doc,
 "\n"
 "The \"fileobj\" argument can be any object that supports the file API.\n");
 
-PyDoc_STRVAR(csv_list_dialects_doc,
-"Return a list of all know dialect names.\n"
-"    names = csv.list_dialects()");
-
-PyDoc_STRVAR(csv_get_dialect_doc,
-"Return the dialect instance associated with name.\n"
-"    dialect = csv.get_dialect(name)");
-
 PyDoc_STRVAR(csv_register_dialect_doc,
 "Create a mapping from a string name to a dialect class.\n"
 "    dialect = csv.register_dialect(name[, dialect[, **fmtparams]])");
 
-PyDoc_STRVAR(csv_unregister_dialect_doc,
-"Delete the name/dialect mapping associated with a string name.\n"
-"    csv.unregister_dialect(name)");
-
-PyDoc_STRVAR(csv_field_size_limit_doc,
-"Sets an upper limit on parsed fields.\n"
-"    csv.field_size_limit([limit])\n"
-"\n"
-"Returns old limit. If limit is not given, no new limit is set and\n"
-"the old limit is returned");
 
 static struct PyMethodDef csv_methods[] = {
     { "reader", (PyCFunction)csv_reader,
         METH_VARARGS | METH_KEYWORDS, csv_reader_doc},
     { "writer", (PyCFunction)csv_writer,
         METH_VARARGS | METH_KEYWORDS, csv_writer_doc},
-    { "list_dialects", (PyCFunction)csv_list_dialects,
-        METH_NOARGS, csv_list_dialects_doc},
     { "register_dialect", (PyCFunction)csv_register_dialect,
-        METH_VARARGS | METH_KEYWORDS, csv_register_dialect_doc},
-    { "unregister_dialect", (PyCFunction)csv_unregister_dialect,
-        METH_O, csv_unregister_dialect_doc},
-    { "get_dialect", (PyCFunction)csv_get_dialect,
-        METH_O, csv_get_dialect_doc},
-    { "field_size_limit", (PyCFunction)csv_field_size_limit,
-        METH_VARARGS, csv_field_size_limit_doc},
+      METH_VARARGS | METH_KEYWORDS, csv_register_dialect_doc},
+    _CSV_LIST_DIALECTS_METHODDEF
+    _CSV_UNREGISTER_DIALECT_METHODDEF
+    _CSV_GET_DIALECT_METHODDEF
+    _CSV_FIELD_SIZE_LIMIT_METHODDEF
     { NULL, NULL }
 };
 
diff -r f44f44b14dfc Modules/clinic/_csv.c.h
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/Modules/clinic/_csv.c.h	Fri Jan 20 18:17:14 2017 +0200
@@ -0,0 +1,50 @@
+/*[clinic input]
+preserve
+[clinic start generated code]*/
+
+PyDoc_STRVAR(_csv_list_dialects__doc__,
+"list_dialects($module, /)\n"
+"--\n"
+"\n"
+"Return a list of all know dialect names.");
+
+#define _CSV_LIST_DIALECTS_METHODDEF    \
+    {"list_dialects", (PyCFunction)_csv_list_dialects, METH_NOARGS, _csv_list_dialects__doc__},
+
+static PyObject *
+_csv_list_dialects_impl(PyObject *module);
+
+static PyObject *
+_csv_list_dialects(PyObject *module, PyObject *Py_UNUSED(ignored))
+{
+    return _csv_list_dialects_impl(module);
+}
+
+PyDoc_STRVAR(_csv_unregister_dialect__doc__,
+"unregister_dialect($module, name, /)\n"
+"--\n"
+"\n"
+"Delete the name/dialect mapping associated with a string name.");
+
+#define _CSV_UNREGISTER_DIALECT_METHODDEF    \
+    {"unregister_dialect", (PyCFunction)_csv_unregister_dialect, METH_O, _csv_unregister_dialect__doc__},
+
+PyDoc_STRVAR(_csv_get_dialect__doc__,
+"get_dialect($module, name, /)\n"
+"--\n"
+"\n"
+"Return the dialect instance associated with name.");
+
+#define _CSV_GET_DIALECT_METHODDEF    \
+    {"get_dialect", (PyCFunction)_csv_get_dialect, METH_O, _csv_get_dialect__doc__},
+
+PyDoc_STRVAR(_csv_field_size_limit__doc__,
+"field_size_limit([limit])\n"
+"Sets an upper limit on parsed fields.\n"
+"\n"
+"Returns old limit. If limit is not given, no new limit is set and\n"
+"the old limit is returned.");
+
+#define _CSV_FIELD_SIZE_LIMIT_METHODDEF    \
+    {"field_size_limit", (PyCFunction)_csv_field_size_limit, METH_O, _csv_field_size_limit__doc__},
+/*[clinic end generated code: output=ce2035869742b7db input=a9049054013a1b77]*/
-------------- next part --------------
diff -r f44f44b14dfc Modules/_lsprof.c
--- a/Modules/_lsprof.c	Fri Jan 20 08:35:18 2017 +0200
+++ b/Modules/_lsprof.c	Fri Jan 20 18:02:02 2017 +0200
@@ -2,6 +2,12 @@
 #include "frameobject.h"
 #include "rotatingtree.h"
 
+/*[clinic input]
+module _lsprof
+class _lsprof.Profiler "ProfilerObject *" "&PyProfiler_Type"
+[clinic start generated code]*/
+/*[clinic end generated code: output=da39a3ee5e6b4b0d input=dea500b5ae122746]*/
+
 /*** Selection of a high-precision timer ***/
 
 #ifdef MS_WINDOWS
@@ -601,46 +607,47 @@ static int statsForEntry(rotating_node_t
     return err;
 }
 
-PyDoc_STRVAR(getstats_doc, "\
-getstats() -> list of profiler_entry objects\n\
-\n\
-Return all information collected by the profiler.\n\
-Each profiler_entry is a tuple-like object with the\n\
-following attributes:\n\
-\n\
-    code          code object\n\
-    callcount     how many times this was called\n\
-    reccallcount  how many times called recursively\n\
-    totaltime     total time in this entry\n\
-    inlinetime    inline time in this entry (not in subcalls)\n\
-    calls         details of the calls\n\
-\n\
-The calls attribute is either None or a list of\n\
-profiler_subentry objects:\n\
-\n\
-    code          called code object\n\
-    callcount     how many times this is called\n\
-    reccallcount  how many times this is called recursively\n\
-    totaltime     total time spent in this call\n\
-    inlinetime    inline time (not in further subcalls)\n\
-");
 
-static PyObject*
-profiler_getstats(ProfilerObject *pObj, PyObject* noarg)
+/*[clinic input]
+_lsprof.Profiler.getstats
+
+Return all information collected by the profiler.
+
+Each profiler_entry is a tuple-like object with the following attributes:
+
+    code          code object
+    callcount     how many times this was called
+    reccallcount  how many times called recursively
+    totaltime     total time in this entry
+    inlinetime    inline time in this entry (not in subcalls)
+    calls         details of the calls
+
+The calls attribute is either None or a list of profiler_subentry objects:
+
+    code          called code object
+    callcount     how many times this is called
+    reccallcount  how many times this is called recursively
+    totaltime     total time spent in this call
+    inlinetime    inline time (not in further subcalls)
+[clinic start generated code]*/
+
+static PyObject *
+_lsprof_Profiler_getstats_impl(ProfilerObject *self)
+/*[clinic end generated code: output=9461b451e9ef0f24 input=a6d5ede1f4cb3f71]*/
 {
     statscollector_t collect;
-    if (pending_exception(pObj))
+    if (pending_exception(self))
         return NULL;
-    if (!pObj->externalTimer)
+    if (!self->externalTimer)
         collect.factor = hpTimerUnit();
-    else if (pObj->externalTimerUnit > 0.0)
-        collect.factor = pObj->externalTimerUnit;
+    else if (self->externalTimerUnit > 0.0)
+        collect.factor = self->externalTimerUnit;
     else
         collect.factor = 1.0 / DOUBLE_TIMER_PRECISION;
     collect.list = PyList_New(0);
     if (collect.list == NULL)
         return NULL;
-    if (RotatingTree_Enum(pObj->profilerEntries, statsForEntry, &collect)
+    if (RotatingTree_Enum(self->profilerEntries, statsForEntry, &collect)
         != 0) {
         Py_DECREF(collect.list);
         return NULL;
@@ -669,31 +676,32 @@ setBuiltins(ProfilerObject *pObj, int nv
     return 0;
 }
 
-PyDoc_STRVAR(enable_doc, "\
-enable(subcalls=True, builtins=True)\n\
-\n\
-Start collecting profiling information.\n\
-If 'subcalls' is True, also records for each function\n\
-statistics separated according to its current caller.\n\
-If 'builtins' is True, records the time spent in\n\
-built-in functions separately from their caller.\n\
-");
 
-static PyObject*
-profiler_enable(ProfilerObject *self, PyObject *args, PyObject *kwds)
+/*[clinic input]
+_lsprof.Profiler.enable
+
+    subcalls: int = -1
+    builtins: int = -1
+    /
+
+Start collecting profiling information.
+
+If 'subcalls' is True, also records for each function
+statistics separated according to its current caller.
+If 'builtins' is True, records the time spent in
+built-in functions separately from their caller.
+[clinic start generated code]*/
+
+static PyObject *
+_lsprof_Profiler_enable_impl(ProfilerObject *self, int subcalls,
+                             int builtins)
+/*[clinic end generated code: output=1e747f9dc1edd571 input=7443f0f3799d770a]*/
 {
-    int subcalls = -1;
-    int builtins = -1;
-    static char *kwlist[] = {"subcalls", "builtins", 0};
-    if (!PyArg_ParseTupleAndKeywords(args, kwds, "|ii:enable",
-                                     kwlist, &subcalls, &builtins))
-        return NULL;
     if (setSubcalls(self, subcalls) < 0 || setBuiltins(self, builtins) < 0)
         return NULL;
     PyEval_SetProfile(profiler_callback, (PyObject*)self);
     self->flags |= POF_ENABLED;
-    Py_INCREF(Py_None);
-    return Py_None;
+    Py_RETURN_NONE;
 }
 
 static void
@@ -712,36 +720,38 @@ flush_unmatched(ProfilerObject *pObj)
 
 }
 
-PyDoc_STRVAR(disable_doc, "\
-disable()\n\
-\n\
-Stop collecting profiling information.\n\
-");
 
-static PyObject*
-profiler_disable(ProfilerObject *self, PyObject* noarg)
+/*[clinic input]
+_lsprof.Profiler.disable
+
+Stop collecting profiling information.
+[clinic start generated code]*/
+
+static PyObject *
+_lsprof_Profiler_disable_impl(ProfilerObject *self)
+/*[clinic end generated code: output=838cffef7f651870 input=05700b3fc68d1f50]*/
 {
     self->flags &= ~POF_ENABLED;
     PyEval_SetProfile(NULL, NULL);
     flush_unmatched(self);
     if (pending_exception(self))
         return NULL;
-    Py_INCREF(Py_None);
-    return Py_None;
+    Py_RETURN_NONE;
 }
 
-PyDoc_STRVAR(clear_doc, "\
-clear()\n\
-\n\
-Clear all profiling information collected so far.\n\
-");
 
-static PyObject*
-profiler_clear(ProfilerObject *pObj, PyObject* noarg)
+/*[clinic input]
+_lsprof.Profiler.clear
+
+Clear all profiling information collected so far.
+[clinic start generated code]*/
+
+static PyObject *
+_lsprof_Profiler_clear_impl(ProfilerObject *self)
+/*[clinic end generated code: output=dd1c668fb84b1335 input=fbe1f88c28be4f98]*/
 {
-    clearEntries(pObj);
-    Py_INCREF(Py_None);
-    return Py_None;
+    clearEntries(self);
+    Py_RETURN_NONE;
 }
 
 static void
@@ -755,51 +765,46 @@ profiler_dealloc(ProfilerObject *op)
     Py_TYPE(op)->tp_free(op);
 }
 
+/*[clinic input]
+_lsprof.Profiler.__init__
+
+    timer: object = NULL
+    timeunit: double = 0.0
+    subcalls: int(c_default="1") = True
+    builtins: int(c_default="1") = True
+    /
+
+Builds a profiler object using the specified timer function.
+
+The default timer is a fast built-in one based on real time.
+For custom timer functions returning integers, timeunit can
+be a float specifying a scale (i.e. how long each integer unit
+is, in seconds).
+[clinic start generated code]*/
+
 static int
-profiler_init(ProfilerObject *pObj, PyObject *args, PyObject *kw)
+_lsprof_Profiler___init___impl(ProfilerObject *self, PyObject *timer,
+                               double timeunit, int subcalls, int builtins)
+/*[clinic end generated code: output=ab5498359fd34283 input=310726c90ffbcfa1]*/
 {
-    PyObject *timer = NULL;
-    double timeunit = 0.0;
-    int subcalls = 1;
-    int builtins = 1;
-    static char *kwlist[] = {"timer", "timeunit",
-                                   "subcalls", "builtins", 0};
-
-    if (!PyArg_ParseTupleAndKeywords(args, kw, "|Odii:Profiler", kwlist,
-                                     &timer, &timeunit,
-                                     &subcalls, &builtins))
+    if (setSubcalls(self, subcalls) < 0 || setBuiltins(self, builtins) < 0)
         return -1;
-
-    if (setSubcalls(pObj, subcalls) < 0 || setBuiltins(pObj, builtins) < 0)
-        return -1;
-    pObj->externalTimerUnit = timeunit;
+    self->externalTimerUnit = timeunit;
     Py_XINCREF(timer);
-    Py_XSETREF(pObj->externalTimer, timer);
+    Py_XSETREF(self->externalTimer, timer);
     return 0;
 }
 
+#include "clinic/_lsprof.c.h"
+
 static PyMethodDef profiler_methods[] = {
-    {"getstats",    (PyCFunction)profiler_getstats,
-                    METH_NOARGS,                        getstats_doc},
-    {"enable",          (PyCFunction)profiler_enable,
-                    METH_VARARGS | METH_KEYWORDS,       enable_doc},
-    {"disable",         (PyCFunction)profiler_disable,
-                    METH_NOARGS,                        disable_doc},
-    {"clear",           (PyCFunction)profiler_clear,
-                    METH_NOARGS,                        clear_doc},
+    _LSPROF_PROFILER_GETSTATS_METHODDEF
+    _LSPROF_PROFILER_ENABLE_METHODDEF
+    _LSPROF_PROFILER_DISABLE_METHODDEF
+    _LSPROF_PROFILER_CLEAR_METHODDEF
     {NULL, NULL}
 };
 
-PyDoc_STRVAR(profiler_doc, "\
-Profiler(custom_timer=None, time_unit=None, subcalls=True, builtins=True)\n\
-\n\
-    Builds a profiler object using the specified timer function.\n\
-    The default timer is a fast built-in one based on real time.\n\
-    For custom timer functions returning integers, time_unit can\n\
-    be a float specifying a scale (i.e. how long each integer unit\n\
-    is, in seconds).\n\
-");
-
 static PyTypeObject PyProfiler_Type = {
     PyVarObject_HEAD_INIT(NULL, 0)
     "_lsprof.Profiler",                     /* tp_name */
@@ -821,7 +826,7 @@ static PyTypeObject PyProfiler_Type = {
     0,                                      /* tp_setattro */
     0,                                      /* tp_as_buffer */
     Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE, /* tp_flags */
-    profiler_doc,                           /* tp_doc */
+    _lsprof_Profiler___init____doc__,       /* tp_doc */
     0,                                      /* tp_traverse */
     0,                                      /* tp_clear */
     0,                                      /* tp_richcompare */
@@ -836,7 +841,7 @@ static PyTypeObject PyProfiler_Type = {
     0,                                      /* tp_descr_get */
     0,                                      /* tp_descr_set */
     0,                                      /* tp_dictoffset */
-    (initproc)profiler_init,                /* tp_init */
+    _lsprof_Profiler___init__,              /* tp_init */
     PyType_GenericAlloc,                    /* tp_alloc */
     PyType_GenericNew,                      /* tp_new */
     PyObject_Del,                           /* tp_free */
diff -r f44f44b14dfc Modules/clinic/_lsprof.c.h
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/Modules/clinic/_lsprof.c.h	Fri Jan 20 18:02:02 2017 +0200
@@ -0,0 +1,152 @@
+/*[clinic input]
+preserve
+[clinic start generated code]*/
+
+PyDoc_STRVAR(_lsprof_Profiler_getstats__doc__,
+"getstats($self, /)\n"
+"--\n"
+"\n"
+"Return all information collected by the profiler.\n"
+"\n"
+"Each profiler_entry is a tuple-like object with the following attributes:\n"
+"\n"
+"    code          code object\n"
+"    callcount     how many times this was called\n"
+"    reccallcount  how many times called recursively\n"
+"    totaltime     total time in this entry\n"
+"    inlinetime    inline time in this entry (not in subcalls)\n"
+"    calls         details of the calls\n"
+"\n"
+"The calls attribute is either None or a list of profiler_subentry objects:\n"
+"\n"
+"    code          called code object\n"
+"    callcount     how many times this is called\n"
+"    reccallcount  how many times this is called recursively\n"
+"    totaltime     total time spent in this call\n"
+"    inlinetime    inline time (not in further subcalls)");
+
+#define _LSPROF_PROFILER_GETSTATS_METHODDEF    \
+    {"getstats", (PyCFunction)_lsprof_Profiler_getstats, METH_NOARGS, _lsprof_Profiler_getstats__doc__},
+
+static PyObject *
+_lsprof_Profiler_getstats_impl(ProfilerObject *self);
+
+static PyObject *
+_lsprof_Profiler_getstats(ProfilerObject *self, PyObject *Py_UNUSED(ignored))
+{
+    return _lsprof_Profiler_getstats_impl(self);
+}
+
+PyDoc_STRVAR(_lsprof_Profiler_enable__doc__,
+"enable($self, subcalls=-1, builtins=-1, /)\n"
+"--\n"
+"\n"
+"Start collecting profiling information.\n"
+"\n"
+"If \'subcalls\' is True, also records for each function\n"
+"statistics separated according to its current caller.\n"
+"If \'builtins\' is True, records the time spent in\n"
+"built-in functions separately from their caller.");
+
+#define _LSPROF_PROFILER_ENABLE_METHODDEF    \
+    {"enable", (PyCFunction)_lsprof_Profiler_enable, METH_FASTCALL, _lsprof_Profiler_enable__doc__},
+
+static PyObject *
+_lsprof_Profiler_enable_impl(ProfilerObject *self, int subcalls,
+                             int builtins);
+
+static PyObject *
+_lsprof_Profiler_enable(ProfilerObject *self, PyObject **args, Py_ssize_t nargs, PyObject *kwnames)
+{
+    PyObject *return_value = NULL;
+    int subcalls = -1;
+    int builtins = -1;
+
+    if (!_PyArg_ParseStack(args, nargs, "|ii:enable",
+        &subcalls, &builtins)) {
+        goto exit;
+    }
+
+    if (!_PyArg_NoStackKeywords("enable", kwnames)) {
+        goto exit;
+    }
+    return_value = _lsprof_Profiler_enable_impl(self, subcalls, builtins);
+
+exit:
+    return return_value;
+}
+
+PyDoc_STRVAR(_lsprof_Profiler_disable__doc__,
+"disable($self, /)\n"
+"--\n"
+"\n"
+"Stop collecting profiling information.");
+
+#define _LSPROF_PROFILER_DISABLE_METHODDEF    \
+    {"disable", (PyCFunction)_lsprof_Profiler_disable, METH_NOARGS, _lsprof_Profiler_disable__doc__},
+
+static PyObject *
+_lsprof_Profiler_disable_impl(ProfilerObject *self);
+
+static PyObject *
+_lsprof_Profiler_disable(ProfilerObject *self, PyObject *Py_UNUSED(ignored))
+{
+    return _lsprof_Profiler_disable_impl(self);
+}
+
+PyDoc_STRVAR(_lsprof_Profiler_clear__doc__,
+"clear($self, /)\n"
+"--\n"
+"\n"
+"Clear all profiling information collected so far.");
+
+#define _LSPROF_PROFILER_CLEAR_METHODDEF    \
+    {"clear", (PyCFunction)_lsprof_Profiler_clear, METH_NOARGS, _lsprof_Profiler_clear__doc__},
+
+static PyObject *
+_lsprof_Profiler_clear_impl(ProfilerObject *self);
+
+static PyObject *
+_lsprof_Profiler_clear(ProfilerObject *self, PyObject *Py_UNUSED(ignored))
+{
+    return _lsprof_Profiler_clear_impl(self);
+}
+
+PyDoc_STRVAR(_lsprof_Profiler___init____doc__,
+"Profiler(timer=None, timeunit=0.0, subcalls=True, builtins=True, /)\n"
+"--\n"
+"\n"
+"Builds a profiler object using the specified timer function.\n"
+"\n"
+"The default timer is a fast built-in one based on real time.\n"
+"For custom timer functions returning integers, timeunit can\n"
+"be a float specifying a scale (i.e. how long each integer unit\n"
+"is, in seconds).");
+
+static int
+_lsprof_Profiler___init___impl(ProfilerObject *self, PyObject *timer,
+                               double timeunit, int subcalls, int builtins);
+
+static int
+_lsprof_Profiler___init__(PyObject *self, PyObject *args, PyObject *kwargs)
+{
+    int return_value = -1;
+    PyObject *timer = NULL;
+    double timeunit = 0.0;
+    int subcalls = 1;
+    int builtins = 1;
+
+    if ((Py_TYPE(self) == &PyProfiler_Type) &&
+        !_PyArg_NoKeywords("Profiler", kwargs)) {
+        goto exit;
+    }
+    if (!PyArg_ParseTuple(args, "|Odii:Profiler",
+        &timer, &timeunit, &subcalls, &builtins)) {
+        goto exit;
+    }
+    return_value = _lsprof_Profiler___init___impl((ProfilerObject *)self, timer, timeunit, subcalls, builtins);
+
+exit:
+    return return_value;
+}
+/*[clinic end generated code: output=205f6aa40ac0e5b7 input=a9049054013a1b77]*/
-------------- next part --------------
diff -r f44f44b14dfc Modules/_tracemalloc.c
--- a/Modules/_tracemalloc.c	Fri Jan 20 08:35:18 2017 +0200
+++ b/Modules/_tracemalloc.c	Fri Jan 20 18:28:16 2017 +0200
@@ -4,6 +4,12 @@
 #include "pythread.h"
 #include "osdefs.h"
 
+#include "clinic/_tracemalloc.c.h"
+/*[clinic input]
+module _tracemalloc
+[clinic start generated code]*/
+/*[clinic end generated code: output=da39a3ee5e6b4b0d input=708a98302fc46e5f]*/
+
 /* Trace memory blocks allocated by PyMem_RawMalloc() */
 #define TRACE_RAW_MALLOC
 
@@ -1161,27 +1167,31 @@ tracemalloc_stop(void)
     tracemalloc_traceback = NULL;
 }
 
-PyDoc_STRVAR(tracemalloc_is_tracing_doc,
-    "is_tracing()->bool\n"
-    "\n"
-    "True if the tracemalloc module is tracing Python memory allocations,\n"
-    "False otherwise.");
 
 
-static PyObject*
-py_tracemalloc_is_tracing(PyObject *self)
+/*[clinic input]
+_tracemalloc.is_tracing
+
+Return True if the tracemalloc module is tracing Python memory allocations.
+[clinic start generated code]*/
+
+static PyObject *
+_tracemalloc_is_tracing_impl(PyObject *module)
+/*[clinic end generated code: output=2d763b42601cd3ef input=8beb4fb5446813be]*/
 {
     return PyBool_FromLong(tracemalloc_config.tracing);
 }
 
-PyDoc_STRVAR(tracemalloc_clear_traces_doc,
-    "clear_traces()\n"
-    "\n"
-    "Clear traces of memory blocks allocated by Python.");
 
+/*[clinic input]
+_tracemalloc.clear_traces
 
-static PyObject*
-py_tracemalloc_clear_traces(PyObject *self)
+Clear traces of memory blocks allocated by Python.
+[clinic start generated code]*/
+
+static PyObject *
+_tracemalloc_clear_traces_impl(PyObject *module)
+/*[clinic end generated code: output=a86080ee41b84197 input=0dab5b6c785183a5]*/
 {
     if (!tracemalloc_config.tracing)
         Py_RETURN_NONE;
@@ -1343,17 +1353,21 @@ tracemalloc_pyobject_decref_cb(_Py_hasht
 }
 
 
-PyDoc_STRVAR(tracemalloc_get_traces_doc,
-    "_get_traces() -> list\n"
-    "\n"
-    "Get traces of all memory blocks allocated by Python.\n"
-    "Return a list of (size: int, traceback: tuple) tuples.\n"
-    "traceback is a tuple of (filename: str, lineno: int) tuples.\n"
-    "\n"
-    "Return an empty list if the tracemalloc module is disabled.");
 
-static PyObject*
-py_tracemalloc_get_traces(PyObject *self, PyObject *obj)
+/*[clinic input]
+_tracemalloc._get_traces
+
+Get traces of all memory blocks allocated by Python.
+
+Return a list of (size: int, traceback: tuple) tuples.
+traceback is a tuple of (filename: str, lineno: int) tuples.
+
+Return an empty list if the tracemalloc module is disabled.
+[clinic start generated code]*/
+
+static PyObject *
+_tracemalloc__get_traces_impl(PyObject *module)
+/*[clinic end generated code: output=e9929876ced4b5cc input=6c7d2230b24255aa]*/
 {
     get_traces_t get_traces;
     int err;
@@ -1439,17 +1453,23 @@ tracemalloc_get_traceback(_PyTraceMalloc
 }
 
 
-PyDoc_STRVAR(tracemalloc_get_object_traceback_doc,
-    "_get_object_traceback(obj)\n"
-    "\n"
-    "Get the traceback where the Python object obj was allocated.\n"
-    "Return a tuple of (filename: str, lineno: int) tuples.\n"
-    "\n"
-    "Return None if the tracemalloc module is disabled or did not\n"
-    "trace the allocation of the object.");
 
-static PyObject*
-py_tracemalloc_get_object_traceback(PyObject *self, PyObject *obj)
+/*[clinic input]
+_tracemalloc._get_object_traceback
+
+    obj: object
+    /
+
+Get the traceback where the Python object obj was allocated.
+
+Return a tuple of (filename: str, lineno: int) tuples.
+Return None if the tracemalloc module is disabled or did not
+trace the allocation of the object.
+[clinic start generated code]*/
+
+static PyObject *
+_tracemalloc__get_object_traceback(PyObject *module, PyObject *obj)
+/*[clinic end generated code: output=41ee0553a658b0aa input=29495f1b21c53212]*/
 {
     PyTypeObject *type;
     void *ptr;
@@ -1503,25 +1523,29 @@ void
 #undef PUTS
 
 
-PyDoc_STRVAR(tracemalloc_start_doc,
-    "start(nframe: int=1)\n"
-    "\n"
-    "Start tracing Python memory allocations. Set also the maximum number \n"
-    "of frames stored in the traceback of a trace to nframe.");
 
-static PyObject*
-py_tracemalloc_start(PyObject *self, PyObject *args)
+/*[clinic input]
+_tracemalloc.start
+
+    nframe: Py_ssize_t = 1
+    /
+
+Start tracing Python memory allocations.
+
+Also set the maximum number of frames stored in the traceback of a
+trace to nframe.
+[clinic start generated code]*/
+
+static PyObject *
+_tracemalloc_start_impl(PyObject *module, Py_ssize_t nframe)
+/*[clinic end generated code: output=0f558d2079511553 input=997841629cc441cb]*/
 {
-    Py_ssize_t nframe = 1;
     int nframe_int;
 
-    if (!PyArg_ParseTuple(args, "|n:start", &nframe))
-        return NULL;
-
     if (nframe < 1 || nframe > MAX_NFRAME) {
         PyErr_Format(PyExc_ValueError,
                      "the number of frames must be in range [1; %i]",
-                     MAX_NFRAME);
+                     (int)MAX_NFRAME);
         return NULL;
     }
     nframe_int = Py_SAFE_DOWNCAST(nframe, Py_ssize_t, int);
@@ -1532,48 +1556,55 @@ py_tracemalloc_start(PyObject *self, PyO
     Py_RETURN_NONE;
 }
 
-PyDoc_STRVAR(tracemalloc_stop_doc,
-    "stop()\n"
-    "\n"
-    "Stop tracing Python memory allocations and clear traces\n"
-    "of memory blocks allocated by Python.");
 
+/*[clinic input]
+_tracemalloc.stop
 
-static PyObject*
-py_tracemalloc_stop(PyObject *self)
+Stop tracing Python memory allocations.
+
+Also clear traces of memory blocks allocated by Python.
+[clinic start generated code]*/
+
+static PyObject *
+_tracemalloc_stop_impl(PyObject *module)
+/*[clinic end generated code: output=c3c42ae03e3955cd input=7478f075e51dae18]*/
 {
     tracemalloc_stop();
     Py_RETURN_NONE;
 }
 
 
-PyDoc_STRVAR(tracemalloc_get_traceback_limit_doc,
-    "get_traceback_limit() -> int\n"
-    "\n"
-    "Get the maximum number of frames stored in the traceback\n"
-    "of a trace.\n"
-    "\n"
-    "By default, a trace of an allocated memory block only stores\n"
-    "the most recent frame: the limit is 1.");
+/*[clinic input]
+_tracemalloc.get_traceback_limit
 
-static PyObject*
-py_tracemalloc_get_traceback_limit(PyObject *self)
+Get the maximum number of frames stored in the traceback of a trace.
+
+By default, a trace of an allocated memory block only stores
+the most recent frame: the limit is 1.
+[clinic start generated code]*/
+
+static PyObject *
+_tracemalloc_get_traceback_limit_impl(PyObject *module)
+/*[clinic end generated code: output=d556d9306ba95567 input=da3cd977fc68ae3b]*/
 {
     return PyLong_FromLong(tracemalloc_config.max_nframe);
 }
 
 
-PyDoc_STRVAR(tracemalloc_get_tracemalloc_memory_doc,
-    "get_tracemalloc_memory() -> int\n"
-    "\n"
-    "Get the memory usage in bytes of the tracemalloc module\n"
-    "used internally to trace memory allocations.");
 
-static PyObject*
-tracemalloc_get_tracemalloc_memory(PyObject *self)
+/*[clinic input]
+_tracemalloc.get_tracemalloc_memory
+
+Get the memory usage in bytes of the tracemalloc module.
+
+This memory is used internally to trace memory allocations.
+[clinic start generated code]*/
+
+static PyObject *
+_tracemalloc_get_tracemalloc_memory_impl(PyObject *module)
+/*[clinic end generated code: output=e3f14e280a55f5aa input=5d919c0f4d5132ad]*/
 {
     size_t size;
-    PyObject *size_obj;
 
     size = _Py_hashtable_size(tracemalloc_tracebacks);
     size += _Py_hashtable_size(tracemalloc_filenames);
@@ -1582,22 +1613,24 @@ tracemalloc_get_tracemalloc_memory(PyObj
     size += _Py_hashtable_size(tracemalloc_traces);
     TABLES_UNLOCK();
 
-    size_obj = PyLong_FromSize_t(size);
-    return Py_BuildValue("N", size_obj);
+    return PyLong_FromSize_t(size);
 }
 
 
-PyDoc_STRVAR(tracemalloc_get_traced_memory_doc,
-    "get_traced_memory() -> (int, int)\n"
-    "\n"
-    "Get the current size and peak size of memory blocks traced\n"
-    "by the tracemalloc module as a tuple: (current: int, peak: int).");
 
-static PyObject*
-tracemalloc_get_traced_memory(PyObject *self)
+/*[clinic input]
+_tracemalloc.get_traced_memory
+
+Get the current size and peak size of memory blocks traced by tracemalloc.
+
+Returns a tuple: (current: int, peak: int).
+[clinic start generated code]*/
+
+static PyObject *
+_tracemalloc_get_traced_memory_impl(PyObject *module)
+/*[clinic end generated code: output=5b167189adb9e782 input=61ddb5478400ff66]*/
 {
     Py_ssize_t size, peak_size;
-    PyObject *size_obj, *peak_size_obj;
 
     if (!tracemalloc_config.tracing)
         return Py_BuildValue("ii", 0, 0);
@@ -1607,32 +1640,20 @@ tracemalloc_get_traced_memory(PyObject *
     peak_size = tracemalloc_peak_traced_memory;
     TABLES_UNLOCK();
 
-    size_obj = PyLong_FromSize_t(size);
-    peak_size_obj = PyLong_FromSize_t(peak_size);
-    return Py_BuildValue("NN", size_obj, peak_size_obj);
+    return Py_BuildValue("nn", size, peak_size);
 }
 
 
 static PyMethodDef module_methods[] = {
-    {"is_tracing", (PyCFunction)py_tracemalloc_is_tracing,
-     METH_NOARGS, tracemalloc_is_tracing_doc},
-    {"clear_traces", (PyCFunction)py_tracemalloc_clear_traces,
-     METH_NOARGS, tracemalloc_clear_traces_doc},
-    {"_get_traces", (PyCFunction)py_tracemalloc_get_traces,
-     METH_NOARGS, tracemalloc_get_traces_doc},
-    {"_get_object_traceback", (PyCFunction)py_tracemalloc_get_object_traceback,
-     METH_O, tracemalloc_get_object_traceback_doc},
-    {"start", (PyCFunction)py_tracemalloc_start,
-      METH_VARARGS, tracemalloc_start_doc},
-    {"stop", (PyCFunction)py_tracemalloc_stop,
-      METH_NOARGS, tracemalloc_stop_doc},
-    {"get_traceback_limit", (PyCFunction)py_tracemalloc_get_traceback_limit,
-     METH_NOARGS, tracemalloc_get_traceback_limit_doc},
-    {"get_tracemalloc_memory", (PyCFunction)tracemalloc_get_tracemalloc_memory,
-     METH_NOARGS, tracemalloc_get_tracemalloc_memory_doc},
-    {"get_traced_memory", (PyCFunction)tracemalloc_get_traced_memory,
-     METH_NOARGS, tracemalloc_get_traced_memory_doc},
-
+    _TRACEMALLOC_IS_TRACING_METHODDEF
+    _TRACEMALLOC_CLEAR_TRACES_METHODDEF
+    _TRACEMALLOC__GET_TRACES_METHODDEF
+    _TRACEMALLOC__GET_OBJECT_TRACEBACK_METHODDEF
+    _TRACEMALLOC_START_METHODDEF
+    _TRACEMALLOC_STOP_METHODDEF
+    _TRACEMALLOC_GET_TRACEBACK_LIMIT_METHODDEF
+    _TRACEMALLOC_GET_TRACEMALLOC_MEMORY_METHODDEF
+    _TRACEMALLOC_GET_TRACED_MEMORY_METHODDEF
     /* sentinel */
     {NULL, NULL}
 };
diff -r f44f44b14dfc Modules/clinic/_tracemalloc.c.h
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/Modules/clinic/_tracemalloc.c.h	Fri Jan 20 18:28:16 2017 +0200
@@ -0,0 +1,192 @@
+/*[clinic input]
+preserve
+[clinic start generated code]*/
+
+PyDoc_STRVAR(_tracemalloc_is_tracing__doc__,
+"is_tracing($module, /)\n"
+"--\n"
+"\n"
+"True if the tracemalloc module is tracing Python memory allocations, False otherwise.");
+
+#define _TRACEMALLOC_IS_TRACING_METHODDEF    \
+    {"is_tracing", (PyCFunction)_tracemalloc_is_tracing, METH_NOARGS, _tracemalloc_is_tracing__doc__},
+
+static PyObject *
+_tracemalloc_is_tracing_impl(PyObject *module);
+
+static PyObject *
+_tracemalloc_is_tracing(PyObject *module, PyObject *Py_UNUSED(ignored))
+{
+    return _tracemalloc_is_tracing_impl(module);
+}
+
+PyDoc_STRVAR(_tracemalloc_clear_traces__doc__,
+"clear_traces($module, /)\n"
+"--\n"
+"\n"
+"Clear traces of memory blocks allocated by Python.");
+
+#define _TRACEMALLOC_CLEAR_TRACES_METHODDEF    \
+    {"clear_traces", (PyCFunction)_tracemalloc_clear_traces, METH_NOARGS, _tracemalloc_clear_traces__doc__},
+
+static PyObject *
+_tracemalloc_clear_traces_impl(PyObject *module);
+
+static PyObject *
+_tracemalloc_clear_traces(PyObject *module, PyObject *Py_UNUSED(ignored))
+{
+    return _tracemalloc_clear_traces_impl(module);
+}
+
+PyDoc_STRVAR(_tracemalloc__get_traces__doc__,
+"_get_traces($module, /)\n"
+"--\n"
+"\n"
+"Get traces of all memory blocks allocated by Python.\n"
+"\n"
+"Return a list of (size: int, traceback: tuple) tuples.\n"
+"traceback is a tuple of (filename: str, lineno: int) tuples.\n"
+"\n"
+"Return an empty list if the tracemalloc module is disabled.");
+
+#define _TRACEMALLOC__GET_TRACES_METHODDEF    \
+    {"_get_traces", (PyCFunction)_tracemalloc__get_traces, METH_NOARGS, _tracemalloc__get_traces__doc__},
+
+static PyObject *
+_tracemalloc__get_traces_impl(PyObject *module);
+
+static PyObject *
+_tracemalloc__get_traces(PyObject *module, PyObject *Py_UNUSED(ignored))
+{
+    return _tracemalloc__get_traces_impl(module);
+}
+
+PyDoc_STRVAR(_tracemalloc__get_object_traceback__doc__,
+"_get_object_traceback($module, obj, /)\n"
+"--\n"
+"\n"
+"Get the traceback where the Python object obj was allocated.\n"
+"\n"
+"Return a tuple of (filename: str, lineno: int) tuples.\n"
+"Return None if the tracemalloc module is disabled or did not\n"
+"trace the allocation of the object.");
+
+#define _TRACEMALLOC__GET_OBJECT_TRACEBACK_METHODDEF    \
+    {"_get_object_traceback", (PyCFunction)_tracemalloc__get_object_traceback, METH_O, _tracemalloc__get_object_traceback__doc__},
+
+PyDoc_STRVAR(_tracemalloc_start__doc__,
+"start($module, nframe=1, /)\n"
+"--\n"
+"\n"
+"Start tracing Python memory allocations.\n"
+"\n"
+"Also set the maximum number of frames stored in the traceback of a\n"
+"trace to nframe.");
+
+#define _TRACEMALLOC_START_METHODDEF    \
+    {"start", (PyCFunction)_tracemalloc_start, METH_FASTCALL, _tracemalloc_start__doc__},
+
+static PyObject *
+_tracemalloc_start_impl(PyObject *module, Py_ssize_t nframe);
+
+static PyObject *
+_tracemalloc_start(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames)
+{
+    PyObject *return_value = NULL;
+    Py_ssize_t nframe = 1;
+
+    if (!_PyArg_ParseStack(args, nargs, "|n:start",
+        &nframe)) {
+        goto exit;
+    }
+
+    if (!_PyArg_NoStackKeywords("start", kwnames)) {
+        goto exit;
+    }
+    return_value = _tracemalloc_start_impl(module, nframe);
+
+exit:
+    return return_value;
+}
+
+PyDoc_STRVAR(_tracemalloc_stop__doc__,
+"stop($module, /)\n"
+"--\n"
+"\n"
+"Stop tracing Python memory allocations.\n"
+"\n"
+"Also clear traces of memory blocks allocated by Python.");
+
+#define _TRACEMALLOC_STOP_METHODDEF    \
+    {"stop", (PyCFunction)_tracemalloc_stop, METH_NOARGS, _tracemalloc_stop__doc__},
+
+static PyObject *
+_tracemalloc_stop_impl(PyObject *module);
+
+static PyObject *
+_tracemalloc_stop(PyObject *module, PyObject *Py_UNUSED(ignored))
+{
+    return _tracemalloc_stop_impl(module);
+}
+
+PyDoc_STRVAR(_tracemalloc_get_traceback_limit__doc__,
+"get_traceback_limit($module, /)\n"
+"--\n"
+"\n"
+"Get the maximum number of frames stored in the traceback of a trace.\n"
+"\n"
+"By default, a trace of an allocated memory block only stores\n"
+"the most recent frame: the limit is 1.");
+
+#define _TRACEMALLOC_GET_TRACEBACK_LIMIT_METHODDEF    \
+    {"get_traceback_limit", (PyCFunction)_tracemalloc_get_traceback_limit, METH_NOARGS, _tracemalloc_get_traceback_limit__doc__},
+
+static PyObject *
+_tracemalloc_get_traceback_limit_impl(PyObject *module);
+
+static PyObject *
+_tracemalloc_get_traceback_limit(PyObject *module, PyObject *Py_UNUSED(ignored))
+{
+    return _tracemalloc_get_traceback_limit_impl(module);
+}
+
+PyDoc_STRVAR(_tracemalloc_get_tracemalloc_memory__doc__,
+"get_tracemalloc_memory($module, /)\n"
+"--\n"
+"\n"
+"Get the memory usage in bytes of the tracemalloc module.\n"
+"\n"
+"This memory is used internally to trace memory allocations.");
+
+#define _TRACEMALLOC_GET_TRACEMALLOC_MEMORY_METHODDEF    \
+    {"get_tracemalloc_memory", (PyCFunction)_tracemalloc_get_tracemalloc_memory, METH_NOARGS, _tracemalloc_get_tracemalloc_memory__doc__},
+
+static PyObject *
+_tracemalloc_get_tracemalloc_memory_impl(PyObject *module);
+
+static PyObject *
+_tracemalloc_get_tracemalloc_memory(PyObject *module, PyObject *Py_UNUSED(ignored))
+{
+    return _tracemalloc_get_tracemalloc_memory_impl(module);
+}
+
+PyDoc_STRVAR(_tracemalloc_get_traced_memory__doc__,
+"get_traced_memory($module, /)\n"
+"--\n"
+"\n"
+"Get the current size and peak size of memory blocks traced by tracemalloc.\n"
+"\n"
+"Returns a tuple: (current: int, peak: int).");
+
+#define _TRACEMALLOC_GET_TRACED_MEMORY_METHODDEF    \
+    {"get_traced_memory", (PyCFunction)_tracemalloc_get_traced_memory, METH_NOARGS, _tracemalloc_get_traced_memory__doc__},
+
+static PyObject *
+_tracemalloc_get_traced_memory_impl(PyObject *module);
+
+static PyObject *
+_tracemalloc_get_traced_memory(PyObject *module, PyObject *Py_UNUSED(ignored))
+{
+    return _tracemalloc_get_traced_memory_impl(module);
+}
+/*[clinic end generated code: output=1e059f24619e23f9 input=a9049054013a1b77]*/
-------------- next part --------------
diff -r f44f44b14dfc Modules/clinic/symtablemodule.c.h
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/Modules/clinic/symtablemodule.c.h	Fri Jan 20 18:02:54 2017 +0200
@@ -0,0 +1,39 @@
+/*[clinic input]
+preserve
+[clinic start generated code]*/
+
+PyDoc_STRVAR(_symtable_symtable__doc__,
+"symtable($module, str, filename, startstr, /)\n"
+"--\n"
+"\n"
+"Return symbol and scope dictionaries used internally by compiler.");
+
+#define _SYMTABLE_SYMTABLE_METHODDEF    \
+    {"symtable", (PyCFunction)_symtable_symtable, METH_FASTCALL, _symtable_symtable__doc__},
+
+static PyObject *
+_symtable_symtable_impl(PyObject *module, const char *str,
+                        PyObject *filename, const char *startstr);
+
+static PyObject *
+_symtable_symtable(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames)
+{
+    PyObject *return_value = NULL;
+    const char *str;
+    PyObject *filename;
+    const char *startstr;
+
+    if (!_PyArg_ParseStack(args, nargs, "sO&s:symtable",
+        &str, PyUnicode_FSDecoder, &filename, &startstr)) {
+        goto exit;
+    }
+
+    if (!_PyArg_NoStackKeywords("symtable", kwnames)) {
+        goto exit;
+    }
+    return_value = _symtable_symtable_impl(module, str, filename, startstr);
+
+exit:
+    return return_value;
+}
+/*[clinic end generated code: output=071dee4d836e2cfd input=a9049054013a1b77]*/
diff -r f44f44b14dfc Modules/symtablemodule.c
--- a/Modules/symtablemodule.c	Fri Jan 20 08:35:18 2017 +0200
+++ b/Modules/symtablemodule.c	Fri Jan 20 18:02:54 2017 +0200
@@ -4,20 +4,33 @@
 #include "Python-ast.h"
 #include "symtable.h"
 
+#include "clinic/symtablemodule.c.h"
+/*[clinic input]
+module _symtable
+[clinic start generated code]*/
+/*[clinic end generated code: output=da39a3ee5e6b4b0d input=f4685845a7100605]*/
+
+
+/*[clinic input]
+_symtable.symtable
+
+    str:       str
+    filename:  object(converter='PyUnicode_FSDecoder')
+    startstr:  str
+    /
+
+Return symbol and scope dictionaries used internally by compiler.
+[clinic start generated code]*/
+
 static PyObject *
-symtable_symtable(PyObject *self, PyObject *args)
+_symtable_symtable_impl(PyObject *module, const char *str,
+                        PyObject *filename, const char *startstr)
+/*[clinic end generated code: output=914b369c9b785956 input=6c615e84d5f408e3]*/
 {
     struct symtable *st;
     PyObject *t;
-
-    char *str;
-    PyObject *filename;
-    char *startstr;
     int start;
 
-    if (!PyArg_ParseTuple(args, "sO&s:symtable",
-                          &str, PyUnicode_FSDecoder, &filename, &startstr))
-        return NULL;
     if (strcmp(startstr, "exec") == 0)
         start = Py_file_input;
     else if (strcmp(startstr, "eval") == 0)
@@ -42,9 +55,7 @@ symtable_symtable(PyObject *self, PyObje
 }
 
 static PyMethodDef symtable_methods[] = {
-    {"symtable",        symtable_symtable,      METH_VARARGS,
-     PyDoc_STR("Return symbol and scope dictionaries"
-               " used internally by compiler.")},
+    _SYMTABLE_SYMTABLE_METHODDEF
     {NULL,              NULL}           /* sentinel */
 };
 

From report at bugs.python.org  Fri Jan 20 11:46:30 2017
From: report at bugs.python.org (Piotr Dobrogost)
Date: Fri, 20 Jan 2017 16:46:30 +0000
Subject: [issue16285] Update urllib quoting to RFC 3986
In-Reply-To: <1350644168.93.0.364254038155.issue16285@psf.upfronthosting.co.za>
Message-ID: <1484930790.03.0.869932713217.issue16285@psf.upfronthosting.co.za>


Changes by Piotr Dobrogost <pd at bugs.python.dobrogost.net>:


----------
nosy: +piotr.dobrogost

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue16285>
_______________________________________

From report at bugs.python.org  Fri Jan 20 12:07:37 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Fri, 20 Jan 2017 17:07:37 +0000
Subject: [issue20186] Derby #18: Convert 31 sites to Argument Clinic across 23
 files
In-Reply-To: <1389140586.13.0.921060348978.issue20186@psf.upfronthosting.co.za>
Message-ID: <1484932057.81.0.23608103108.issue20186@psf.upfronthosting.co.za>


Raymond Hettinger added the comment:

The application of AC to enumerate() lost information about the start argument and the signature of the call.   We're going backwards.

---- New help -----------------------------------------------------
class enumerate(object)
 |  Return an enumerate object.
 |
 |    iterable
 |      an object supporting iteration
 |
 |  The enumerate object yields pairs containing a count (from start, which
 |  defaults to zero) and a value yielded by the iterable argument.
 |
 |  enumerate is useful for obtaining an indexed list:
 |      (0, seq[0]), (1, seq[1]), (2, seq[2]), ...

---- Old help -----------------------------------------------------

class enumerate(object)
 |  enumerate(iterable[, start]) -> iterator for index, value of iterable
 |  
 |  Return an enumerate object.  iterable must be another object that supports
 |  iteration.  The enumerate object yields pairs containing a count (from
 |  start, which defaults to zero) and a value yielded by the iterable argument.
 |  enumerate is useful for obtaining an indexed list:
 |      (0, seq[0]), (1, seq[1]), (2, seq[2]), ...


Also, reversed() lost the indication of its signature:
      reversed(sequence) -> reverse iterator over values of the sequence

And the help doesn't have the usual:

    iterable
    |      an object supporting iteration

----------
nosy: +rhettinger

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue20186>
_______________________________________

From report at bugs.python.org  Fri Jan 20 12:17:41 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Fri, 20 Jan 2017 17:17:41 +0000
Subject: [issue20186] Derby #18: Convert 31 sites to Argument Clinic across 23
 files
In-Reply-To: <1389140586.13.0.921060348978.issue20186@psf.upfronthosting.co.za>
Message-ID: <1484932661.37.0.172111478866.issue20186@psf.upfronthosting.co.za>


Raymond Hettinger added the comment:

When reviewing AC patches, we should always compare the help() before and after.  

Also, if the code already had fast parsing like:
   if (!PyArg_UnpackTuple(args, "reversed", 1, 1, &seq) )
there needs to be a before and after timing to make sure there wasn't a performance regression.

When PyArg_NoKeywords was present, we need to verify that the AC version also precludes keyword arguments (to prevent the creation of unhelpful keyword args and to keep compatibility with other versions of Python).

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue20186>
_______________________________________

From report at bugs.python.org  Fri Jan 20 12:37:55 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Fri, 20 Jan 2017 17:37:55 +0000
Subject: [issue28735] Mock is equal to ANY but MagicMock is not
In-Reply-To: <1479485234.58.0.222215885157.issue28735@psf.upfronthosting.co.za>
Message-ID: <1484933875.6.0.0519432865528.issue28735@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

Proposed patch fixes comparing MagicMock with ANY.

----------
keywords: +patch
stage:  -> patch review
Added file: http://bugs.python.org/file46361/magicmock_eq.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28735>
_______________________________________

From report at bugs.python.org  Fri Jan 20 13:04:07 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Fri, 20 Jan 2017 18:04:07 +0000
Subject: [issue20186] Derby #18: Convert 31 sites to Argument Clinic across 23
 files
In-Reply-To: <1389140586.13.0.921060348978.issue20186@psf.upfronthosting.co.za>
Message-ID: <1484935447.63.0.359499619291.issue20186@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

Microbenchmarks:

$ ./python -m perf timeit --duplicate 100 "enumerate('abc')"
Unpatched:  Median +- std dev: 1.76 us +- 0.10 us
Patched:    Median +- std dev: 1.61 us +- 0.07 us

$ ./python -m perf timeit --duplicate 100 "enumerate('abc', 1)"
Unpatched:  Median +- std dev: 2.14 us +- 0.09 us
Patched:    Median +- std dev: 1.76 us +- 0.07 us

$ ./python -m perf timeit --duplicate 100 "reversed('abc')"
Unpatched:  Median +- std dev: 1.20 us +- 0.06 us
Patched:    Median +- std dev: 1.20 us +- 0.07 us

enumerate() is 9-21% faster (due to avoiding of tuple creating), reversed() is not changed (Argument Clinic generates the same parsing code for it).

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue20186>
_______________________________________

From report at bugs.python.org  Fri Jan 20 13:18:02 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Fri, 20 Jan 2017 18:18:02 +0000
Subject: [issue20186] Derby #18: Convert 31 sites to Argument Clinic across 23
 files
In-Reply-To: <1389140586.13.0.921060348978.issue20186@psf.upfronthosting.co.za>
Message-ID: <1484936282.59.0.957311234593.issue20186@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

Here is a patch that restores ol docstrings for enumerate() and reversed(). But it may be better to change pydoc so that it would output a text signature of class constructor if available.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue20186>
_______________________________________

From report at bugs.python.org  Fri Jan 20 13:18:19 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Fri, 20 Jan 2017 18:18:19 +0000
Subject: [issue20186] Derby #18: Convert 31 sites to Argument Clinic across 23
 files
In-Reply-To: <1389140586.13.0.921060348978.issue20186@psf.upfronthosting.co.za>
Message-ID: <1484936299.45.0.206508700724.issue20186@psf.upfronthosting.co.za>


Changes by Serhiy Storchaka <storchaka+cpython at gmail.com>:


Added file: http://bugs.python.org/file46362/enumobject-docstrings.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue20186>
_______________________________________

From report at bugs.python.org  Fri Jan 20 15:48:08 2017
From: report at bugs.python.org (Zach Riggle)
Date: Fri, 20 Jan 2017 20:48:08 +0000
Subject: [issue29335] Python 2.7 subprocess module does not check WIFSTOPPED
 on SIGCHLD
Message-ID: <1484945288.47.0.935636853008.issue29335@psf.upfronthosting.co.za>


New submission from Zach Riggle:

The attached script hits some "This should never happen" code in the subprocess module.

These lines here:
https://github.com/python/cpython/blob/2.7/Lib/subprocess.py#L1036-L1038

The root cause is a lack of checking WIFSTOPPED and WSTOPSIG in the handler.

When a process elects into being ptraced via PTRACE_TRACEME, it is stopped on the SIGSEGV instead of terminating, allowing the user to attach a debugger before the kernel destroys the process.

This bug makes it impossible to wait on any process which crashes, which is set up to wait for a debugger.

----------
components: Library (Lib)
files: bug.py
messages: 285921
nosy: Zach Riggle
priority: normal
severity: normal
status: open
title: Python 2.7 subprocess module does not check WIFSTOPPED on SIGCHLD
versions: Python 2.7
Added file: http://bugs.python.org/file46363/bug.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29335>
_______________________________________

From report at bugs.python.org  Fri Jan 20 15:58:35 2017
From: report at bugs.python.org (Greg Stark)
Date: Fri, 20 Jan 2017 20:58:35 +0000
Subject: [issue29334] ssl.SSLObject method getpeercert() is buggy,
 do_handshake() is strange
In-Reply-To: <1484914427.68.0.380715465195.issue29334@psf.upfronthosting.co.za>
Message-ID: <1484945915.9.0.076541812831.issue29334@psf.upfronthosting.co.za>


Greg Stark added the comment:

adding script the illustrates the bug.

----------
Added file: http://bugs.python.org/file46364/sslbugs.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29334>
_______________________________________

From report at bugs.python.org  Fri Jan 20 16:02:48 2017
From: report at bugs.python.org (Ned Deily)
Date: Fri, 20 Jan 2017 21:02:48 +0000
Subject: [issue29335] Python 2.7 subprocess module does not check WIFSTOPPED
 on SIGCHLD
In-Reply-To: <1484945288.47.0.935636853008.issue29335@psf.upfronthosting.co.za>
Message-ID: <1484946168.93.0.479645664575.issue29335@psf.upfronthosting.co.za>


Changes by Ned Deily <nad at python.org>:


----------
nosy: +gregory.p.smith

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29335>
_______________________________________

From report at bugs.python.org  Fri Jan 20 16:10:31 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Fri, 20 Jan 2017 21:10:31 +0000
Subject: [issue26729] Incorrect __text_signature__ for sorted
In-Reply-To: <1460310880.55.0.462687598027.issue26729@psf.upfronthosting.co.za>
Message-ID: <1484946631.81.0.834407554601.issue26729@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

See also issue29327 and issue29331.

sorted_3.patch LGTM.

----------
stage: patch review -> commit review
versions: +Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue26729>
_______________________________________

From report at bugs.python.org  Fri Jan 20 16:32:09 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Fri, 20 Jan 2017 21:32:09 +0000
Subject: [issue8706] accept keyword arguments on most base type methods and
 builtins
In-Reply-To: <1273777855.43.0.0259016277933.issue8706@psf.upfronthosting.co.za>
Message-ID: <1484947929.48.0.392316436528.issue8706@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

Most overhead of supporting keyword arguments when pass positional arguments was removed in issue29029. But still there is an overhead of passing argument by keywords. It can be decreased when convert functions to Argument Clinic or use new private argument parsing API. This is not very easy issue.

----------
versions: +Python 3.7 -Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue8706>
_______________________________________

From report at bugs.python.org  Fri Jan 20 18:13:59 2017
From: report at bugs.python.org (Berker Peksag)
Date: Fri, 20 Jan 2017 23:13:59 +0000
Subject: [issue28735] Mock is equal to ANY but MagicMock is not
In-Reply-To: <1479485234.58.0.222215885157.issue28735@psf.upfronthosting.co.za>
Message-ID: <1484954039.06.0.173960907559.issue28735@psf.upfronthosting.co.za>


Berker Peksag added the comment:

Serhiy's patch looks pretty good to me, thanks! Also, thanks for the report, Rafael. I definitely forgot to fix this case after fixing issue 25195.

----------
stage: patch review -> commit review

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28735>
_______________________________________

From report at bugs.python.org  Fri Jan 20 18:15:39 2017
From: report at bugs.python.org (Elizabeth Myers)
Date: Fri, 20 Jan 2017 23:15:39 +0000
Subject: [issue29328] struct module should support variable-length strings
In-Reply-To: <1484850272.25.0.472309940141.issue29328@psf.upfronthosting.co.za>
Message-ID: <1484954139.55.0.102940740323.issue29328@psf.upfronthosting.co.za>


Elizabeth Myers added the comment:

Hi,

After discussing this on the python-ideas ML a bit more, this is actually a feature people want a great deal. It can't cover every use case, but to expand it further than this proposal and make it do so is way beyond the scope of this proposal.

It may not be completely useful for every protocol, but it is sufficiently useful for many people who have simpler use cases. There is no reason to prevent the addition of this feature other than what boils down to "well code will have to be written..."

I don't buy the argument that it's "outside the scope of the module" more than I think it's more "I don't like the idea of struct being used for non-fixed data." C structures support zero-terminated char arrays, and there is already a Pascal string type.

I didn't realise there'd be this much opposition to just adding two format specifiers... :/

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29328>
_______________________________________

From report at bugs.python.org  Fri Jan 20 18:24:02 2017
From: report at bugs.python.org (Elizabeth Myers)
Date: Fri, 20 Jan 2017 23:24:02 +0000
Subject: [issue29328] struct module should support variable-length strings
In-Reply-To: <1484850272.25.0.472309940141.issue29328@psf.upfronthosting.co.za>
Message-ID: <1484954642.39.0.607289378317.issue29328@psf.upfronthosting.co.za>


Elizabeth Myers added the comment:

Also, to add to the discussion:

* Rejecting this because "it doesn't cover every use case" is a red herring at best. If this can't cover your use case, odds are the struct module can *never* cover it. That is no reason to reject it alone; you would need something more heavyweight than the struct module anyway.

* If the module can't cover your use case with this feature, it can't cover it right now, so why obstruct it? It covers my use cases for it just fine.

* Not everyone needs something more heavyweight, or wants to import some bigger module just because they need variable-length strings.

* If the real goal is to discourage use of the struct module, too bad. People are actually using it in production and it serves its (rather small) purpose very well. Other people would like to use the module for their use cases, but presently cannot, and this proposal would help cover their particular cases.

* The fact that the netstruct module exists with this feature is proof enough there's demand; not to mention the discussion on the python-ideas ML shows that many people already would find this very useful. It's not like I'm proposing adding braces or some horrible huge proposal, I'm adding two format specifiers. *Two.*

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29328>
_______________________________________

From report at bugs.python.org  Fri Jan 20 19:44:57 2017
From: report at bugs.python.org (Berker Peksag)
Date: Sat, 21 Jan 2017 00:44:57 +0000
Subject: [issue29332] Uniform SelectSelector._select behavior
In-Reply-To: <1484900945.96.0.01738502245.issue29332@psf.upfronthosting.co.za>
Message-ID: <1484959497.66.0.110913230944.issue29332@psf.upfronthosting.co.za>


Berker Peksag added the comment:

Thanks for the report! This is a duplicate of issue 27634.

Since we got multiple reports [1][2] about the same problem on selectors34 (backport of selector module) and use of Gevent is very popular, I think we should reconsider the idea of wrapping select.select with staticmethod().

Also, if I remember correctly, this was fixed on Gevent's side before (although I know that you didn't mention Gevent in your report)

Victor and Charles-Fran?ois, what do you think?

[1] https://github.com/berkerpeksag/selectors34/pull/4
[2] https://github.com/berkerpeksag/selectors34/issues/5

----------
nosy: +berker.peksag, haypo

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29332>
_______________________________________

From report at bugs.python.org  Fri Jan 20 19:45:01 2017
From: report at bugs.python.org (Gustavo Goretkin)
Date: Sat, 21 Jan 2017 00:45:01 +0000
Subject: [issue28595] shlex.shlex should not augment wordchars
In-Reply-To: <1478166136.94.0.995190174918.issue28595@psf.upfronthosting.co.za>
Message-ID: <1484959501.15.0.500675331669.issue28595@psf.upfronthosting.co.za>


Gustavo Goretkin added the comment:

>Instead of trying to enumerate all possible wordchars, I think a more robust solution is to use whitespace_split to include *all* characters not otherwise considered special.

I agree with that approach.

Also note that dash/hyphen gets incorrectly tokenized.

>>> import shlex
>>> list(shlex.shlex("mkdir -p somepath"))
['mkdir', '-', 'p', 'somepath']

White listing all valid word characters is not good, because the surrogateescape mechanism can include all sorts of "characters".

In bash:

$ echo mkdir $(echo -ne "Bad\xffButLegalPath")
mkdir Bad?ButLegalPath

the path is one token.

However currently in shlex, it gets broken into multiple tokens:

>>> list(shlex.shlex(b"mkdir Bad\ffButLegalPath".decode("utf-8", "surrogoateescape")))
['mkdir', 'Bad', '\x0c', 'fButLegalPath']

----------
nosy: +Gustavo Goretkin

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28595>
_______________________________________

From report at bugs.python.org  Fri Jan 20 19:53:38 2017
From: report at bugs.python.org (Josh Rosenberg)
Date: Sat, 21 Jan 2017 00:53:38 +0000
Subject: [issue29331] Simplify argument parsing in sorted() and list.sort()
In-Reply-To: <1484895579.74.0.0168401872287.issue29331@psf.upfronthosting.co.za>
Message-ID: <1484960018.26.0.423567622578.issue29331@psf.upfronthosting.co.za>


Changes by Josh Rosenberg <shadowranger+python at gmail.com>:


----------
nosy: +josh.r

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29331>
_______________________________________

From report at bugs.python.org  Fri Jan 20 20:19:54 2017
From: report at bugs.python.org (Gustavo Goretkin)
Date: Sat, 21 Jan 2017 01:19:54 +0000
Subject: [issue28595] shlex.shlex should not augment wordchars
In-Reply-To: <1478166136.94.0.995190174918.issue28595@psf.upfronthosting.co.za>
Message-ID: <1484961594.22.0.0397128359898.issue28595@psf.upfronthosting.co.za>


Gustavo Goretkin added the comment:

Sorry, I typo'd that last example pretty badly. Should be

>>> list(shlex.shlex(b"mkdir Bad\xffButLegalPath".decode("utf-8", "surrogateescape")))
['mkdir', 'Bad', '\udcff', 'ButLegalPath']

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28595>
_______________________________________

From report at bugs.python.org  Fri Jan 20 20:48:11 2017
From: report at bugs.python.org (Ammar Askar)
Date: Sat, 21 Jan 2017 01:48:11 +0000
Subject: [issue29326] Blank lines in ._pth file are not ignored
In-Reply-To: <1484844872.11.0.819349056222.issue29326@psf.upfronthosting.co.za>
Message-ID: <1484963291.94.0.950511252517.issue29326@psf.upfronthosting.co.za>


Ammar Askar added the comment:

I've attached a patch that makes the site package ignore empty lines in .pth files. Along with a test for this behavior. I'm not really familiar with the site machinery so I hope the test and code are fine.

----------
keywords: +patch
nosy: +ammar2
Added file: http://bugs.python.org/file46365/empty_path.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29326>
_______________________________________

From report at bugs.python.org  Fri Jan 20 21:13:30 2017
From: report at bugs.python.org (Evan Andrews)
Date: Sat, 21 Jan 2017 02:13:30 +0000
Subject: [issue28595] shlex.shlex should not augment wordchars
In-Reply-To: <1478166136.94.0.995190174918.issue28595@psf.upfronthosting.co.za>
Message-ID: <1484964810.47.0.215851768915.issue28595@psf.upfronthosting.co.za>


Evan Andrews added the comment:

Unfortunately shlex.shlex's defaults are probably going to remain that way for a long time in order to avoid breaking backwards compatibility. Presumably shlex.split was added so you didn't have to remember to set posix and whitespace_split yourself.

The particular problem I'm addressing in this issue is that the new punctuation_chars argument doesn't currently work with whitespace_split.

>>> def split(text, ws=False, pc=False):
...     s = shlex.shlex(text, posix=True, punctuation_chars=pc)
...     s.whitespace_split = ws
...     return list(s)
...
>>> split('foo,bar>baz')
['foo', ',', 'bar', '>', 'baz']
>>> split('foo,bar>baz', ws=True)
['foo,bar>baz']
>>> split('foo,bar>baz', pc=True)
['foo', ',', 'bar', '>', 'baz']
>>> split('foo,bar>baz', ws=True, pc=True)
['foo,bar>baz']

With my patch, the last example outputs ['foo,bar', '>', 'baz'].

Before the release of 3.6 I was arguing that punctuation_chars should not attempt to augment wordchars at all, since the idea of wordchars is inherently incorrect as you point out. Now I think it's too late to change, hence my patch treats this as a new feature in 3.7.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28595>
_______________________________________

From report at bugs.python.org  Fri Jan 20 21:29:01 2017
From: report at bugs.python.org (Cameron Simpson)
Date: Sat, 21 Jan 2017 02:29:01 +0000
Subject: [issue29328] struct module should support variable-length strings
In-Reply-To: <1484850272.25.0.472309940141.issue29328@psf.upfronthosting.co.za>
Message-ID: <1484965741.84.0.51230625667.issue29328@psf.upfronthosting.co.za>


Changes by Cameron Simpson <cs at zip.com.au>:


----------
nosy: +cameron

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29328>
_______________________________________

From report at bugs.python.org  Fri Jan 20 22:30:45 2017
From: report at bugs.python.org (INADA Naoki)
Date: Sat, 21 Jan 2017 03:30:45 +0000
Subject: [issue29336] merge tuples in module
Message-ID: <1484969445.33.0.406694538272.issue29336@psf.upfronthosting.co.za>


New submission from INADA Naoki:

Tuples consists of None, True, False, str and bytes can be merged safely.
To avoid memory leak, this patch shares such tuples in compile unit (module, in most case.)

----------
files: merge-tuples.patch
keywords: patch
messages: 285933
nosy: haypo, inada.naoki
priority: normal
severity: normal
status: open
title: merge tuples in module
Added file: http://bugs.python.org/file46366/merge-tuples.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29336>
_______________________________________

From report at bugs.python.org  Fri Jan 20 22:37:58 2017
From: report at bugs.python.org (INADA Naoki)
Date: Sat, 21 Jan 2017 03:37:58 +0000
Subject: [issue29336] merge tuples in module
In-Reply-To: <1484969445.33.0.406694538272.issue29336@psf.upfronthosting.co.za>
Message-ID: <1484969878.08.0.757368572704.issue29336@psf.upfronthosting.co.za>


INADA Naoki added the comment:

I tried this patch with attached script.

```
$ venv/bin/pip install django flask sqlalchemy
$ PYTHONTRACEMALLOC=5 venv/bin/python3 tuplemem.py > tuples.txt
$ sort tuples.txt | uniq -c | sort -nr > tuplecount
```

## default
memory: (32254693, 32292635)
tuples: 64968

head -n10 tuplecount-default
   5479 (None,)
   3069 ('self',)
    727 (<class 'object'>,)
    688 ('__class__',)
    321 ('NotImplementedError',)
    287 ('self', 'other')
    264 (None, None)
    207 (False,)
    193 (None, 0)
    176 (None, False)

## patched
memory: (31224697, 31261892)
tuples: 51298

head -n10 tuplecount-patched
   1437 (None,)
    727 (<class 'object'>,)
    328 ('self',)
    264 (None, None)
    207 (False,)
    193 (None, 0)
    138 ('__class__',)
    114 (True,)
    112 (None, False)
    110 ('target', 'fn')


I'll try again with my company's private codebase in next week.

----------
Added file: http://bugs.python.org/file46367/tuplemem.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29336>
_______________________________________

From report at bugs.python.org  Fri Jan 20 22:45:59 2017
From: report at bugs.python.org (Wen Adam)
Date: Sat, 21 Jan 2017 03:45:59 +0000
Subject: [issue29332] Uniform SelectSelector._select behavior
In-Reply-To: <1484900945.96.0.01738502245.issue29332@psf.upfronthosting.co.za>
Message-ID: <1484970359.18.0.907961122868.issue29332@psf.upfronthosting.co.za>


Wen Adam added the comment:

Thx for your replying, Peksag.

I encountered this problem is really because of Gevent and Selector34. Gevent fixed this issue but only works for python3.4+(https://github.com/gevent/gevent/blob/master/src/gevent/monkey.py#L497) by a tricking way. python 2.x still has this problem

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29332>
_______________________________________

From report at bugs.python.org  Fri Jan 20 23:15:01 2017
From: report at bugs.python.org (Berker Peksag)
Date: Sat, 21 Jan 2017 04:15:01 +0000
Subject: [issue29332] Uniform SelectSelector._select behavior
In-Reply-To: <1484900945.96.0.01738502245.issue29332@psf.upfronthosting.co.za>
Message-ID: <1484972101.53.0.83847849853.issue29332@psf.upfronthosting.co.za>


Berker Peksag added the comment:

Great! Since this is no longer an issue on Python 3.4+, I'm going to close this as a duplicate of issue 27634 and merge https://github.com/berkerpeksag/selectors34/pull/4.

----------
resolution:  -> duplicate
stage:  -> resolved
status: open -> closed
superseder:  -> selectors.SelectSelectors fails if select.select was patched

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29332>
_______________________________________

From report at bugs.python.org  Fri Jan 20 23:33:58 2017
From: report at bugs.python.org (Wen Adam)
Date: Sat, 21 Jan 2017 04:33:58 +0000
Subject: [issue29332] Uniform SelectSelector._select behavior
In-Reply-To: <1484900945.96.0.01738502245.issue29332@psf.upfronthosting.co.za>
Message-ID: <1484973238.06.0.458675769237.issue29332@psf.upfronthosting.co.za>


Wen Adam added the comment:

I know it works on python3.4, but black magic still exists in selectors lib?and 3rd-party module have to prepare two function to make compatibility.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29332>
_______________________________________

From report at bugs.python.org  Sat Jan 21 00:11:26 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sat, 21 Jan 2017 05:11:26 +0000
Subject: [issue29336] merge tuples in module
In-Reply-To: <1484969445.33.0.406694538272.issue29336@psf.upfronthosting.co.za>
Message-ID: <1484975486.67.0.528236776285.issue29336@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

$ ./python -Wa -b -c "lambda: 'a'; lambda: b'a'"
sys:1: BytesWarning: Comparison between bytes and string
sys:1: BytesWarning: Comparison between bytes and string
sys:1: BytesWarning: Comparison between bytes and string
sys:1: BytesWarning: Comparison between bytes and string
sys:1: BytesWarning: Comparison between bytes and string
sys:1: BytesWarning: Comparison between bytes and string
sys:1: BytesWarning: Comparison between bytes and string

Look at _PyCode_ConstantKey(), it may help.

----------
components: +Interpreter Core
nosy: +serhiy.storchaka
type:  -> enhancement
versions: +Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29336>
_______________________________________

From report at bugs.python.org  Sat Jan 21 00:18:28 2017
From: report at bugs.python.org (INADA Naoki)
Date: Sat, 21 Jan 2017 05:18:28 +0000
Subject: [issue29336] merge tuples in module
In-Reply-To: <1484969445.33.0.406694538272.issue29336@psf.upfronthosting.co.za>
Message-ID: <1484975908.51.0.341949406032.issue29336@psf.upfronthosting.co.za>


INADA Naoki added the comment:

Oh, thanks.  I hadn't checked the warning.
Since bytes are not important in this time, I'll remove bytes support.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29336>
_______________________________________

From report at bugs.python.org  Sat Jan 21 00:45:55 2017
From: report at bugs.python.org (INADA Naoki)
Date: Sat, 21 Jan 2017 05:45:55 +0000
Subject: [issue29336] merge tuples in module
In-Reply-To: <1484969445.33.0.406694538272.issue29336@psf.upfronthosting.co.za>
Message-ID: <1484977555.3.0.100531741144.issue29336@psf.upfronthosting.co.za>


Changes by INADA Naoki <songofacandy at gmail.com>:


Added file: http://bugs.python.org/file46368/mergetuples2.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29336>
_______________________________________

From report at bugs.python.org  Sat Jan 21 02:09:46 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sat, 21 Jan 2017 07:09:46 +0000
Subject: [issue29337] BytesWarning at compile time
Message-ID: <1484982586.78.0.422468246503.issue29337@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka:

BytesWarning can be raised during compilation.

$ ./python -Wa -b -c "lambda: 'a'; lambda: b'a'"
sys:1: BytesWarning: Comparison between bytes and string
sys:1: BytesWarning: Comparison between bytes and string
sys:1: BytesWarning: Comparison between bytes and string
sys:1: BytesWarning: Comparison between bytes and string

_PyCode_ConstantKey() should produce keys that don't allow comparing bytes with strings nested in tuples or frozensets. Currently it returns (tuple, ('a',), ((str, 'a'),)) for ('a',) and (tuple, (b'a',), ((bytes, b'a'),)) for (b'a',). Key tuples have the same size and the same first element. Comparing second elements emits a BytesWarning.

----------
components: Interpreter Core
messages: 285940
nosy: haypo, inada.naoki, serhiy.storchaka
priority: normal
severity: normal
status: open
title: BytesWarning at compile time
type: behavior
versions: Python 3.5, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29337>
_______________________________________

From report at bugs.python.org  Sat Jan 21 02:09:48 2017
From: report at bugs.python.org (Mariatta Wijaya)
Date: Sat, 21 Jan 2017 07:09:48 +0000
Subject: [issue29314] asyncio.async deprecation warning is missing stacklevel=2
In-Reply-To: <1484770067.77.0.974182324744.issue29314@psf.upfronthosting.co.za>
Message-ID: <1484982588.63.0.43498488819.issue29314@psf.upfronthosting.co.za>


Mariatta Wijaya added the comment:

Attached patch adds stacklevel=2 to the deprecation warning.

----------
keywords: +patch
nosy: +Mariatta
Added file: http://bugs.python.org/file46369/issue29314.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29314>
_______________________________________

From report at bugs.python.org  Sat Jan 21 02:13:37 2017
From: report at bugs.python.org (INADA Naoki)
Date: Sat, 21 Jan 2017 07:13:37 +0000
Subject: [issue8706] accept keyword arguments on most base type methods and
 builtins
In-Reply-To: <1273777855.43.0.0259016277933.issue8706@psf.upfronthosting.co.za>
Message-ID: <1484982817.97.0.0811145300936.issue8706@psf.upfronthosting.co.za>


INADA Naoki added the comment:

TL;DR

Changing positional argument name doesn't break backward compatibility.
After start accepting keyword argument, it's name is part of API and should be stable.

For example, document says `str.startswith(prefix[, start[, end]])`.
But this patch seems using `sub` instead of `prefix`.
https://docs.python.org/3.5/library/stdtypes.html#str.startswith

Keyword name should be chosen carefully, like choosing method name.
So I'm -1 to adding keyword argument support to many method in one issue.

----------
nosy: +inada.naoki

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue8706>
_______________________________________

From report at bugs.python.org  Sat Jan 21 03:06:43 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sat, 21 Jan 2017 08:06:43 +0000
Subject: [issue29337] BytesWarning at compile time
In-Reply-To: <1484982586.78.0.422468246503.issue29337@psf.upfronthosting.co.za>
Message-ID: <1484986003.49.0.992472257645.issue29337@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

Proposed patch fixes the issue.

----------
keywords: +patch
stage:  -> patch review
Added file: http://bugs.python.org/file46370/constant-key-bytes-warning.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29337>
_______________________________________

From report at bugs.python.org  Sat Jan 21 03:48:06 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sat, 21 Jan 2017 08:48:06 +0000
Subject: [issue29336] merge tuples in module
In-Reply-To: <1484969445.33.0.406694538272.issue29336@psf.upfronthosting.co.za>
Message-ID: <1484988486.81.0.143441109138.issue29336@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

Following patch uses _PyCode_ConstantKey() as a key. This allows supporting integers, floats, bytes, etc.

But I think we can go further and merge constants recursively.

See also issue28813.

----------
nosy: +rhettinger
Added file: http://bugs.python.org/file46371/merge-constants.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29336>
_______________________________________

From report at bugs.python.org  Sat Jan 21 04:20:13 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sat, 21 Jan 2017 09:20:13 +0000
Subject: [issue28489] Fix comment in tokenizer.c
In-Reply-To: <1476984216.11.0.150131580141.issue28489@psf.upfronthosting.co.za>
Message-ID: <1484990413.0.0.45966088007.issue28489@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

There are just 8 legal combinations if ignore case:

>>> import tokenize
>>> sorted({x.lower() for x in tokenize._all_string_prefixes() if x})
['b', 'br', 'f', 'fr', 'r', 'rb', 'rf', 'u']

----------
nosy: +serhiy.storchaka

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28489>
_______________________________________

From report at bugs.python.org  Sat Jan 21 04:42:37 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sat, 21 Jan 2017 09:42:37 +0000
Subject: [issue29338] Output the text signature in the help of a class
Message-ID: <1484991756.99.0.970269270756.issue29338@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka:

Pydoc outputs the text signature for C functions. It defines parameters and default values and is a part of function description.

Help on built-in function format in module builtins:

format(value, format_spec='', /)
    Return value.__format__(format_spec)
    
    format_spec defaults to the empty string

When builtin or extension class is converted to Argument Clinic, the generated docstring and text signature of __new__ or __init__ methods usually is used as class docstring and text signature. But pydoc doesn't output the text signature for classes. The important part of information is lost.

For example, for just converted builtin enumerate class:

Help on class enumerate in module builtins:

class enumerate(object)
 |  Return an enumerate object.
 |  
 |    iterable
 |      an object supporting iteration
 |  
 |  The enumerate object yields pairs containing a count (from start, which
 |  defaults to zero) and a value yielded by the iterable argument.
 |  
 |  enumerate is useful for obtaining an indexed list:
 |      (0, seq[0]), (1, seq[1]), (2, seq[2]), ...
 |  
 |  Methods defined here:
 |  
 |  __getattribute__(self, name, /)
 |      Return getattr(self, name).
 |  
 |  __iter__(self, /)
 |      Implement iter(self).
 |  
 |  __new__(*args, **kwargs) from builtins.type
 |      Create and return a new object.  See help(type) for accurate signature.
 |  
 |  __next__(self, /)
 |      Implement next(self).
 |  
 |  __reduce__(...)
 |      Return state information for pickling.

The iterable and start parameters of the constructor are referred but not defined.

----------
components: Argument Clinic, Library (Lib)
messages: 285946
nosy: larry, ncoghlan, rhettinger, serhiy.storchaka, yselivanov
priority: normal
severity: normal
status: open
title: Output the text signature in the help of a class
type: enhancement
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29338>
_______________________________________

From report at bugs.python.org  Sat Jan 21 06:19:11 2017
From: report at bugs.python.org (Eric V. Smith)
Date: Sat, 21 Jan 2017 11:19:11 +0000
Subject: [issue28489] Fix comment in tokenizer.c
In-Reply-To: <1476984216.11.0.150131580141.issue28489@psf.upfronthosting.co.za>
Message-ID: <1484997551.64.0.757370184971.issue28489@psf.upfronthosting.co.za>


Eric V. Smith added the comment:

Right, that's basically what _all_string_prefixes() does: it starts with the 6 unique prefixes that are case- and order- independent ('b', 'r', 'u', 'f', 'br', 'fr'), and adds the cased and ordered versions.

If you're saying that we should list those 8, and say "with all combinations of case", then I think we'd better off listing the 6 and saying "with all combinations of case and order". That's mainly because if "fbr" gets added, then the list of ordered ones gets larger.

But it's just a comment. I think we should just commit Ryan's last patch.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28489>
_______________________________________

From report at bugs.python.org  Sat Jan 21 06:19:54 2017
From: report at bugs.python.org (Martin Panter)
Date: Sat, 21 Jan 2017 11:19:54 +0000
Subject: [issue12067] Doc: remove errors about mixed-type comparisons.
In-Reply-To: <1305237573.86.0.646542413513.issue12067@psf.upfronthosting.co.za>
Message-ID: <1484997594.76.0.00721475433959.issue12067@psf.upfronthosting.co.za>


Martin Panter added the comment:

Here is a port of the documentation to Python 2. Main differences:

* Default rules for order comparisons are different
* Not all kinds of objects inherit from object()
* str(), unicode() compatibility
* xrange() only seems to have default comparability
* NAN, ?binary sequences? and sets not listed

----------
Added file: http://bugs.python.org/file46372/expressions-py2.7.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue12067>
_______________________________________

From report at bugs.python.org  Sat Jan 21 06:22:28 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sat, 21 Jan 2017 11:22:28 +0000
Subject: [issue29104] Left bracket remains in format string result when '\'
 preceeds it
In-Reply-To: <1483012585.51.0.342717701316.issue29104@psf.upfronthosting.co.za>
Message-ID: <1484997748.96.0.0619328489639.issue29104@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

Ping.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29104>
_______________________________________

From report at bugs.python.org  Sat Jan 21 06:24:46 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sat, 21 Jan 2017 11:24:46 +0000
Subject: [issue28489] Fix comment in tokenizer.c
In-Reply-To: <1476984216.11.0.150131580141.issue28489@psf.upfronthosting.co.za>
Message-ID: <1484997886.67.0.769815264476.issue28489@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

No, I'm just saying that the list of combinations is not so large. Ryan's patch LGTM.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28489>
_______________________________________

From report at bugs.python.org  Sat Jan 21 06:26:23 2017
From: report at bugs.python.org (Xavier de Gaye)
Date: Sat, 21 Jan 2017 11:26:23 +0000
Subject: [issue29267] Cannot override some flags in CFLAGS from the
 command-line
In-Reply-To: <1484333637.07.0.801577004436.issue29267@psf.upfronthosting.co.za>
Message-ID: <1484997983.43.0.156795584664.issue29267@psf.upfronthosting.co.za>


Xavier de Gaye added the comment:

Please try:
make CFLAGS_NODIST=-std=gnu99

----------
nosy: +xdegaye

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29267>
_______________________________________

From report at bugs.python.org  Sat Jan 21 06:26:38 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sat, 21 Jan 2017 11:26:38 +0000
Subject: [issue12067] Doc: remove errors about mixed-type comparisons.
In-Reply-To: <1305237573.86.0.646542413513.issue12067@psf.upfronthosting.co.za>
Message-ID: <1484997998.33.0.590372722094.issue12067@psf.upfronthosting.co.za>


Changes by Serhiy Storchaka <storchaka+cpython at gmail.com>:


----------
nosy: +r.david.murray

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue12067>
_______________________________________

From report at bugs.python.org  Sat Jan 21 06:30:37 2017
From: report at bugs.python.org (Mark Dickinson)
Date: Sat, 21 Jan 2017 11:30:37 +0000
Subject: [issue29328] struct module should support variable-length strings
In-Reply-To: <1484850272.25.0.472309940141.issue29328@psf.upfronthosting.co.za>
Message-ID: <1484998237.9.0.539610418335.issue29328@psf.upfronthosting.co.za>


Mark Dickinson added the comment:

A couple of questions that haven't been brought up yet:

1. Do you have any thoughts on how alignment should behave for '@'-style structs containing variable-length strings? I suspect the easiest solution may be simply to disallow that combination, and only allow variable-length strings for "standard" struct types (those with a format string starting with one of "=", "<", ">", "!"), where alignment isn't an issue.

2. For the Struct object, what should the .size attribute give for a variable-length struct? (Or should accessing the attribute raise an exception?)

3. Any thoughts about how the internal representation of the Struct object would need to change? I guess you'd want to drop the "offset" field of the "formatcode" struct, and compute the offsets on the fly during packing and unpacking (or would you try to keep the offset for the non-variable-length cases?). You'd probably also want to find a way to encode the information about whether the struct is variable-length or not in the PyStructObject struct. A key requirement here is that there should be no substantial performance regression for packing / unpacking with structs that don't use the variable-length feature. It doesn't seem likely to me that getting rid of the precalculated offset would cause such a regression, but it's something that should be checked.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29328>
_______________________________________

From report at bugs.python.org  Sat Jan 21 07:00:25 2017
From: report at bugs.python.org (Xavier de Gaye)
Date: Sat, 21 Jan 2017 12:00:25 +0000
Subject: [issue29273] test___all__ alters utf8 locale setting
In-Reply-To: <1484378754.56.0.202012710496.issue29273@psf.upfronthosting.co.za>
Message-ID: <1485000025.03.0.720757159256.issue29273@psf.upfronthosting.co.za>


Xavier de Gaye added the comment:

The rationale supporting this change is sound and the patch LGTM.

----------
nosy: +xdegaye

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29273>
_______________________________________

From report at bugs.python.org  Sat Jan 21 07:22:17 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sat, 21 Jan 2017 12:22:17 +0000
Subject: [issue29328] struct module should support variable-length strings
In-Reply-To: <1484850272.25.0.472309940141.issue29328@psf.upfronthosting.co.za>
Message-ID: <1485001337.53.0.181329118684.issue29328@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

If you want to add the support variable-size fields, this is incompatible with the struct module design.

If you want to add the support of variable-length strings inside fixed-size fields (as with the 'p' format unit), I think this case is not enough common.

After getting so much negative responses from core developers well-versed in internals of the struct module, I think this issue should be closed.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29328>
_______________________________________

From report at bugs.python.org  Sat Jan 21 07:43:25 2017
From: report at bugs.python.org (Roundup Robot)
Date: Sat, 21 Jan 2017 12:43:25 +0000
Subject: [issue29282] Fused multiply-add: proposal to add math.fma()
In-Reply-To: <1484560897.19.0.0955404368118.issue29282@psf.upfronthosting.co.za>
Message-ID: <20170121124321.111947.47659.7D215EBC@psf.io>


Roundup Robot added the comment:

New changeset b33012ef1417 by Mark Dickinson in branch 'default':
Issue #29282: add fused multiply-add function, math.fma.
https://hg.python.org/cpython/rev/b33012ef1417

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29282>
_______________________________________

From report at bugs.python.org  Sat Jan 21 07:45:13 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sat, 21 Jan 2017 12:45:13 +0000
Subject: [issue29282] Fused multiply-add: proposal to add math.fma()
In-Reply-To: <1484560897.19.0.0955404368118.issue29282@psf.upfronthosting.co.za>
Message-ID: <1485002713.58.0.74008561865.issue29282@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

Cross fingers...

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29282>
_______________________________________

From report at bugs.python.org  Sat Jan 21 08:07:14 2017
From: report at bugs.python.org (Mark Dickinson)
Date: Sat, 21 Jan 2017 13:07:14 +0000
Subject: [issue29282] Fused multiply-add: proposal to add math.fma()
In-Reply-To: <1484560897.19.0.0955404368118.issue29282@psf.upfronthosting.co.za>
Message-ID: <1485004034.92.0.168061579243.issue29282@psf.upfronthosting.co.za>


Mark Dickinson added the comment:

Failures on the Windows buildbot (http://buildbot.python.org/all/builders/AMD64%20Windows8.1%20Non-Debug%203.x/builds/238/steps/test/logs/stdio) shown below.

It looks as though Windows is emulating the FMA operation on this machine (and not doing a particularly good job of it). That means that if we want to support Windows (and we do), we may have to emulate ourselves, preferably using something a bit more efficient than the fractions.Fraction module.

I'll let the buildbots complete, to see what else fails, and then roll back the commit. The patch clearly isn't good enough in its current state.


======================================================================
ERROR: test_fma_overflow (test.test_math.FMATests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\buildarea\3.x.ware-win81-release\build\lib\test\test_math.py", line 1565, in test_fma_overflow
    self.assertEqual(math.fma(a, b, -c),
OverflowError: overflow in fma

======================================================================
FAIL: test_fma_zero_result (test.test_math.FMATests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\buildarea\3.x.ware-win81-release\build\lib\test\test_math.py", line 1524, in test_fma_zero_result
    self.assertIsNegativeZero(math.fma(tiny, -tiny, 0.0))
  File "D:\buildarea\3.x.ware-win81-release\build\lib\test\test_math.py", line 1642, in assertIsNegativeZero
    msg="Expected a negative zero, got {!r}".format(value)
AssertionError: False is not true : Expected a negative zero, got 0.0

======================================================================
FAIL: test_random (test.test_math.FMATests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\buildarea\3.x.ware-win81-release\build\lib\test\test_math.py", line 1623, in test_random
    self.assertEqual(math.fma(a, b, c), expected)
AssertionError: 0.5506672157701096 != 0.5506672157701097

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29282>
_______________________________________

From report at bugs.python.org  Sat Jan 21 08:11:09 2017
From: report at bugs.python.org (Roundup Robot)
Date: Sat, 21 Jan 2017 13:11:09 +0000
Subject: [issue29282] Fused multiply-add: proposal to add math.fma()
In-Reply-To: <1484560897.19.0.0955404368118.issue29282@psf.upfronthosting.co.za>
Message-ID: <20170121131106.5923.53827.8ED5174B@psf.io>


Roundup Robot added the comment:

New changeset b5a5f13500b9 by Mark Dickinson in branch 'default':
Issue #29282: Backed out changeset b33012ef1417
https://hg.python.org/cpython/rev/b5a5f13500b9

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29282>
_______________________________________

From report at bugs.python.org  Sat Jan 21 08:17:43 2017
From: report at bugs.python.org (Mark Dickinson)
Date: Sat, 21 Jan 2017 13:17:43 +0000
Subject: [issue29282] Fused multiply-add: proposal to add math.fma()
In-Reply-To: <1484560897.19.0.0955404368118.issue29282@psf.upfronthosting.co.za>
Message-ID: <1485004663.64.0.259449021727.issue29282@psf.upfronthosting.co.za>


Mark Dickinson added the comment:

Also failures on Gentoo: here b is positive (possibly +inf), and c is finite, so we expect an infinite result. Instead, we're apparently getting a NaN. I don't have a good guess about what's causing this: the rest of the tests are passing, so it's unlikely that we're using a bad FMA emulation. Maybe an optimization bug?

======================================================================
ERROR: test_fma_infinities (test.test_math.FMATests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.installed/build/target/lib/python3.7/test/test_math.py", line 1482, in test_fma_infinities
    self.assertEqual(math.fma(math.inf, b, c), math.inf)
ValueError: invalid operation in fma

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29282>
_______________________________________

From report at bugs.python.org  Sat Jan 21 11:21:10 2017
From: report at bugs.python.org (Steve Dower)
Date: Sat, 21 Jan 2017 16:21:10 +0000
Subject: [issue29326] Blank lines in ._pth file are not ignored
In-Reply-To: <1484844872.11.0.819349056222.issue29326@psf.upfronthosting.co.za>
Message-ID: <1485015670.48.0.518320125736.issue29326@psf.upfronthosting.co.za>


Steve Dower added the comment:

The fix for this issue is needed in PC/getpathp.c as this is a special feature in path generation on Windows.

I'm not aware of any such issue with .pth files - the underscore in ._pth is deliberate.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29326>
_______________________________________

From report at bugs.python.org  Sat Jan 21 11:37:52 2017
From: report at bugs.python.org (Tom Krauss)
Date: Sat, 21 Jan 2017 16:37:52 +0000
Subject: [issue16899] Add support for C99 complex type (_Complex) as
 ctypes.c_complex
In-Reply-To: <1357682763.46.0.590255439385.issue16899@psf.upfronthosting.co.za>
Message-ID: <1485016672.18.0.618915092502.issue16899@psf.upfronthosting.co.za>


Tom Krauss added the comment:

I'm trying to add support for this in cffi, which uses ctypes... apparently this is now supported in libffi (https://github.com/libffi/libffi, v3.2 Nov 2014). 
It would be nice if this issue could be re-opened, or another one created for the same purpose.

----------
nosy: +Tom Krauss

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue16899>
_______________________________________

From report at bugs.python.org  Sat Jan 21 11:40:30 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sat, 21 Jan 2017 16:40:30 +0000
Subject: [issue29338] Output the text signature in the help of a class
In-Reply-To: <1484991756.99.0.970269270756.issue29338@psf.upfronthosting.co.za>
Message-ID: <1485016830.93.0.412460103123.issue29338@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

Proposed patch adds a text signature at the start of class description.

----------
keywords: +patch
stage:  -> patch review
Added file: http://bugs.python.org/file46373/pydoc-class-signature.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29338>
_______________________________________

From report at bugs.python.org  Sat Jan 21 11:56:21 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sat, 21 Jan 2017 16:56:21 +0000
Subject: [issue20185] Derby #17: Convert 49 sites to Argument Clinic across 13
 files
In-Reply-To: <1389140539.55.0.106381625615.issue20185@psf.upfronthosting.co.za>
Message-ID: <1485017781.62.0.462269278435.issue20185@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

Martin, you are the one who looked at these patches for last three years. Do you want to take this issue to you and update Vajrasky's patches?

Now there is good performance argument for converting builtins to Argument Clinic.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue20185>
_______________________________________

From report at bugs.python.org  Sat Jan 21 12:55:02 2017
From: report at bugs.python.org (ppperry)
Date: Sat, 21 Jan 2017 17:55:02 +0000
Subject: [issue29199] test_regrtest fails if PCBuild directory doesn't exist
In-Reply-To: <1483838095.77.0.195289202994.issue29199@psf.upfronthosting.co.za>
Message-ID: <1485021302.63.0.596671796443.issue29199@psf.upfronthosting.co.za>


Changes by ppperry <mapreader at olum.org>:


----------
nosy: +ezio.melotti, michael.foord

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29199>
_______________________________________

From report at bugs.python.org  Sat Jan 21 16:05:32 2017
From: report at bugs.python.org (Roundup Robot)
Date: Sat, 21 Jan 2017 21:05:32 +0000
Subject: [issue29331] Simplify argument parsing in sorted() and list.sort()
In-Reply-To: <1484895579.74.0.0168401872287.issue29331@psf.upfronthosting.co.za>
Message-ID: <20170121210529.56146.56065.85085915@psf.io>


Roundup Robot added the comment:

New changeset 69bd5c497a82 by Serhiy Storchaka in branch 'default':
Issue #29331: Simplified argument parsing in sorted() and list.sort().
https://hg.python.org/cpython/rev/69bd5c497a82

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29331>
_______________________________________

From report at bugs.python.org  Sat Jan 21 16:06:23 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sat, 21 Jan 2017 21:06:23 +0000
Subject: [issue29331] Simplify argument parsing in sorted() and list.sort()
In-Reply-To: <1484895579.74.0.0168401872287.issue29331@psf.upfronthosting.co.za>
Message-ID: <1485032783.61.0.346471882325.issue29331@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

Thanks Victor.

----------
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29331>
_______________________________________

From report at bugs.python.org  Sat Jan 21 16:17:52 2017
From: report at bugs.python.org (Roundup Robot)
Date: Sat, 21 Jan 2017 21:17:52 +0000
Subject: [issue28735] Mock is equal to ANY but MagicMock is not
In-Reply-To: <1479485234.58.0.222215885157.issue28735@psf.upfronthosting.co.za>
Message-ID: <20170121211749.15068.47020.0B1BBB15@psf.io>


Roundup Robot added the comment:

New changeset 442eb26b1ca4 by Serhiy Storchaka in branch '3.5':
Issue #28735: Fixed the comparison of mock.MagickMock with mock.ANY.
https://hg.python.org/cpython/rev/442eb26b1ca4

New changeset 4a38781538f7 by Serhiy Storchaka in branch '3.6':
Issue #28735: Fixed the comparison of mock.MagickMock with mock.ANY.
https://hg.python.org/cpython/rev/4a38781538f7

New changeset 597515fcb343 by Serhiy Storchaka in branch 'default':
Issue #28735: Fixed the comparison of mock.MagickMock with mock.ANY.
https://hg.python.org/cpython/rev/597515fcb343

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28735>
_______________________________________

From report at bugs.python.org  Sat Jan 21 16:21:02 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sat, 21 Jan 2017 21:21:02 +0000
Subject: [issue28735] Mock is equal to ANY but MagicMock is not
In-Reply-To: <1479485234.58.0.222215885157.issue28735@psf.upfronthosting.co.za>
Message-ID: <1485033661.99.0.936476686728.issue28735@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

Thanks Berker.

Maybe _Call.__eq__ also should return NotImplemented in some cases, but I don't have a demonstrating example.

----------
resolution:  -> fixed
stage: commit review -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28735>
_______________________________________

From report at bugs.python.org  Sat Jan 21 17:01:17 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sat, 21 Jan 2017 22:01:17 +0000
Subject: [issue29311] Argument Clinic: convert dict methods
In-Reply-To: <1484758450.31.0.717694851278.issue29311@psf.upfronthosting.co.za>
Message-ID: <1485036077.03.0.00720193179637.issue29311@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

Proposed patch fixes/improves docstrings of dict and OrderedDict methods.

----------
nosy: +eric.snow, rhettinger
stage:  -> resolved
Added file: http://bugs.python.org/file46374/dict-docstrings.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29311>
_______________________________________

From report at bugs.python.org  Sat Jan 21 17:45:55 2017
From: report at bugs.python.org (Steve Dower)
Date: Sat, 21 Jan 2017 22:45:55 +0000
Subject: [issue29241] sys._enablelegacywindowsfsencoding() don't apply to
 os.fsencode and os.fsdecode
In-Reply-To: <1484137681.59.0.0301192140292.issue29241@psf.upfronthosting.co.za>
Message-ID: <1485038755.41.0.716196576786.issue29241@psf.upfronthosting.co.za>


Steve Dower added the comment:

Thanks for checking that.

I don't think it's worth retaining the cache on Windows in the face of the broken behaviour. Any real-world case where a lot of paths are being encoded or decoded is also likely to involve file-system access which will dwarf the encoding time. Further, passing bytes on Windows will result in another decode/encode cycle anyway, so there will be a bigger performance impact in using str (though even then, probably only when the str is already represented using 16-bit characters).

Unless somebody wants to make a case for having a more complex mechanism to reset the cache, I'll make the change to remove it (protected by an 'if sys.platform.startswith('win')' check).

----------
assignee:  -> steve.dower
versions: +Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29241>
_______________________________________

From report at bugs.python.org  Sat Jan 21 17:56:15 2017
From: report at bugs.python.org (12345 67890)
Date: Sat, 21 Jan 2017 22:56:15 +0000
Subject: [issue29339] Interactive: Move to same indentation level as previous
 line
Message-ID: <1485039375.56.0.473197770315.issue29339@psf.upfronthosting.co.za>


New submission from 12345 67890:

Not sure if anyone has ever thought about this, but it would be nice if the interactive interpreter set the same indentation level as the previous line. Of course this isn't urgent, but what do others think?

----------
messages: 285970
nosy: 12345 67890
priority: normal
severity: normal
status: open
title: Interactive: Move to same indentation level as previous line
type: enhancement

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29339>
_______________________________________

From report at bugs.python.org  Sat Jan 21 18:37:20 2017
From: report at bugs.python.org (Nick Sweeting)
Date: Sat, 21 Jan 2017 23:37:20 +0000
Subject: [issue29340] SSL destructor segfaults in python3.6 threads when an
 unverified-cert connection is closed
Message-ID: <1485041840.84.0.00558025824464.issue29340@psf.upfronthosting.co.za>


New submission from Nick Sweeting:

I mistakenly thought this bug was a pyOpenSSL bug, so I reported it there (https://github.com/pyca/pyopenssl/issues/588), but it looks like this is actually a bug in CPython.

See the link for a nicer formatted bug report with more details.

Code to reproduce: https://gist.github.com/pirate/6649314e02384274b29e04364c9d0c16

Crash dump:
https://gist.github.com/pirate/e1485110093d2d9fc49596e6ff481777

Description:
The bug is a segmentation fault when closing several secure websockets from inside concurrent python3.6 threads: 
```pythohn
'python3.6 libssl_bug.py' terminated by signal SIGSEGV (Address boundary error)
```

It does not occur when doing it in a single thread, only when closing multiple threads at once.


```python
threads = []
for _ in range(NUM_THREADS):
    t = SocketThread('wss://echo.websocket.org/', ssl_opt={'cert_reqs': 0})
    t.start()
    threads.append(t)

sleep(4)

for t in threads:
    t.keep_running = False
    t.ws.close()  # libssl segfaults on python3.6 when closing a wss:// connection with cert_reqs=0
    t.join()

```

This is my first python bug report, so apologies if I didn't get the formatting right or if I'm missing some info.

----------
assignee: christian.heimes
components: SSL, macOS
files: libssl_bug.py
messages: 285971
nosy: Nick Sweeting, christian.heimes, ned.deily, ronaldoussoren
priority: normal
severity: normal
status: open
title: SSL destructor segfaults in python3.6 threads when an unverified-cert connection is closed
type: crash
versions: Python 3.6
Added file: http://bugs.python.org/file46375/libssl_bug.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29340>
_______________________________________

From report at bugs.python.org  Sat Jan 21 18:43:08 2017
From: report at bugs.python.org (Christian Heimes)
Date: Sat, 21 Jan 2017 23:43:08 +0000
Subject: [issue28134] socket.socket(fileno=fd) does not work as documented
In-Reply-To: <1473794011.65.0.923929152137.issue28134@psf.upfronthosting.co.za>
Message-ID: <1485042188.11.0.718308820236.issue28134@psf.upfronthosting.co.za>


Christian Heimes added the comment:

Here is another example of how broken and dangerous fileno argument is. getpeername() is neither a valid IPv4 tuple nor a valid IPv6 tuple. It's all messed up:

>>> import socket
>>> s = socket.create_connection(('www.python.org', 443))
>>> s
<socket.socket fd=3, family=AddressFamily.AF_INET6, type=SocketKind.SOCK_STREAM, proto=6, laddr=('2003:58:bc4a:3b00:56ee:75ff:fe47:ca7b', 59730, 0, 0), raddr=('2a04:4e42:1b::223', 443, 0, 0)>
>>> socket.socket(fileno=s.fileno())
<socket.socket fd=3, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('2003:58:bc4a:3b00::%2550471192', 59730, 0, 2550471192), raddr=('2a04:4e42:1b:0:700c:e70b:ff7f:0%2550471192', 443, 0, 2550471192)>

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28134>
_______________________________________

From report at bugs.python.org  Sat Jan 21 19:00:49 2017
From: report at bugs.python.org (Christian Heimes)
Date: Sun, 22 Jan 2017 00:00:49 +0000
Subject: [issue29340] SSL destructor segfaults in python3.6 threads when an
 unverified-cert connection is closed
In-Reply-To: <1485041840.84.0.00558025824464.issue29340@psf.upfronthosting.co.za>
Message-ID: <1485043249.1.0.316271078428.issue29340@psf.upfronthosting.co.za>


Christian Heimes added the comment:

I can reproduce the crash under Linux with OpenSSL 1.0.2j and Python 2.7, 3.5 and 3.6. Python 2.7 doesn't crash every time, 1 out of 5 times maybe.

#0  ssl_buf_freelist_free (list=0x7fffe8003b50) at ssl_lib.c:2094
#1  0x00007fffef3d68d7 in SSL_CTX_free (a=0x7fffe8006cf0) at ssl_lib.c:2182
#2  0x00007fffed23c60b in context_dealloc (self=0x7fffecffa9e8) at /home/heimes/dev/python/3.6/Modules/_ssl.c:2787
#3  0x00000000004c8b74 in subtype_dealloc (self=0x7fffecffa9e8) at Objects/typeobject.c:1222
#4  0x00000000004a4bc7 in dict_dealloc (mp=0x7fffed00a480) at Objects/dictobject.c:2011
#5  0x00000000004c8be2 in subtype_dealloc (self=0x7fffecfffce0) at Objects/typeobject.c:1207

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29340>
_______________________________________

From report at bugs.python.org  Sat Jan 21 19:09:23 2017
From: report at bugs.python.org (Christian Heimes)
Date: Sun, 22 Jan 2017 00:09:23 +0000
Subject: [issue29340] SSL destructor segfaults in python3.6 threads when an
 unverified-cert connection is closed
In-Reply-To: <1485041840.84.0.00558025824464.issue29340@psf.upfronthosting.co.za>
Message-ID: <1485043763.19.0.481074749526.issue29340@psf.upfronthosting.co.za>


Christian Heimes added the comment:

SSL_CTX_free:

    if (a->rbuf_freelist)
        ssl_buf_freelist_free(a->rbuf_freelist);

(gdb) p list
$1 = (SSL3_BUF_FREELIST *) 0x7fffe8003b50
(gdb) p *list
$2 = {chunklen = 33096, len = 1, head = 0x7fffe8031c00}
(gdb) p *list->head
$3 = {next = 0x5000030317000000}
(gdb) p *list->head->next
Cannot access memory at address 0x5000030317000000

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29340>
_______________________________________

From report at bugs.python.org  Sat Jan 21 19:29:07 2017
From: report at bugs.python.org (STINNER Victor)
Date: Sun, 22 Jan 2017 00:29:07 +0000
Subject: [issue29241] sys._enablelegacywindowsfsencoding() don't apply to
 os.fsencode and os.fsdecode
In-Reply-To: <1485038755.41.0.716196576786.issue29241@psf.upfronthosting.co.za>
Message-ID: <CAMpsgwa90NMwbo407U+N2fS8wmfm=1=8-JO-eaTCv=NEoydn2Q@mail.gmail.com>


STINNER Victor added the comment:

Can't we just update the cache when the function changes the encoding?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29241>
_______________________________________

From report at bugs.python.org  Sat Jan 21 19:35:07 2017
From: report at bugs.python.org (Christian Heimes)
Date: Sun, 22 Jan 2017 00:35:07 +0000
Subject: [issue29340] SSL destructor segfaults in python3.6 threads when an
 unverified-cert connection is closed
In-Reply-To: <1485041840.84.0.00558025824464.issue29340@psf.upfronthosting.co.za>
Message-ID: <1485045307.01.0.60729710577.issue29340@psf.upfronthosting.co.za>


Christian Heimes added the comment:

I can't reproduce the crash with OpenSSL 1.1.0 and LibreSSL 2.5.0. You might have found a bug in OpenSSL.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29340>
_______________________________________

From report at bugs.python.org  Sat Jan 21 19:38:57 2017
From: report at bugs.python.org (Nick Sweeting)
Date: Sun, 22 Jan 2017 00:38:57 +0000
Subject: [issue29340] SSL destructor segfaults in python3.6 threads when an
 unverified-cert connection is closed
In-Reply-To: <1485041840.84.0.00558025824464.issue29340@psf.upfronthosting.co.za>
Message-ID: <1485045537.94.0.253268853199.issue29340@psf.upfronthosting.co.za>


Nick Sweeting added the comment:

Actually I suspected it was OpenSSL first, I filed the report on their github issues, then went on a fun little wild goose chase that ended in the CPython issue tracker. :)

https://github.com/openssl/openssl/issues/2260

Thanks for helping debug this so quickly!

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29340>
_______________________________________

From report at bugs.python.org  Sat Jan 21 20:08:39 2017
From: report at bugs.python.org (Martin Panter)
Date: Sun, 22 Jan 2017 01:08:39 +0000
Subject: [issue29182] Remove the warning in urllib docs that it doesn't do
 certificate validate by default.
In-Reply-To: <1483722117.81.0.319313681571.issue29182@psf.upfronthosting.co.za>
Message-ID: <1485047319.95.0.802881375229.issue29182@psf.upfronthosting.co.za>


Martin Panter added the comment:

The warning for urllib2.urlopen() was removed in revision 1882157b298a. However, a couple other warnings were converted to ?Changed in version 2.7.9? in revision fb83916c3ea1, which seems safer to me.

Removing documentation almost seems like a step backwards. The usual approach for new features is to document the new behaviour, and when it changed. This has also been done for the ?context? etc parameters added in a bug fix release. So I suggest to document that the certificate is verified since 2.7.9, but not beforehand.

----------
nosy: +martin.panter

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29182>
_______________________________________

From report at bugs.python.org  Sat Jan 21 20:20:26 2017
From: report at bugs.python.org (Steve Dower)
Date: Sun, 22 Jan 2017 01:20:26 +0000
Subject: [issue29241] sys._enablelegacywindowsfsencoding() don't apply to
 os.fsencode and os.fsdecode
In-Reply-To: <1484137681.59.0.0301192140292.issue29241@psf.upfronthosting.co.za>
Message-ID: <1485048026.26.0.208729064383.issue29241@psf.upfronthosting.co.za>


Steve Dower added the comment:

How?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29241>
_______________________________________

From report at bugs.python.org  Sat Jan 21 20:37:59 2017
From: report at bugs.python.org (Martin Panter)
Date: Sun, 22 Jan 2017 01:37:59 +0000
Subject: [issue20185] Derby #17: Convert 49 sites to Argument Clinic across 13
 files
In-Reply-To: <1389140539.55.0.106381625615.issue20185@psf.upfronthosting.co.za>
Message-ID: <1485049079.86.0.157862274354.issue20185@psf.upfronthosting.co.za>


Martin Panter added the comment:

Will keep this in mind, but my time is rather limited, so I may not get to it (and I wouldn?t want to discourage other people from working on it)

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue20185>
_______________________________________

From report at bugs.python.org  Sat Jan 21 21:35:29 2017
From: report at bugs.python.org (Martin Panter)
Date: Sun, 22 Jan 2017 02:35:29 +0000
Subject: [issue29311] Argument Clinic: convert dict methods
In-Reply-To: <1484758450.31.0.717694851278.issue29311@psf.upfronthosting.co.za>
Message-ID: <1485052529.87.0.103640813595.issue29311@psf.upfronthosting.co.za>


Martin Panter added the comment:

Patch looks good, apart from one little thing (see review)

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29311>
_______________________________________

From report at bugs.python.org  Sat Jan 21 21:41:53 2017
From: report at bugs.python.org (Martin Panter)
Date: Sun, 22 Jan 2017 02:41:53 +0000
Subject: [issue1732367] Document the constants in the socket module
Message-ID: <1485052913.93.0.468417947129.issue1732367@psf.upfronthosting.co.za>


Changes by Martin Panter <vadmium+py at gmail.com>:


----------
dependencies: +Document socket.SOL_SOCKET

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue1732367>
_______________________________________

From report at bugs.python.org  Sat Jan 21 21:45:11 2017
From: report at bugs.python.org (Martin Panter)
Date: Sun, 22 Jan 2017 02:45:11 +0000
Subject: [issue26273] Expose TCP_CONGESTION and TCP_USER_TIMEOUT to the socket
 module
In-Reply-To: <1454489684.49.0.598051421532.issue26273@psf.upfronthosting.co.za>
Message-ID: <1485053111.81.0.46593497702.issue26273@psf.upfronthosting.co.za>


Martin Panter added the comment:

Patch looks good to me.

BTW in Issue 27409 I proposed a patch listing more of these.

----------
nosy: +martin.panter

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue26273>
_______________________________________

From report at bugs.python.org  Sat Jan 21 22:18:11 2017
From: report at bugs.python.org (Senthil Kumaran)
Date: Sun, 22 Jan 2017 03:18:11 +0000
Subject: [issue29182] Remove the warning in urllib docs that it doesn't do
 certificate validate by default.
In-Reply-To: <1483722117.81.0.319313681571.issue29182@psf.upfronthosting.co.za>
Message-ID: <1485055091.21.0.799766286591.issue29182@psf.upfronthosting.co.za>


Senthil Kumaran added the comment:

@Martin, that's a sound advice. I agree to it. I'll change it to a note (instead of warning) which mentions about certificate verification since 2.7.9.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29182>
_______________________________________

From report at bugs.python.org  Sat Jan 21 23:02:55 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Sun, 22 Jan 2017 04:02:55 +0000
Subject: [issue29290] argparse breaks long lines on NO-BREAK SPACE
In-Reply-To: <1484622521.42.0.0580919403747.issue29290@psf.upfronthosting.co.za>
Message-ID: <1485057775.43.0.613491529355.issue29290@psf.upfronthosting.co.za>


Xiang Zhang added the comment:

v2 addresses the comments. I didn't receive the review notification mail so just saw them today. :-(

----------
stage:  -> patch review
Added file: http://bugs.python.org/file46376/argparse-help-non-breaking-spaces-2.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29290>
_______________________________________

From report at bugs.python.org  Sat Jan 21 23:04:44 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Sun, 22 Jan 2017 04:04:44 +0000
Subject: [issue29099] sqlite3 timestamp converter cannot handle timezone
In-Reply-To: <1482966269.67.0.620298847284.issue29099@psf.upfronthosting.co.za>
Message-ID: <1485057884.22.0.987580973593.issue29099@psf.upfronthosting.co.za>


Xiang Zhang added the comment:

Ping for review for timestamptz-3.patch.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29099>
_______________________________________

From report at bugs.python.org  Sat Jan 21 23:45:40 2017
From: report at bugs.python.org (Martin Panter)
Date: Sun, 22 Jan 2017 04:45:40 +0000
Subject: [issue28785] Clarify the behavior of __eq__() returning NotImplemented
In-Reply-To: <1479975264.5.0.247033798803.issue28785@psf.upfronthosting.co.za>
Message-ID: <1485060340.05.0.338967276948.issue28785@psf.upfronthosting.co.za>


Changes by Martin Panter <vadmium+py at gmail.com>:


----------
title: Clarify the behavior of NotImplemented -> Clarify the behavior of __eq__() returning NotImplemented

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28785>
_______________________________________

From report at bugs.python.org  Sat Jan 21 23:54:48 2017
From: report at bugs.python.org (Martin Panter)
Date: Sun, 22 Jan 2017 04:54:48 +0000
Subject: [issue15997] NotImplemented needs to be documented
In-Reply-To: <1348206637.53.0.485410334327.issue15997@psf.upfronthosting.co.za>
Message-ID: <1485060888.39.0.645107463981.issue15997@psf.upfronthosting.co.za>


Changes by Martin Panter <vadmium+py at gmail.com>:


----------
dependencies: +Clarify the behavior of __eq__() returning NotImplemented

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue15997>
_______________________________________

From report at bugs.python.org  Sun Jan 22 00:18:47 2017
From: report at bugs.python.org (Roundup Robot)
Date: Sun, 22 Jan 2017 05:18:47 +0000
Subject: [issue29092] Sync os.stat's doc and doc string
In-Reply-To: <1482914545.99.0.849195642081.issue29092@psf.upfronthosting.co.za>
Message-ID: <20170122051844.7098.22076.35D19EF1@psf.io>


Roundup Robot added the comment:

New changeset 1f30e114cbc8 by Xiang Zhang in branch '3.5':
Issue #29092: Sync os.stat's doc and docstring on path type.
https://hg.python.org/cpython/rev/1f30e114cbc8

New changeset 409ffea5cccf by Xiang Zhang in branch '3.6':
Issue #29092: Sync os.stat's doc and docstring on path type.
https://hg.python.org/cpython/rev/409ffea5cccf

New changeset cee9d322178f by Xiang Zhang in branch 'default':
Issue #29092: Merge 3.6.
https://hg.python.org/cpython/rev/cee9d322178f

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29092>
_______________________________________

From report at bugs.python.org  Sun Jan 22 00:19:21 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Sun, 22 Jan 2017 05:19:21 +0000
Subject: [issue29092] Sync os.stat's doc and doc string
In-Reply-To: <1482914545.99.0.849195642081.issue29092@psf.upfronthosting.co.za>
Message-ID: <1485062361.68.0.704901002719.issue29092@psf.upfronthosting.co.za>


Changes by Xiang Zhang <angwerzx at 126.com>:


----------
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29092>
_______________________________________

From report at bugs.python.org  Sun Jan 22 00:35:53 2017
From: report at bugs.python.org (INADA Naoki)
Date: Sun, 22 Jan 2017 05:35:53 +0000
Subject: [issue28663] Higher virtual memory usage on recent Linux versions
In-Reply-To: <1478853233.74.0.582487882076.issue28663@psf.upfronthosting.co.za>
Message-ID: <1485063353.25.0.468479131469.issue28663@psf.upfronthosting.co.za>


INADA Naoki added the comment:

@ProgVal, could you try following?
I doubt that this difference just shows rlimit behavior, not memory usage.

---
$ cat rlimit_difference_linux_versions.py 
import sys
import queue
import multiprocessing
from urllib.request import Request, urlopen
import resource
import tracemalloc

def fetch():
    url = 'http://python.org/'
    request = Request(url)
    fd = urlopen(request)
    return 'ok'

fetch()
print(tracemalloc.get_traced_memory())

$ /usr/bin/time python3 rlimit_difference_linux_versions.py 
(0, 0)
0.12user 0.01system 0:01.47elapsed 9%CPU (0avgtext+0avgdata 18372maxresident)k
0inputs+0outputs (0major+2639minor)pagefaults 0swaps

$ PYTHONTRACEMALLOC=1 /usr/bin/time python3 rlimit_difference_linux_versions.py 
(6072093, 6140554)
0.22user 0.01system 0:01.52elapsed 15%CPU (0avgtext+0avgdata 22640maxresident)k
0inputs+0outputs (0major+3942minor)pagefaults 0swaps

----------
nosy: +inada.naoki

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28663>
_______________________________________

From report at bugs.python.org  Sun Jan 22 00:47:41 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Sun, 22 Jan 2017 05:47:41 +0000
Subject: [issue29341] Missing accepting path-like object in docstrings of os
 module functions
Message-ID: <1485064061.59.0.597221589705.issue29341@psf.upfronthosting.co.za>


New submission from Xiang Zhang:

PathLike objects are added in 3.6 and they are mentioned in the documentation. But in some os module functions' docstrings, acceptable types of path parameter are mentioned and they are not altered to mention path-like object. For example:

chown(path, uid, gid, *, dir_fd=None, follow_symlinks=True)
    Change the owner and group id of path to the numeric uid and gid.\
    
      path
        Path to be examined; can be string, bytes, or open-file-descriptor int.

----------
messages: 285988
nosy: xiang.zhang
priority: normal
severity: normal
status: open
title: Missing accepting path-like object in docstrings of os module functions
versions: Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29341>
_______________________________________

From report at bugs.python.org  Sun Jan 22 00:48:33 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Sun, 22 Jan 2017 05:48:33 +0000
Subject: [issue29341] Missing accepting path-like object in docstrings of os
 module functions
In-Reply-To: <1485064061.59.0.597221589705.issue29341@psf.upfronthosting.co.za>
Message-ID: <1485064113.61.0.963128401157.issue29341@psf.upfronthosting.co.za>


Changes by Xiang Zhang <angwerzx at 126.com>:


----------
stage:  -> needs patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29341>
_______________________________________

From report at bugs.python.org  Sun Jan 22 01:18:23 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sun, 22 Jan 2017 06:18:23 +0000
Subject: [issue29099] sqlite3 timestamp converter cannot handle timezone
In-Reply-To: <1482966269.67.0.620298847284.issue29099@psf.upfronthosting.co.za>
Message-ID: <1485065903.14.0.877652726296.issue29099@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

Added few minor comments on Rietveld. Technically the patch LGTM. But the behavior change of the timestamp converter for input without timezone offset perhaps needs wider discussion. We have several options:

0. Current behavior. Silently drop the timezone if there are microseconds, and raise an error when there are no microseconds. This is bad for many reasons, but some user code can depend on this.

1. Just drop the timezone, don't raise an error when there are no microseconds. The converter will become returning incorrect result instead of just failing on unexpected input. This will fix the user code that is aware of this, but currently fails when input doesn't have microseconds.

2. Return aware datetime for input with timezone offset (as in original patch). Returning aware or naive datetime depending on input can break a user code that is not prepared for this.

3. Raise an error for input with timezone offset, always return naive datetime objects from this converter. This can break the user code that depends on current behavior and works with the input containing the same timezone offsets.

Any option can break some code. I prefer option 3, but want to hear thoughts of other core developers. Maybe discuss this on Python-Dev?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29099>
_______________________________________

From report at bugs.python.org  Sun Jan 22 01:19:16 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sun, 22 Jan 2017 06:19:16 +0000
Subject: [issue29341] Missing accepting path-like object in docstrings of os
 module functions
In-Reply-To: <1485064061.59.0.597221589705.issue29341@psf.upfronthosting.co.za>
Message-ID: <1485065956.35.0.347353679365.issue29341@psf.upfronthosting.co.za>


Changes by Serhiy Storchaka <storchaka+cpython at gmail.com>:


----------
assignee:  -> docs at python
components: +Documentation
keywords: +easy
nosy: +docs at python

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29341>
_______________________________________

From report at bugs.python.org  Sun Jan 22 01:24:38 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Sun, 22 Jan 2017 06:24:38 +0000
Subject: [issue29099] sqlite3 timestamp converter cannot handle timezone
In-Reply-To: <1482966269.67.0.620298847284.issue29099@psf.upfronthosting.co.za>
Message-ID: <1485066278.73.0.351551028223.issue29099@psf.upfronthosting.co.za>


Xiang Zhang added the comment:

> Maybe discuss this on Python-Dev?

It's fine. Could you compose a mail instead of me? I am not good at that. :-(

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29099>
_______________________________________

From report at bugs.python.org  Sun Jan 22 01:26:47 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sun, 22 Jan 2017 06:26:47 +0000
Subject: [issue29290] argparse breaks long lines on NO-BREAK SPACE
In-Reply-To: <1484622521.42.0.0580919403747.issue29290@psf.upfronthosting.co.za>
Message-ID: <1485066407.57.0.954850280835.issue29290@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

LGTM.

You perhaps need to change your email provider Xiang. It fails too often.

----------
assignee:  -> xiang.zhang
stage: patch review -> commit review

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29290>
_______________________________________

From report at bugs.python.org  Sun Jan 22 01:32:20 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sun, 22 Jan 2017 06:32:20 +0000
Subject: [issue29099] sqlite3 timestamp converter cannot handle timezone
In-Reply-To: <1482966269.67.0.620298847284.issue29099@psf.upfronthosting.co.za>
Message-ID: <1485066740.74.0.426454539552.issue29099@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

I'm even worse at that. :-(

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29099>
_______________________________________

From report at bugs.python.org  Sun Jan 22 01:33:59 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Sun, 22 Jan 2017 06:33:59 +0000
Subject: [issue29099] sqlite3 timestamp converter cannot handle timezone
In-Reply-To: <1482966269.67.0.620298847284.issue29099@psf.upfronthosting.co.za>
Message-ID: <1485066839.77.0.711095778199.issue29099@psf.upfronthosting.co.za>


Xiang Zhang added the comment:

> I'm even worse at that. :-(

LoL. Okay I'll do that. :-)

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29099>
_______________________________________

From report at bugs.python.org  Sun Jan 22 01:43:19 2017
From: report at bugs.python.org (Roundup Robot)
Date: Sun, 22 Jan 2017 06:43:19 +0000
Subject: [issue29290] argparse breaks long lines on NO-BREAK SPACE
In-Reply-To: <1484622521.42.0.0580919403747.issue29290@psf.upfronthosting.co.za>
Message-ID: <20170122064316.7055.79138.0F9892D9@psf.io>


Roundup Robot added the comment:

New changeset 98cde683b9c6 by Xiang Zhang in branch '3.5':
Issue #29290: argparse help messages won't wrap at non-breaking spaces.
https://hg.python.org/cpython/rev/98cde683b9c6

New changeset 1754722ec296 by Xiang Zhang in branch '3.6':
Issue #29290: Merge 3.5.
https://hg.python.org/cpython/rev/1754722ec296

New changeset c47a72627f0c by Xiang Zhang in branch 'default':
Issue #29290: Merge 3.6.
https://hg.python.org/cpython/rev/c47a72627f0c

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29290>
_______________________________________

From report at bugs.python.org  Sun Jan 22 01:45:22 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Sun, 22 Jan 2017 06:45:22 +0000
Subject: [issue29290] argparse breaks long lines on NO-BREAK SPACE
In-Reply-To: <1484622521.42.0.0580919403747.issue29290@psf.upfronthosting.co.za>
Message-ID: <1485067522.09.0.740352210212.issue29290@psf.upfronthosting.co.za>


Xiang Zhang added the comment:

Thanks Serhiy. 

BTW, #16623 is about 2.7 and the cause is wrap doesn't handle unicode non-breaking spaces right. So it's not the same thing as here.

----------
resolution:  -> fixed
stage: commit review -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29290>
_______________________________________

From report at bugs.python.org  Sun Jan 22 01:53:39 2017
From: report at bugs.python.org (Zach Riggle)
Date: Sun, 22 Jan 2017 06:53:39 +0000
Subject: [issue29335] Python 2.7 subprocess module does not check WIFSTOPPED
 on SIGCHLD
In-Reply-To: <1484945288.47.0.935636853008.issue29335@psf.upfronthosting.co.za>
Message-ID: <1485068019.89.0.356122895286.issue29335@psf.upfronthosting.co.za>


Zach Riggle added the comment:

To further clarify the report:

When the attached proof-of-concept is executed, a RuntimeException is raised, which has a comment "Should never happen".

The issue isn't due to SIGCHLD, but rather following a waitpid() call.  The code attempts to suss the exit code / reason for waitpid() returning, but does not check for WIFSTOPPED in its handler.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29335>
_______________________________________

From report at bugs.python.org  Sun Jan 22 03:19:41 2017
From: report at bugs.python.org (ProgVal)
Date: Sun, 22 Jan 2017 08:19:41 +0000
Subject: [issue28663] Higher virtual memory usage on recent Linux versions
In-Reply-To: <1478853233.74.0.582487882076.issue28663@psf.upfronthosting.co.za>
Message-ID: <1485073181.19.0.0424758628593.issue28663@psf.upfronthosting.co.za>


ProgVal added the comment:

first command:
(0, 0)
0.14user 0.01system 0:00.94elapsed 16%CPU (0avgtext+0avgdata 18484maxresident)k
560inputs+0outputs (0major+2702minor)pagefaults 0swaps

second command:
(6663744, 6732519)
0.18user 0.01system 0:00.96elapsed 20%CPU (0avgtext+0avgdata 22572maxresident)k
480inputs+0outputs (2major+4081minor)pagefaults 0swaps

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28663>
_______________________________________

From report at bugs.python.org  Sun Jan 22 03:23:51 2017
From: report at bugs.python.org (Mark Dickinson)
Date: Sun, 22 Jan 2017 08:23:51 +0000
Subject: [issue16899] Add support for C99 complex type (_Complex) as
 ctypes.c_complex
In-Reply-To: <1357682763.46.0.590255439385.issue16899@psf.upfronthosting.co.za>
Message-ID: <1485073431.46.0.812874647222.issue16899@psf.upfronthosting.co.za>


Mark Dickinson added the comment:

Thanks, Tom. Re-opening.

----------
resolution: postponed -> 
status: closed -> open

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue16899>
_______________________________________

From report at bugs.python.org  Sun Jan 22 03:53:35 2017
From: report at bugs.python.org (INADA Naoki)
Date: Sun, 22 Jan 2017 08:53:35 +0000
Subject: [issue28663] Higher virtual memory usage on recent Linux versions
In-Reply-To: <1478853233.74.0.582487882076.issue28663@psf.upfronthosting.co.za>
Message-ID: <1485075215.65.0.693885684669.issue28663@psf.upfronthosting.co.za>


INADA Naoki added the comment:

On Linux 3.16? 4.7?  What is difference between 3.16 and 4.7?

Again, I doubt Python's memory consumption increased by Linux version.
Isn't rlimit more strict for now?

Even if memory usage is really grow, I don't think it's a Python's issue.
Maybe, environment issue or kernel issue.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28663>
_______________________________________

From report at bugs.python.org  Sun Jan 22 04:16:56 2017
From: report at bugs.python.org (ProgVal)
Date: Sun, 22 Jan 2017 09:16:56 +0000
Subject: [issue28663] Higher virtual memory usage on recent Linux versions
In-Reply-To: <1478853233.74.0.582487882076.issue28663@psf.upfronthosting.co.za>
Message-ID: <1485076616.89.0.884498445609.issue28663@psf.upfronthosting.co.za>


ProgVal added the comment:

> On Linux 3.16? 4.7?  What is difference between 3.16 and 4.7?

On 4.8 (which has the same issue as 4.7).
The difference is that on 4.7, the rlimit has to be set incredibly high for the child process not to hit the limit.

> Again, I doubt Python's memory consumption increased by Linux version.
> Isn't rlimit more strict for now?

That's what I think too, because I would have noticed if the interpreter actually started using 20MB for such a simple operation.

> Even if memory usage is really grow, I don't think it's a Python's issue.
> Maybe, environment issue or kernel issue.

I agree. But I don't know how to reproduce this issue without multiprocessing.Queue.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28663>
_______________________________________

From report at bugs.python.org  Sun Jan 22 05:05:22 2017
From: report at bugs.python.org (Xavier de Gaye)
Date: Sun, 22 Jan 2017 10:05:22 +0000
Subject: [issue28180] sys.getfilesystemencoding() should default to utf-8
In-Reply-To: <1474024622.65.0.838685461995.issue28180@psf.upfronthosting.co.za>
Message-ID: <1485079522.72.0.240822981866.issue28180@psf.upfronthosting.co.za>


Xavier de Gaye added the comment:

> On Android, setlocale(CATEGORY, "") does not look for the locale environment variables (LANG, ...) but sets the 'C' locale instead

FWIW the source code of setlocale() on bionic (Android libc) is at https://android.googlesource.com/platform/bionic/+/master/libc/bionic/locale.cpp#144

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28180>
_______________________________________

From report at bugs.python.org  Sun Jan 22 06:20:17 2017
From: report at bugs.python.org (Marian Beermann)
Date: Sun, 22 Jan 2017 11:20:17 +0000
Subject: [issue29342] os.posix_fadvise misreports errors
Message-ID: <1485084017.58.0.131551905677.issue29342@psf.upfronthosting.co.za>


New submission from Marian Beermann:

It has been observed that posix_fadvise will not report the original error if the syscall fails. Eg. https://bugs.alpinelinux.org/issues/6592

>>> os.posix_fadvise(-1, 0, 0, os.POSIX_FADV_DONTNEED)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OSError: [Errno 0] Error

Should report EBADF

>>> os.posix_fadvise(16, 0, 0, os.POSIX_FADV_DONTNEED)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OSError: [Errno 0] Error

Ditto

>>> os.posix_fadvise(0, 0, 0, 12345)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OSError: [Errno 0] Error

Should be EINVAL

...

This might be because unlike most syscall wrappers posix_fadvise does not set errno but only returns it.

----------
components: IO, Library (Lib)
messages: 286002
nosy: enkore
priority: normal
severity: normal
status: open
title: os.posix_fadvise misreports errors
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29342>
_______________________________________

From report at bugs.python.org  Sun Jan 22 06:24:41 2017
From: report at bugs.python.org (Marian Beermann)
Date: Sun, 22 Jan 2017 11:24:41 +0000
Subject: [issue29342] os.posix_fadvise misreports errors
In-Reply-To: <1485084017.58.0.131551905677.issue29342@psf.upfronthosting.co.za>
Message-ID: <1485084281.16.0.0429055212746.issue29342@psf.upfronthosting.co.za>


Marian Beermann added the comment:

Indeed, os.posix_fadvise tries to fetch the error from errno, via posix_error()

static PyObject *
os_posix_fadvise_impl(PyObject *module, int fd, Py_off_t offset,
                      Py_off_t length, int advice)
/*[clinic end generated code: output=412ef4aa70c98642 input=0fbe554edc2f04b5]*/
{
    int result;
    int async_err = 0;

    do {
        Py_BEGIN_ALLOW_THREADS
        result = posix_fadvise(fd, offset, length, advice);
        Py_END_ALLOW_THREADS
    } while (result != 0 && errno == EINTR &&
             !(async_err = PyErr_CheckSignals()));
    if (result != 0)
        return (!async_err) ? posix_error() : NULL;
    Py_RETURN_NONE;
}

------------------------------^ posix_error() using errno not result

posix_fallocate has the same caveat and mishandles errors as well:

static PyObject *
os_posix_fallocate_impl(PyObject *module, int fd, Py_off_t offset,
                        Py_off_t length)
/*[clinic end generated code: output=73f107139564aa9d input=d7a2ef0ab2ca52fb]*/
{
    int result;
    int async_err = 0;

    do {
        Py_BEGIN_ALLOW_THREADS
        result = posix_fallocate(fd, offset, length);
        Py_END_ALLOW_THREADS
    } while (result != 0 && errno == EINTR &&
             !(async_err = PyErr_CheckSignals()));
    if (result != 0)
        return (!async_err) ? posix_error() : NULL;
    Py_RETURN_NONE;
}

------------------------------^ posix_error() using errno not result

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29342>
_______________________________________

From report at bugs.python.org  Sun Jan 22 07:11:57 2017
From: report at bugs.python.org (Christian Heimes)
Date: Sun, 22 Jan 2017 12:11:57 +0000
Subject: [issue29343] sock.close() raises OSError EBADF when socket's fd is
 closed
Message-ID: <1485087117.22.0.587639771074.issue29343@psf.upfronthosting.co.za>


New submission from Christian Heimes:

In Python 3.6 the behavior of socket's close method has changed. In Python 2.7, 3.5 and earlier, socket.close() ignored EBADF. Starting with Python 3.6 socket.close() raises an OSError exception when its internal fd has been closed.

Python 2.7.12 (default, Sep 29 2016, 12:52:02) 
[GCC 6.2.1 20160916 (Red Hat 6.2.1-2)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import os, socket
>>> sock = socket.socket()
>>> os.close(sock.fileno())
>>> sock.close()

Python 3.5.2 (default, Sep 14 2016, 11:28:32) 
[GCC 6.2.1 20160901 (Red Hat 6.2.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import socket
>>> sock = socket.socket()
>>> import os, socket
>>> os.close(sock.fileno())
>>> sock.close()

Python 3.6.0+ (3.6:ea0c488b9bac, Jan 14 2017, 14:08:17) 
[GCC 6.3.1 20161221 (Red Hat 6.3.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import os, socket
>>> sock = socket.socket()
>>> os.close(sock.fileno())
>>> sock.close()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/heimes/dev/python/python36/lib/python3.6/socket.py", line 417, in close
    self._real_close()
  File "/home/heimes/dev/python/python36/lib/python3.6/socket.py", line 411, in _real_close
    _ss.close(self)
OSError: [Errno 9] Bad file descriptor


Abstraction layers such as the socket class should ignore EBADF in close(). A debug message or a warning is ok, but an exception is wrong. Even the man page for close discourages it, http://man7.org/linux/man-pages/man2/close.2.html

       Note, however, that a failure return should be used only for
       diagnostic purposes (i.e., a warning to the application that there
       may still be I/O pending or there may have been failed I/O) or
       remedial purposes (e.g., writing the file once more or creating a
       backup).

----------
components: Library (Lib)
keywords: 3.6regression
messages: 286004
nosy: christian.heimes
priority: normal
severity: normal
status: open
title: sock.close() raises OSError EBADF when socket's fd is closed
type: behavior
versions: Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29343>
_______________________________________

From report at bugs.python.org  Sun Jan 22 07:22:42 2017
From: report at bugs.python.org (INADA Naoki)
Date: Sun, 22 Jan 2017 12:22:42 +0000
Subject: [issue28999] Use Py_RETURN_NONE and like
In-Reply-To: <1481969387.72.0.775552115596.issue28999@psf.upfronthosting.co.za>
Message-ID: <1485087762.24.0.713121862229.issue28999@psf.upfronthosting.co.za>


INADA Naoki added the comment:

While patch looks safe, some developer may dislike such a large patch
without fixing real issue.

Anyway, Coccinelle seems very interesting tool for refactoring.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28999>
_______________________________________

From report at bugs.python.org  Sun Jan 22 07:51:17 2017
From: report at bugs.python.org (John Taylor)
Date: Sun, 22 Jan 2017 12:51:17 +0000
Subject: [issue29284] Include thread_name_prefix in the
 concurrent.futures.ThreadPoolExecutor example 17.4.2.1
In-Reply-To: <1484584964.59.0.239435362381.issue29284@psf.upfronthosting.co.za>
Message-ID: <1485089477.55.0.359256533213.issue29284@psf.upfronthosting.co.za>


John Taylor added the comment:

I have updated the Python 3.6 example for 17.4.2.1. ThreadPoolExecutor Example.  Please see the attachment.  On my system this is the output:

thread name: loader_0
thread name: loader_1
thread name: loader_2
thread name: loader_3
thread name: loader_4
'http://example.com/' page is 1270 bytes
'http://www.foxnews.com/' page is 67351 bytes
'http://www.cnn.com/' page is 137164 bytes
'http://europe.wsj.com/' page is 914169 bytes
'http://www.bbc.co.uk/' page is 229503 bytes

Could you please consider adding this to the official documentation?

----------
Added file: http://bugs.python.org/file46377/ThreadPoolExec_Example.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29284>
_______________________________________

From report at bugs.python.org  Sun Jan 22 07:53:19 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sun, 22 Jan 2017 12:53:19 +0000
Subject: [issue28999] Use Py_RETURN_NONE and like
In-Reply-To: <1481969387.72.0.775552115596.issue28999@psf.upfronthosting.co.za>
Message-ID: <1485089598.99.0.10201128985.issue28999@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

That is why I haven't pushed the patch. Thanks you for your LGTMs Victor and Inada, but this is not enough.

In general I think that it is better to use Py_RETURN_NONE than "Py_INCREF(Py_None); return Py_None;". Some replacements already was done in the past, and some currently reviewed patches include also such changes. This can distract attention from the main purpose of patches. I think that pushing all changes in one big commit will make less harm than add small changes in other patches here and there. Concinelle proved his reliability and it is easy to check these changes.

I'll push this patch if Raymond or other more conservative core developers accept it (or selected parts of it).

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28999>
_______________________________________

From report at bugs.python.org  Sun Jan 22 08:17:19 2017
From: report at bugs.python.org (Akira Li)
Date: Sun, 22 Jan 2017 13:17:19 +0000
Subject: [issue28876] bool of large range raises OverflowError
In-Reply-To: <1480930662.45.0.497603219066.issue28876@psf.upfronthosting.co.za>
Message-ID: <1485091039.47.0.665092236715.issue28876@psf.upfronthosting.co.za>


Akira Li added the comment:

Following the python-dev discussion [1] I've added a variant of the patch that uses c99 designated initializers [2]

[1] https://mail.python.org/pipermail/python-dev/2017-January/147175.html
[2] https://gcc.gnu.org/onlinedocs/gcc/Designated-Inits.html

----------
Added file: http://bugs.python.org/file46378/range_bool-c99-designated-initializers.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28876>
_______________________________________

From report at bugs.python.org  Sun Jan 22 08:23:20 2017
From: report at bugs.python.org (Antoine Pitrou)
Date: Sun, 22 Jan 2017 13:23:20 +0000
Subject: [issue29343] sock.close() raises OSError EBADF when socket's fd is
 closed
In-Reply-To: <1485087117.22.0.587639771074.issue29343@psf.upfronthosting.co.za>
Message-ID: <1485091400.86.0.242896795862.issue29343@psf.upfronthosting.co.za>


Antoine Pitrou added the comment:

Closing a socket whose fd has already been closed before is a bug waiting to happen.  Indeed it feels harmless if you just get a EBADF, but if the given fd gets reused in the meantime for another file or socket, your close() method is going to close a resource which doesn't belong to the socket (and then the fd can be reused for yet another resource without its owner knowing, and confusion/hilarity ensues).

----------
nosy: +neologix, pitrou

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29343>
_______________________________________

From report at bugs.python.org  Sun Jan 22 08:47:41 2017
From: report at bugs.python.org (Martin Panter)
Date: Sun, 22 Jan 2017 13:47:41 +0000
Subject: [issue29343] sock.close() raises OSError EBADF when socket's fd is
 closed
In-Reply-To: <1485087117.22.0.587639771074.issue29343@psf.upfronthosting.co.za>
Message-ID: <1485092861.97.0.0667849204017.issue29343@psf.upfronthosting.co.za>


Martin Panter added the comment:

I added this behaviour in 3.6 on purpose via Issue 26685.

The change also impacted Yury (see the original bug thread), but if I understood correctly, he eventually decided that it highlighted a problem in asyncio or his code (though the resulting asyncio pull request seems to have stalled). Before he came to that decision, I did suggest temporarily using DeprecationWarning instead of an exception: <https://bugs.python.org/issue26685#msg278949>.

IMO passing a freed file descriptor to close() is asking for trouble. The descriptor could be recycled, by another thread, or even internally by a function call in the same thread. Another problem is if you don?t end up calling socket.close(), the garbage collector may arbitrarily close an FD in use in the future.

Your example code was not realistic, but I would say the solution is either call socket.detach() rather than socket.fileno(), or don?t call os.close() and just call socket.close().

I think that Linux man page is talking more about asynchronous errors from a previous write call. Even if we tolerated other errors from close(), I would still like to treat EBADF as a real error.

----------
nosy: +martin.panter

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29343>
_______________________________________

From report at bugs.python.org  Sun Jan 22 09:09:45 2017
From: report at bugs.python.org (Jeremy Bustamante)
Date: Sun, 22 Jan 2017 14:09:45 +0000
Subject: [issue29344] sock_recv not detected a coroutine
Message-ID: <1485094185.41.0.725056014669.issue29344@psf.upfronthosting.co.za>


New submission from Jeremy Bustamante:

Documemtation says sock_recv is a coroutine
https://docs.python.org/3.6/library/asyncio-eventloop.html#low-level-socket-operations

But following code says it isnt:

import asyncio
loop = asyncio.get_event_loop()

asyncio.iscoroutinefunction(loop.sock_recv)
False

asyncio.iscoroutine(loop.sock_recv)
False

----------
components: asyncio
messages: 286011
nosy: Jeremy Bustamante, gvanrossum, yselivanov
priority: normal
severity: normal
status: open
title: sock_recv not detected a coroutine
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29344>
_______________________________________

From report at bugs.python.org  Sun Jan 22 10:11:17 2017
From: report at bugs.python.org (Masayuki Yamamoto)
Date: Sun, 22 Jan 2017 15:11:17 +0000
Subject: [issue25658] PyThread assumes pthread_key_t is an integer,
 which is against POSIX
In-Reply-To: <1447861816.88.0.632584646231.issue25658@psf.upfronthosting.co.za>
Message-ID: <1485097877.97.0.0162193966329.issue25658@psf.upfronthosting.co.za>


Masayuki Yamamoto added the comment:

Above said, I updated minor changes to the version 2 patch.  Several codes have kept the words "thread local" and "TLS" because they have pointed programming method or other meanings, not CPython TLS API itself. (e.g. _decimal module)

----------
Added file: http://bugs.python.org/file46379/pythread-tss-3.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue25658>
_______________________________________

From report at bugs.python.org  Sun Jan 22 11:13:51 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sun, 22 Jan 2017 16:13:51 +0000
Subject: [issue29058] Mark new limited C API
In-Reply-To: <1482538310.89.0.583492092721.issue29058@psf.upfronthosting.co.za>
Message-ID: <1485101631.24.0.611852464896.issue29058@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

Do you suggest to emit a warning when the user just defines Py_LIMITED_API without specifying concrete version?

    #define Py_LIMITED_API

I think this would break too much code. And contradicts to the documentation.

If you mean something different, could you provide a patch?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29058>
_______________________________________

From report at bugs.python.org  Sun Jan 22 11:14:54 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sun, 22 Jan 2017 16:14:54 +0000
Subject: [issue29084] C API of OrderedDict
In-Reply-To: <1482847157.79.0.799208161515.issue29084@psf.upfronthosting.co.za>
Message-ID: <1485101694.63.0.523297662544.issue29084@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

Eric, what are your thoughts?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29084>
_______________________________________

From report at bugs.python.org  Sun Jan 22 12:02:49 2017
From: report at bugs.python.org (Just a Person)
Date: Sun, 22 Jan 2017 17:02:49 +0000
Subject: [issue29345] More lost updates with multiprocessing.Value and .Array
Message-ID: <1485104569.63.0.130882514769.issue29345@psf.upfronthosting.co.za>


New submission from Just a Person:

Lately, I have been having trouble using the multiprocessing library's shared memory on Windows. Often, updating the .value property seems to fail for no reason. As shown in the attached video, changing ```if __name__ == '__main__':``` in the sample code from the documentation to  ```if True:``` causes the program to not work.

This issue does not arise under Linux as far as I can tell (testing the same code).

Any explanation/fix would be helpful.

----------
components: Windows
files: 2017-01-22-1140-25.mp4
messages: 286015
nosy: Just a Person, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: More lost updates with multiprocessing.Value and .Array
type: behavior
versions: Python 3.6
Added file: http://bugs.python.org/file46380/2017-01-22-1140-25.mp4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29345>
_______________________________________

From report at bugs.python.org  Sun Jan 22 12:12:08 2017
From: report at bugs.python.org (Senthil Kumaran)
Date: Sun, 22 Jan 2017 17:12:08 +0000
Subject: [issue27973] urllib.urlretrieve() fails on second ftp transfer
In-Reply-To: <1473174596.52.0.952829730129.issue27973@psf.upfronthosting.co.za>
Message-ID: <1485105128.58.0.422081983841.issue27973@psf.upfronthosting.co.za>


Senthil Kumaran added the comment:

I spent time digging for a proper fix for this issue.

I've come to a conclusion that this commit, https://hg.python.org/cpython/rev/44d02a5d59fb (10 May 2016) in 2.7.12, was a mistake and needs to be reverted.

The reason this change was made was apparently, it fixed a particular problem with retrieving files in 3.x, and the change was backported (issue26960). It was reported and fixed in 3.x code line here http://bugs.python.org/issue16270 (This is an improper change too and it needs to be reverted, we will come to it at the end [3].

Why is this a problem?

1. The change made was essentially taking the logic of draining ftp response from endtransfer method. Historically, in the module, endtransfer() has been used before closing the connection and it is correct to drain response (voidresp() method call).

2. If we don't drain responses, we end up with problems like the current (issue27973) bug report.

3. The problem with issue16270 was the fail transfer failed only when urllopen was used as a context manager (which calls close implicitly on the same host). But ftp scenarios were designed for reusing the same host without calling close from a long time (3a) and context manager scenario is not applicable to 2.7 code (3b).

Here are some references on how the module shaped up:
 
3a. https://hg.python.org/cpython/rev/6e0eddfa404a - it talks about repeated retriving of the same file from host without closing as a feature.

3b. The urllopen context manager was applicable only in 3.x so the original problem of issue16270 was not reproducible in 2.7 and it was improper to backport those changes (issue26960). Even issue16270 it is improper to change the code in endtransfer, because the problem is specific with context-manager usecase of urlopen, regular usage is just fine.

This patch fixes the problem for 2.7. I have included tests in test_urllibnet to cover the scenarios that were reported. Please review this.

For 3.x code, I will reopen issue16270, I will port this patch with test cases and an additional case for context manager scenario.

----------
Added file: http://bugs.python.org/file46381/issue27973.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue27973>
_______________________________________

From report at bugs.python.org  Sun Jan 22 12:23:27 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sun, 22 Jan 2017 17:23:27 +0000
Subject: [issue29336] merge tuples in module
In-Reply-To: <1484969445.33.0.406694538272.issue29336@psf.upfronthosting.co.za>
Message-ID: <1485105807.04.0.0771673381008.issue29336@psf.upfronthosting.co.za>


Changes by Serhiy Storchaka <storchaka+cpython at gmail.com>:


Removed file: http://bugs.python.org/file46371/merge-constants.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29336>
_______________________________________

From report at bugs.python.org  Sun Jan 22 12:23:55 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sun, 22 Jan 2017 17:23:55 +0000
Subject: [issue29336] merge tuples in module
In-Reply-To: <1484969445.33.0.406694538272.issue29336@psf.upfronthosting.co.za>
Message-ID: <1485105835.86.0.775600004515.issue29336@psf.upfronthosting.co.za>


Changes by Serhiy Storchaka <storchaka+cpython at gmail.com>:


Added file: http://bugs.python.org/file46382/merge-constants.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29336>
_______________________________________

From report at bugs.python.org  Sun Jan 22 12:28:23 2017
From: report at bugs.python.org (Davin Potts)
Date: Sun, 22 Jan 2017 17:28:23 +0000
Subject: [issue29345] More lost updates with multiprocessing.Value and .Array
In-Reply-To: <1485104569.63.0.130882514769.issue29345@psf.upfronthosting.co.za>
Message-ID: <1485106103.68.0.55838579271.issue29345@psf.upfronthosting.co.za>


Davin Potts added the comment:

I'm having difficulty watching your video attachment.  Would it be possible to instead describe, preferably with example code that others can similarly try to reproduce the behavior, what you're experiencing?

Please keep in mind what the documentation repeatedly advises about the need for capturing your process-creating multiprocessing calls inside a "if __name__ == '__main__'" clause, especially on Windows platforms.

----------
nosy: +davin

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29345>
_______________________________________

From report at bugs.python.org  Sun Jan 22 12:32:12 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sun, 22 Jan 2017 17:32:12 +0000
Subject: [issue29083] Readd PyArg_VaParse to the stable API
In-Reply-To: <1482846112.52.0.144835029316.issue29083@psf.upfronthosting.co.za>
Message-ID: <1485106332.55.0.285190037076.issue29083@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

There are also issues with PyArg_ValidateKeywordArguments(), PyArg_UnpackTuple() and Py_BuildValue(). Their declarations are not available if define PY_SSIZE_T_CLEAN and Py_LIMITED_API < 3.3. But they were available in Python 3.2.

Proposed patch fixes these glitches.

----------
assignee:  -> serhiy.storchaka
keywords: +patch
stage:  -> patch review
Added file: http://bugs.python.org/file46383/PyArg_VaParse-limited-api.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29083>
_______________________________________

From report at bugs.python.org  Sun Jan 22 12:43:32 2017
From: report at bugs.python.org (Eryk Sun)
Date: Sun, 22 Jan 2017 17:43:32 +0000
Subject: [issue29345] More lost updates with multiprocessing.Value and .Array
In-Reply-To: <1485104569.63.0.130882514769.issue29345@psf.upfronthosting.co.za>
Message-ID: <1485107012.94.0.487377429083.issue29345@psf.upfronthosting.co.za>


Eryk Sun added the comment:

Change the last print to test the exit code, as follows:

    if p.exitcode != 0:
        print('Child failed with exit code:', p.exitcode)
    else:
        print(num.value)
        print(arr[:])

Note that when you fail to limit creating a new process to just the "__main__" script, the child process ends up failing. Run it in a command prompt to see the following RuntimeError from the child:

    RuntimeError:
            An attempt has been made to start a new process before the
            current process has finished its bootstrapping phase.

            This probably means that you are not using fork to start your
            child processes and you have forgotten to use the proper idiom
            in the main module:

                if __name__ == '__main__':
                    freeze_support()
                    ...

            The "freeze_support()" line can be omitted if the program
            is not going to be frozen to produce an executable.

Since there's no fork() in the Windows API (the NT kernel actually implements forking for other subsystems such as the old POSIX system and the new Linux system, but the Windows API wasn't designed for it), multiprocessing has to import the main script in a new process, in which case the __name__ is not "__main__". But since you're not checking the __name__, it ends up trying to start another Process instance. You're lucky that this API has been made smart enough to detect this. It used to blow up recursively creating processes.

----------
nosy: +eryksun
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29345>
_______________________________________

From report at bugs.python.org  Sun Jan 22 12:46:13 2017
From: report at bugs.python.org (Senthil Kumaran)
Date: Sun, 22 Jan 2017 17:46:13 +0000
Subject: [issue16270] urllib hangs when closing connection
In-Reply-To: <1350509929.7.0.680667591801.issue16270@psf.upfronthosting.co.za>
Message-ID: <1485107173.7.0.741280308393.issue16270@psf.upfronthosting.co.za>


Senthil Kumaran added the comment:

The original problem here was retrieving files failed (hung) when it was tried via context-manager

1. For e.g after "reverting" the changes, if you use the original file without the context manager, it will succeed.

fobj = urllib.request.urlopen( url )

That indicates the fix in changing something in endtransfer method was improper. I have given more information here: http://bugs.python.org/issue27973#msg286016

The proper fix will be revert the endtransfer changes, but fix it in contextmanager scenario. Add test cases to cover all these scenarios.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue16270>
_______________________________________

From report at bugs.python.org  Sun Jan 22 12:47:34 2017
From: report at bugs.python.org (Eli Collins)
Date: Sun, 22 Jan 2017 17:47:34 +0000
Subject: [issue29346] datetime.utcfromtimestamp() returns strange result for
 very large values
Message-ID: <1485107254.51.0.0185006280427.issue29346@psf.upfronthosting.co.za>


New submission from Eli Collins:

I've found an odd behavior when passing very large values to ``datetime.datetime.utcfromtimestamp()`` and ``.fromtimestamp()`` under python 3.6.

Under python 3.5, ``utcfromtimestamp(1<<40)`` would throw a ValueError that the year was out of range.  Under python 3.6, this now returns a datetime in year 36812 (which seems reasonable given the input).  

The unexpected behavior occurs when increasing the bits passed:   ``utcfromtimestamp(1<<41)`` returns a datetime with a *smaller* year (6118).  This pattern proceeds as the bits are increased, with the years increasing & then wrapping around again, up to the point where it exceeds time_t (at that point, python 3.6 throws the same OSError as 3.5).

It looks to me like 3.6 dropped a bounds check somewhere, and is now truncating high bits off the resulting year?

---

Attached is the "dump_timestamp_output.py" script that I was using to examine boundary behavior of utctimestamp() when I found this bug.

System was running Linux Mint 18.1 x86_64, using the python 3.6.0 build from https://launchpad.net/~fkrull/+archive/ubuntu/deadsnakes (ubuntu's python 3.6.0 build also shows this behavior).

----------
components: Library (Lib)
files: dump_timestamp_output.py
messages: 286021
nosy: Eli Collins
priority: normal
severity: normal
status: open
title: datetime.utcfromtimestamp() returns strange result for very large values
type: behavior
versions: Python 3.6
Added file: http://bugs.python.org/file46384/dump_timestamp_output.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29346>
_______________________________________

From report at bugs.python.org  Sun Jan 22 12:57:21 2017
From: report at bugs.python.org (Gregory P. Smith)
Date: Sun, 22 Jan 2017 17:57:21 +0000
Subject: [issue29335] Python 2.7 subprocess module does not check WIFSTOPPED
 on SIGCHLD
In-Reply-To: <1484945288.47.0.935636853008.issue29335@psf.upfronthosting.co.za>
Message-ID: <1485107841.86.0.452323476938.issue29335@psf.upfronthosting.co.za>


Changes by Gregory P. Smith <greg at krypto.org>:


----------
versions: +Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29335>
_______________________________________

From report at bugs.python.org  Sun Jan 22 12:57:39 2017
From: report at bugs.python.org (Gregory P. Smith)
Date: Sun, 22 Jan 2017 17:57:39 +0000
Subject: [issue29335] Python 2.7 subprocess module does not check WIFSTOPPED
 on SIGCHLD
In-Reply-To: <1484945288.47.0.935636853008.issue29335@psf.upfronthosting.co.za>
Message-ID: <1485107859.96.0.311280505152.issue29335@psf.upfronthosting.co.za>


Changes by Gregory P. Smith <greg at krypto.org>:


----------
assignee:  -> gregory.p.smith

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29335>
_______________________________________

From report at bugs.python.org  Sun Jan 22 12:57:58 2017
From: report at bugs.python.org (Gregory P. Smith)
Date: Sun, 22 Jan 2017 17:57:58 +0000
Subject: [issue29335] subprocess module does not check WIFSTOPPED on SIGCHLD
In-Reply-To: <1484945288.47.0.935636853008.issue29335@psf.upfronthosting.co.za>
Message-ID: <1485107878.88.0.435066710227.issue29335@psf.upfronthosting.co.za>


Changes by Gregory P. Smith <greg at krypto.org>:


----------
title: Python 2.7 subprocess module does not check WIFSTOPPED on SIGCHLD -> subprocess module does not check WIFSTOPPED on SIGCHLD

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29335>
_______________________________________

From report at bugs.python.org  Sun Jan 22 12:58:23 2017
From: report at bugs.python.org (Eryk Sun)
Date: Sun, 22 Jan 2017 17:58:23 +0000
Subject: [issue29345] More lost updates with multiprocessing.Value and .Array
In-Reply-To: <1485104569.63.0.130882514769.issue29345@psf.upfronthosting.co.za>
Message-ID: <1485107903.45.0.742397244562.issue29345@psf.upfronthosting.co.za>


Eryk Sun added the comment:

>    if p.exitcode != 0:
>        print('Child failed with exit code:', p.exitcode)

Interestingly the exit code is always 1 when running under pythonw.exe. I just inspected what's going on by setting sys.stderr to a file and discovered the following issue:

      File "C:\Program Files\Python35\lib\multiprocessing\process.py", line 268, in _bootstrap
        sys.stdout.flush()
    AttributeError: 'NoneType' object has no attribute 'flush'

Calling flush() on stderr and stdout should be gated by a check that they're not None and have a flush method. Or simply ignore the AttributeError.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29345>
_______________________________________

From report at bugs.python.org  Sun Jan 22 13:24:57 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sun, 22 Jan 2017 18:24:57 +0000
Subject: [issue29346] datetime.utcfromtimestamp() returns strange result for
 very large values
In-Reply-To: <1485107254.51.0.0185006280427.issue29346@psf.upfronthosting.co.za>
Message-ID: <1485109497.06.0.754223653793.issue29346@psf.upfronthosting.co.za>


Changes by Serhiy Storchaka <storchaka+cpython at gmail.com>:


----------
nosy: +belopolsky, lemburg

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29346>
_______________________________________

From report at bugs.python.org  Sun Jan 22 13:27:53 2017
From: report at bugs.python.org (Gregory P. Smith)
Date: Sun, 22 Jan 2017 18:27:53 +0000
Subject: [issue29335] subprocess module does not check WIFSTOPPED on SIGCHLD
In-Reply-To: <1484945288.47.0.935636853008.issue29335@psf.upfronthosting.co.za>
Message-ID: <1485109673.35.0.299254598079.issue29335@psf.upfronthosting.co.za>


Gregory P. Smith added the comment:

The attached patch should fix it.

I want to incorporate a bug.py like regression test into test_subprocess.py.

----------
keywords: +patch
stage:  -> test needed
Added file: http://bugs.python.org/file46385/issue29335-gps01.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29335>
_______________________________________

From report at bugs.python.org  Sun Jan 22 13:41:24 2017
From: report at bugs.python.org (Steve Dower)
Date: Sun, 22 Jan 2017 18:41:24 +0000
Subject: [issue29058] Mark new limited C API
In-Reply-To: <1482538310.89.0.583492092721.issue29058@psf.upfronthosting.co.za>
Message-ID: <1485110484.46.0.0921630847165.issue29058@psf.upfronthosting.co.za>


Steve Dower added the comment:

I don't care enough to argue about it with you. Let's just fix the API as soon as we can and apologize to people who hit inconsistencies in earlier versions.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29058>
_______________________________________

From report at bugs.python.org  Sun Jan 22 13:52:42 2017
From: report at bugs.python.org (Marc-Andre Lemburg)
Date: Sun, 22 Jan 2017 18:52:42 +0000
Subject: [issue29346] datetime.utcfromtimestamp() returns strange result for
 very large values
In-Reply-To: <1485107254.51.0.0185006280427.issue29346@psf.upfronthosting.co.za>
Message-ID: <1485111162.46.0.402379452127.issue29346@psf.upfronthosting.co.za>


Marc-Andre Lemburg added the comment:

According to the datetime.h file, the valid range for year is 1-9999, so it's a bit surprising that Python 3.6 allows dates outside this range.

Internally, the year is represented using 2 bytes, so you could represent years outside the range and up to 65535 as well.

Here's what mxDateTime outputs for the given timestamps:

 >>> from mx.DateTime import *
 >>> DateTimeFromTicks(1<<40)
<mx.DateTime.DateTime object for '36812-02-20 01:36:16.00' at 7f17a38a2390>
 >>> DateTimeFromTicks(1<<41)
<mx.DateTime.DateTime object for '71654-04-10 03:12:32.00' at 7f17a1bdf858>

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29346>
_______________________________________

From report at bugs.python.org  Sun Jan 22 14:16:06 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sun, 22 Jan 2017 19:16:06 +0000
Subject: [issue29058] Mark new limited C API
In-Reply-To: <1482538310.89.0.583492092721.issue29058@psf.upfronthosting.co.za>
Message-ID: <1485112566.2.0.255430896503.issue29058@psf.upfronthosting.co.za>


Changes by Serhiy Storchaka <storchaka+cpython at gmail.com>:


----------
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29058>
_______________________________________

From report at bugs.python.org  Sun Jan 22 14:22:44 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sun, 22 Jan 2017 19:22:44 +0000
Subject: [issue28769] Make PyUnicode_AsUTF8 returning "const char *" rather of
 "char *"
In-Reply-To: <1479801324.65.0.962202188144.issue28769@psf.upfronthosting.co.za>
Message-ID: <1485112964.6.0.909287586659.issue28769@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

Stefan, what are your thoughts about this? The patch touches _decimal.c.

----------
nosy: +skrah

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28769>
_______________________________________

From report at bugs.python.org  Sun Jan 22 14:42:48 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sun, 22 Jan 2017 19:42:48 +0000
Subject: [issue28749] Fixed the documentation of the mapping codec APIs
In-Reply-To: <1479637798.25.0.994184606714.issue28749@psf.upfronthosting.co.za>
Message-ID: <1485114168.37.0.947999423927.issue28749@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

How can we move this issue forward? Marc-Andre, have I answered to your objections?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28749>
_______________________________________

From report at bugs.python.org  Sun Jan 22 15:15:19 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sun, 22 Jan 2017 20:15:19 +0000
Subject: [issue17720] pickle.py's load_appends should call append() on objects
 other than lists
In-Reply-To: <1365876289.91.0.454785839611.issue17720@psf.upfronthosting.co.za>
Message-ID: <1485116119.24.0.644367803342.issue17720@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

>From PEP 7:

    listitems    Optional, and new in this PEP.
                 If this is not None, it should be an iterator (not a
                 sequence!) yielding successive list items.  These list
                 items will be pickled, and appended to the object using
                 either obj.append(item) or obj.extend(list_of_items).
                 This is primarily used for list subclasses, but may
                 be used by other classes as long as they have append()
                 and extend() methods with the appropriate signature.
                 (Whether append() or extend() is used depends on which
                 pickle protocol version is used as well as the number
                 of items to append, so both must be supported.)

Both append() or extend() must be supported, therefore old code was correct. C implementation can be optimized by using extend().

----------
status: closed -> open

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue17720>
_______________________________________

From report at bugs.python.org  Sun Jan 22 15:25:23 2017
From: report at bugs.python.org (Stefan Krah)
Date: Sun, 22 Jan 2017 20:25:23 +0000
Subject: [issue28769] Make PyUnicode_AsUTF8 returning "const char *" rather of
 "char *"
In-Reply-To: <1479801324.65.0.962202188144.issue28769@psf.upfronthosting.co.za>
Message-ID: <1485116723.26.0.571143596007.issue28769@psf.upfronthosting.co.za>


Stefan Krah added the comment:

For _decimal I'm happy with just the cast from the first patch -- you have a one line diff and it's easy to see the focus of the issue.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28769>
_______________________________________

From report at bugs.python.org  Sun Jan 22 15:43:09 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sun, 22 Jan 2017 20:43:09 +0000
Subject: [issue28769] Make PyUnicode_AsUTF8 returning "const char *" rather of
 "char *"
In-Reply-To: <1479801324.65.0.962202188144.issue28769@psf.upfronthosting.co.za>
Message-ID: <1485117789.32.0.718187459144.issue28769@psf.upfronthosting.co.za>


Changes by Serhiy Storchaka <storchaka+cpython at gmail.com>:


----------
assignee:  -> serhiy.storchaka

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28769>
_______________________________________

From report at bugs.python.org  Sun Jan 22 16:07:27 2017
From: report at bugs.python.org (Roundup Robot)
Date: Sun, 22 Jan 2017 21:07:27 +0000
Subject: [issue28769] Make PyUnicode_AsUTF8 returning "const char *" rather of
 "char *"
In-Reply-To: <1479801324.65.0.962202188144.issue28769@psf.upfronthosting.co.za>
Message-ID: <20170122210724.111849.79053.550800C3@psf.io>


Roundup Robot added the comment:

New changeset 0d89212941f4 by Serhiy Storchaka in branch 'default':
Issue #28769: The result of PyUnicode_AsUTF8AndSize() and PyUnicode_AsUTF8()
https://hg.python.org/cpython/rev/0d89212941f4

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28769>
_______________________________________

From report at bugs.python.org  Sun Jan 22 16:08:03 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sun, 22 Jan 2017 21:08:03 +0000
Subject: [issue28769] Make PyUnicode_AsUTF8 returning "const char *" rather of
 "char *"
In-Reply-To: <1479801324.65.0.962202188144.issue28769@psf.upfronthosting.co.za>
Message-ID: <1485119283.01.0.125655481254.issue28769@psf.upfronthosting.co.za>


Changes by Serhiy Storchaka <storchaka+cpython at gmail.com>:


----------
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28769>
_______________________________________

From report at bugs.python.org  Sun Jan 22 16:15:49 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sun, 22 Jan 2017 21:15:49 +0000
Subject: [issue29196] Remove old-deprecated plistlib features
In-Reply-To: <1483807940.54.0.471253591392.issue29196@psf.upfronthosting.co.za>
Message-ID: <1485119749.43.0.571629481857.issue29196@psf.upfronthosting.co.za>


Changes by Serhiy Storchaka <storchaka+cpython at gmail.com>:


----------
assignee:  -> ronaldoussoren

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29196>
_______________________________________

From report at bugs.python.org  Sun Jan 22 16:21:49 2017
From: report at bugs.python.org (Gregory P. Smith)
Date: Sun, 22 Jan 2017 21:21:49 +0000
Subject: [issue29335] subprocess module does not check WIFSTOPPED on SIGCHLD
In-Reply-To: <1484945288.47.0.935636853008.issue29335@psf.upfronthosting.co.za>
Message-ID: <1485120109.19.0.980144082475.issue29335@psf.upfronthosting.co.za>


Gregory P. Smith added the comment:

test added.

----------
stage: test needed -> patch review
type:  -> behavior
Added file: http://bugs.python.org/file46386/issue29335-gps02.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29335>
_______________________________________

From report at bugs.python.org  Sun Jan 22 17:20:26 2017
From: report at bugs.python.org (Davin Potts)
Date: Sun, 22 Jan 2017 22:20:26 +0000
Subject: [issue29284] Include thread_name_prefix in the
 concurrent.futures.ThreadPoolExecutor example 17.4.2.1
In-Reply-To: <1484584964.59.0.239435362381.issue29284@psf.upfronthosting.co.za>
Message-ID: <1485123626.82.0.732966052797.issue29284@psf.upfronthosting.co.za>


Changes by Davin Potts <python at discontinuity.net>:


----------
nosy: +davin

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29284>
_______________________________________

From report at bugs.python.org  Sun Jan 22 17:39:18 2017
From: report at bugs.python.org (Christian Heimes)
Date: Sun, 22 Jan 2017 22:39:18 +0000
Subject: [issue29136] Add OP_NO_TLSv1_3
In-Reply-To: <1483391244.76.0.29431388133.issue29136@psf.upfronthosting.co.za>
Message-ID: <1485124758.09.0.0289749815824.issue29136@psf.upfronthosting.co.za>


Christian Heimes added the comment:

memo to me: Update the TLS cipher list to include TLS 1.3 ciphers. TLS 1.3 uses a disjunct set of cipher suites. No member of the current cipher suite set is compatible with TLS 1.3. Handshake with TLS 1.3 enabled servers is going to fail.

As of today OpenSSL 1.1.1-dev provides one of five TLS 1.3 ciphers: TLS13-AES-128-GCM-SHA256. TLS13-AES-256-GCM-SHA384 and TLS13-CHACHA20-POLY1305 are not yet implemented as are CCM block mode.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29136>
_______________________________________

From report at bugs.python.org  Sun Jan 22 18:21:56 2017
From: report at bugs.python.org (desbma)
Date: Sun, 22 Jan 2017 23:21:56 +0000
Subject: [issue29212] Python 3.6 logging thread name regression with
 concurrent.future threads
In-Reply-To: <1483938963.09.0.315595404976.issue29212@psf.upfronthosting.co.za>
Message-ID: <1485127316.17.0.751891143268.issue29212@psf.upfronthosting.co.za>


desbma added the comment:

Ping

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29212>
_______________________________________

From report at bugs.python.org  Sun Jan 22 18:24:23 2017
From: report at bugs.python.org (Ned Deily)
Date: Sun, 22 Jan 2017 23:24:23 +0000
Subject: [issue29212] Python 3.6 logging thread name regression with
 concurrent.future threads
In-Reply-To: <1483938963.09.0.315595404976.issue29212@psf.upfronthosting.co.za>
Message-ID: <1485127463.98.0.0488328333631.issue29212@psf.upfronthosting.co.za>


Changes by Ned Deily <nad at python.org>:


----------
nosy: +bquinlan

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29212>
_______________________________________

From report at bugs.python.org  Sun Jan 22 18:45:01 2017
From: report at bugs.python.org (Alexander Belopolsky)
Date: Sun, 22 Jan 2017 23:45:01 +0000
Subject: [issue29346] datetime.utcfromtimestamp() returns strange result for
 very large values
In-Reply-To: <1485107254.51.0.0185006280427.issue29346@psf.upfronthosting.co.za>
Message-ID: <1485128701.07.0.25128470412.issue29346@psf.upfronthosting.co.za>


Alexander Belopolsky added the comment:

This looks like a duplicate of issue #29100 ("datetime.fromtimestamp() doesn't check min/max year anymore: regression of Python 3.6").

----------
keywords: +3.6regression
nosy: +haypo

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29346>
_______________________________________

From report at bugs.python.org  Sun Jan 22 20:27:04 2017
From: report at bugs.python.org (INADA Naoki)
Date: Mon, 23 Jan 2017 01:27:04 +0000
Subject: [issue29212] Python 3.6 logging thread name regression with
 concurrent.future threads
In-Reply-To: <1483938963.09.0.315595404976.issue29212@psf.upfronthosting.co.za>
Message-ID: <1485134824.55.0.844588043915.issue29212@psf.upfronthosting.co.za>


INADA Naoki added the comment:

I don't think this is a regression, bug, bogos naming.

I agree that "<concurrent.futures.thread.ThreadPoolExecutor object at 0x7f96711ac278>_0" is ugly.
But the name describes what is the thread, than "Thread-1".

How about giving default thread_name_prefix less ugly?
e.g. "ThreadPoolExecutor-worker".

----------
nosy: +inada.naoki

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29212>
_______________________________________

From report at bugs.python.org  Sun Jan 22 20:30:45 2017
From: report at bugs.python.org (Roundup Robot)
Date: Mon, 23 Jan 2017 01:30:45 +0000
Subject: [issue29335] subprocess module does not check WIFSTOPPED on SIGCHLD
In-Reply-To: <1484945288.47.0.935636853008.issue29335@psf.upfronthosting.co.za>
Message-ID: <20170123013039.111947.4926.708B3BA9@psf.io>


Roundup Robot added the comment:

New changeset 269296b2a047 by Gregory P. Smith in branch '3.5':
Issue #29335: Fix subprocess.Popen.wait() when the child process has
https://hg.python.org/cpython/rev/269296b2a047

New changeset ed5255a61648 by Gregory P. Smith in branch '3.6':
Issue #29335: Fix subprocess.Popen.wait() when the child process has
https://hg.python.org/cpython/rev/ed5255a61648

New changeset 4f5e7d018195 by Gregory P. Smith in branch 'default':
Issue #29335: Fix subprocess.Popen.wait() when the child process has
https://hg.python.org/cpython/rev/4f5e7d018195

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29335>
_______________________________________

From report at bugs.python.org  Sun Jan 22 20:36:31 2017
From: report at bugs.python.org (Eric Lafontaine)
Date: Mon, 23 Jan 2017 01:36:31 +0000
Subject: [issue28879] smtplib send_message should add Date header if it is
 missing, per RFC5322
In-Reply-To: <1480952895.22.0.418354723664.issue28879@psf.upfronthosting.co.za>
Message-ID: <1485135391.64.0.437536729395.issue28879@psf.upfronthosting.co.za>


Eric Lafontaine added the comment:

Hi,

I've implemented the heuristic, but it's messy with the issue of this ticket. 

I'm going to do some clean-up and separate the issue from the heuristic and post them separated.

Date issue ;
Test Case
Documentation
Implementation

Heuristic of Resent ;
Test Case
Documentation
Implementation


Regards,
Eric Lafontaine

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28879>
_______________________________________

From report at bugs.python.org  Sun Jan 22 20:44:37 2017
From: report at bugs.python.org (Roundup Robot)
Date: Mon, 23 Jan 2017 01:44:37 +0000
Subject: [issue28556] typing.py upgrades
In-Reply-To: <1477756009.46.0.442279756181.issue28556@psf.upfronthosting.co.za>
Message-ID: <20170123014434.55951.99553.3033D6AB@psf.io>


Roundup Robot added the comment:

New changeset 7bf6b4fe2c3c by Guido van Rossum in branch '3.5':
Issue #28556: various style fixes for typing.py
https://hg.python.org/cpython/rev/7bf6b4fe2c3c

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28556>
_______________________________________

From report at bugs.python.org  Sun Jan 22 20:48:47 2017
From: report at bugs.python.org (Roundup Robot)
Date: Mon, 23 Jan 2017 01:48:47 +0000
Subject: [issue28556] typing.py upgrades
In-Reply-To: <1477756009.46.0.442279756181.issue28556@psf.upfronthosting.co.za>
Message-ID: <20170123014844.6110.72447.E973AD6A@psf.io>


Roundup Robot added the comment:

New changeset f100619e7137 by Guido van Rossum in branch '3.5':
Issue #28556: Allow defining methods in NamedTuple class syntax (#362)
https://hg.python.org/cpython/rev/f100619e7137

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28556>
_______________________________________

From report at bugs.python.org  Sun Jan 22 20:49:05 2017
From: report at bugs.python.org (Roundup Robot)
Date: Mon, 23 Jan 2017 01:49:05 +0000
Subject: [issue28556] typing.py upgrades
In-Reply-To: <1477756009.46.0.442279756181.issue28556@psf.upfronthosting.co.za>
Message-ID: <20170123014903.111706.81607.76CB3ED8@psf.io>


Roundup Robot added the comment:

New changeset 41277771755d by Guido van Rossum in branch '3.6':
Issue #28556: various style fixes for typing.py (3.5->3.6)
https://hg.python.org/cpython/rev/41277771755d

New changeset a7e69479ee22 by Guido van Rossum in branch 'default':
Issue #28556: various style fixes for typing.py (3.6->3.7)
https://hg.python.org/cpython/rev/a7e69479ee22

New changeset ba272f947c40 by Guido van Rossum in branch '3.6':
Issue #28556: Allow defining methods in NamedTuple class syntax (#362) (3.5->3.6)
https://hg.python.org/cpython/rev/ba272f947c40

New changeset 69c5b800df86 by Guido van Rossum in branch 'default':
Issue #28556: Allow defining methods in NamedTuple class syntax (#362) (3.6->3.7)
https://hg.python.org/cpython/rev/69c5b800df86

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28556>
_______________________________________

From report at bugs.python.org  Sun Jan 22 20:55:47 2017
From: report at bugs.python.org (Guido van Rossum)
Date: Mon, 23 Jan 2017 01:55:47 +0000
Subject: [issue28635] Update What's New for 3.6
In-Reply-To: <1478553156.63.0.138924593802.issue28635@psf.upfronthosting.co.za>
Message-ID: <1485136547.14.0.760260725822.issue28635@psf.upfronthosting.co.za>


Guido van Rossum added the comment:

Can this be closed (since 3.6 has been released)? Or is there still more you'd like to do specifically to complete this issue?

----------
nosy: +gvanrossum

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28635>
_______________________________________

From report at bugs.python.org  Sun Jan 22 21:56:04 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Mon, 23 Jan 2017 02:56:04 +0000
Subject: [issue29212] Python 3.6 logging thread name regression with
 concurrent.future threads
In-Reply-To: <1483938963.09.0.315595404976.issue29212@psf.upfronthosting.co.za>
Message-ID: <1485140164.98.0.389490343537.issue29212@psf.upfronthosting.co.za>


Changes by Xiang Zhang <angwerzx at 126.com>:


----------
nosy: +gregory.p.smith

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29212>
_______________________________________

From report at bugs.python.org  Sun Jan 22 22:07:26 2017
From: report at bugs.python.org (Elvis Pranskevichus)
Date: Mon, 23 Jan 2017 03:07:26 +0000
Subject: [issue28635] Update What's New for 3.6
In-Reply-To: <1478553156.63.0.138924593802.issue28635@psf.upfronthosting.co.za>
Message-ID: <1485140846.07.0.421893314625.issue28635@psf.upfronthosting.co.za>


Elvis Pranskevichus added the comment:

I'm pretty sure we're done with What's New for 3.6.0, so, unless it's worth keeping this open for any 3.6.x edits, I'm for closing this.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28635>
_______________________________________

From report at bugs.python.org  Sun Jan 22 22:11:19 2017
From: report at bugs.python.org (INADA Naoki)
Date: Mon, 23 Jan 2017 03:11:19 +0000
Subject: [issue29314] asyncio.async deprecation warning is missing stacklevel=2
In-Reply-To: <1484770067.77.0.974182324744.issue29314@psf.upfronthosting.co.za>
Message-ID: <1485141079.34.0.16501335431.issue29314@psf.upfronthosting.co.za>


INADA Naoki added the comment:

LGTM.

----------
nosy: +inada.naoki

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29314>
_______________________________________

From report at bugs.python.org  Sun Jan 22 22:11:53 2017
From: report at bugs.python.org (INADA Naoki)
Date: Mon, 23 Jan 2017 03:11:53 +0000
Subject: [issue29314] asyncio.async deprecation warning is missing stacklevel=2
In-Reply-To: <1484770067.77.0.974182324744.issue29314@psf.upfronthosting.co.za>
Message-ID: <1485141113.61.0.523694333587.issue29314@psf.upfronthosting.co.za>


Changes by INADA Naoki <songofacandy at gmail.com>:


----------
stage: needs patch -> commit review

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29314>
_______________________________________

From report at bugs.python.org  Sun Jan 22 22:24:01 2017
From: report at bugs.python.org (Saida Dhanavath)
Date: Mon, 23 Jan 2017 03:24:01 +0000
Subject: [issue29347] Python 2.7.8 is crashing while creating weakref for a
 given object.
Message-ID: <1485141841.25.0.0993319189306.issue29347@psf.upfronthosting.co.za>


New submission from Saida Dhanavath:

We are using python 2.7.8 on Ubuntu 14.04 to host our services. In one of the crashes python interpreter got segmentation fault while initializing weakref for a given object. Please find snip of backtraces as given below.

#0  0x00007f62aa86951a in clear_weakref (self=0x7f5a1ed17520) at Objects/weakrefobject.c:65
#1  proxy_dealloc (self=0x7f5a1ed17520) at Objects/weakrefobject.c:540
#2  0x00007f62aa869b8b in PyWeakref_NewProxy (ob=<optimized out>, callback=<optimized out>) at Objects/weakrefobject.c:855
#3  0x00007f62aa901e56 in weakref_proxy (self=<optimized out>, args=<optimized out>) at ./Modules/_weakref.c:73
#4  0x00007f62aa8a929b in call_function (oparg=<optimized out>, pp_stack=0x7f5d36661c90) at Python/ceval.c:4033
.
.
.


Have tried to root cause the issue and found that PyWeakref_NewProxy at Objects/weakrefobject.c creates new isntance of PyWeakReference struct and does not intialize wr_prev and wr_next of new isntance. These pointers can have garbage and point to random memory locations. 

As per comment in the code there could be a race while creating new instance and some other thread could have created weakref by the time current thread returns from new_weakref function. If it finds weakref created, current thread destroys instance created by itself and uses the one created by some other thread.


Python should not crash while destroying the isntance created in the same interpreter function. As per my understanding, both wr_prev and wr_next of PyWeakReference instance should be initialized to NULL to avoid segfault.

----------
components: Interpreter Core
messages: 286044
nosy: dhanavaths
priority: normal
severity: normal
status: open
title: Python 2.7.8 is crashing while creating weakref for a given object.
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29347>
_______________________________________

From report at bugs.python.org  Sun Jan 22 22:32:13 2017
From: report at bugs.python.org (Saida Dhanavath)
Date: Mon, 23 Jan 2017 03:32:13 +0000
Subject: [issue29347] Python 2.7.8 is crashing while creating weakref for a
 given object.
In-Reply-To: <1485141841.25.0.0993319189306.issue29347@psf.upfronthosting.co.za>
Message-ID: <1485142333.19.0.292916665625.issue29347@psf.upfronthosting.co.za>


Changes by Saida Dhanavath <saidadhanavath.l at gmail.com>:


----------
type:  -> crash

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29347>
_______________________________________

From report at bugs.python.org  Sun Jan 22 23:13:12 2017
From: report at bugs.python.org (Ned Deily)
Date: Mon, 23 Jan 2017 04:13:12 +0000
Subject: [issue28635] Update What's New for 3.6
In-Reply-To: <1478553156.63.0.138924593802.issue28635@psf.upfronthosting.co.za>
Message-ID: <1485144792.25.0.794549655733.issue28635@psf.upfronthosting.co.za>


Ned Deily added the comment:

I don't see any reason to keep this open.  Thanks so much, Elvis and Yury, for doing such a great job again!

----------
resolution:  -> fixed
stage:  -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28635>
_______________________________________

From report at bugs.python.org  Sun Jan 22 23:25:02 2017
From: report at bugs.python.org (Ned Deily)
Date: Mon, 23 Jan 2017 04:25:02 +0000
Subject: [issue29335] subprocess module does not check WIFSTOPPED on SIGCHLD
In-Reply-To: <1484945288.47.0.935636853008.issue29335@psf.upfronthosting.co.za>
Message-ID: <1485145502.79.0.663583974131.issue29335@psf.upfronthosting.co.za>


Ned Deily added the comment:

Among other buildbot failures:

http://buildbot.python.org/all/builders/x86%20Tiger%203.6/builds/142/steps/test/logs/stdio

======================================================================
ERROR: test_child_terminated_in_stopped_state (test.test_subprocess.POSIXProcessTestCase)
Test wait() behavior when waitpid returns WIFSTOPPED; issue29335.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/db3l/buildarea/3.6.bolen-tiger/build/Lib/test/test_subprocess.py", line 2514, in test_child_terminated_in_stopped_state
    libc = ctypes.CDLL(libc_name)
  File "/Users/db3l/buildarea/3.6.bolen-tiger/build/Lib/ctypes/__init__.py", line 348, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: dlopen(libc..dylib, 6): image not found

----------------------------------------------------------------------
Ran 260 tests in 102.297s


Also, http://buildbot.python.org/all/builders/x86%20Ubuntu%20Shared%203.x/builds/240/steps/test/logs/stdio

----------
nosy: +ned.deily

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29335>
_______________________________________

From report at bugs.python.org  Mon Jan 23 00:47:50 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Mon, 23 Jan 2017 05:47:50 +0000
Subject: [issue28999] Use Py_RETURN_NONE and like
In-Reply-To: <1481969387.72.0.775552115596.issue28999@psf.upfronthosting.co.za>
Message-ID: <1485150470.07.0.141477672695.issue28999@psf.upfronthosting.co.za>


Raymond Hettinger added the comment:

I believe this is a safe change.  All of the replaced pairs on consecutive lines so there are no intervening operations.

For Modules/xxsubtype.c, I think the code should remain as-is.  It is intended to be the simplest possible example of how to make a subtype and its clarity is reduced by requiring that someone learn the macro.  That said, it would be reasonable to add a comment that this pair could also have been coded as Py_RETURN_NONE (i.e. use it as a teachable moment).

----------
assignee:  -> serhiy.storchaka
nosy: +rhettinger

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28999>
_______________________________________

From report at bugs.python.org  Mon Jan 23 00:50:15 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Mon, 23 Jan 2017 05:50:15 +0000
Subject: [issue28876] bool of large range raises OverflowError
In-Reply-To: <1480930662.45.0.497603219066.issue28876@psf.upfronthosting.co.za>
Message-ID: <1485150615.86.0.718708859264.issue28876@psf.upfronthosting.co.za>


Raymond Hettinger added the comment:

This patch looks ready to go.  I'll wait a bit to see it there are any other comments.  If not, I'll apply it shortly.

----------
assignee:  -> rhettinger

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28876>
_______________________________________

From report at bugs.python.org  Mon Jan 23 00:53:53 2017
From: report at bugs.python.org (INADA Naoki)
Date: Mon, 23 Jan 2017 05:53:53 +0000
Subject: [issue28999] Use Py_RETURN_NONE and like
In-Reply-To: <1481969387.72.0.775552115596.issue28999@psf.upfronthosting.co.za>
Message-ID: <1485150833.16.0.0329076286021.issue28999@psf.upfronthosting.co.za>


INADA Naoki added the comment:

Oh, I feel three LGTMs are positive signal.
As I commented on ML, I think "ask forgiveness than permission" is
realistic approach for patches like this.
But it's up to you.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28999>
_______________________________________

From report at bugs.python.org  Mon Jan 23 01:38:41 2017
From: report at bugs.python.org (Roundup Robot)
Date: Mon, 23 Jan 2017 06:38:41 +0000
Subject: [issue29335] subprocess module does not check WIFSTOPPED on SIGCHLD
In-Reply-To: <1484945288.47.0.935636853008.issue29335@psf.upfronthosting.co.za>
Message-ID: <20170123063838.63002.78580.73CCACE2@psf.io>


Roundup Robot added the comment:

New changeset 8e3d412f8e89 by Gregory P. Smith in branch '2.7':
Issue #29335: Fix subprocess.Popen.wait() when the child process has
https://hg.python.org/cpython/rev/8e3d412f8e89

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29335>
_______________________________________

From report at bugs.python.org  Mon Jan 23 01:39:24 2017
From: report at bugs.python.org (Tang Weizhi)
Date: Mon, 23 Jan 2017 06:39:24 +0000
Subject: [issue29348] this comment 'iff' should be 'if'?
Message-ID: <1485153564.45.0.440343066023.issue29348@psf.upfronthosting.co.za>


New submission from Tang Weizhi:

I think this comment 'if' should be more clearly, or 'iff' is the abbreviation of 'if and only if'?

----------
assignee: docs at python
components: Documentation
messages: 286051
nosy: Tangwz, docs at python
priority: normal
pull_requests: 20
severity: normal
status: open
title: this comment 'iff' should be 'if'?
type: enhancement
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29348>
_______________________________________

From report at bugs.python.org  Mon Jan 23 01:46:35 2017
From: report at bugs.python.org (INADA Naoki)
Date: Mon, 23 Jan 2017 06:46:35 +0000
Subject: [issue29348] this comment 'iff' should be 'if'?
In-Reply-To: <1485153564.45.0.440343066023.issue29348@psf.upfronthosting.co.za>
Message-ID: <1485153995.89.0.537288721033.issue29348@psf.upfronthosting.co.za>


INADA Naoki added the comment:

"iff" is short form of "if and only if".

----------
nosy: +inada.naoki
resolution:  -> not a bug
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29348>
_______________________________________

From report at bugs.python.org  Mon Jan 23 02:32:28 2017
From: report at bugs.python.org (INADA Naoki)
Date: Mon, 23 Jan 2017 07:32:28 +0000
Subject: [issue28876] bool of large range raises OverflowError
In-Reply-To: <1480930662.45.0.497603219066.issue28876@psf.upfronthosting.co.za>
Message-ID: <1485156748.79.0.297482859973.issue28876@psf.upfronthosting.co.za>


INADA Naoki added the comment:

LGTM, except 2-space indent.

----------
nosy: +inada.naoki

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28876>
_______________________________________

From report at bugs.python.org  Mon Jan 23 02:47:49 2017
From: report at bugs.python.org (Roundup Robot)
Date: Mon, 23 Jan 2017 07:47:49 +0000
Subject: [issue28999] Use Py_RETURN_NONE and like
In-Reply-To: <1481969387.72.0.775552115596.issue28999@psf.upfronthosting.co.za>
Message-ID: <20170123074746.15241.91478.9D359DC8@psf.io>


Roundup Robot added the comment:

New changeset df87db35833e by Serhiy Storchaka in branch 'default':
Issue #28999: Use Py_RETURN_NONE, Py_RETURN_TRUE and Py_RETURN_FALSE wherever
https://hg.python.org/cpython/rev/df87db35833e

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28999>
_______________________________________

From report at bugs.python.org  Mon Jan 23 02:54:50 2017
From: report at bugs.python.org (Gregory P. Smith)
Date: Mon, 23 Jan 2017 07:54:50 +0000
Subject: [issue29335] subprocess module does not check WIFSTOPPED on SIGCHLD
In-Reply-To: <1484945288.47.0.935636853008.issue29335@psf.upfronthosting.co.za>
Message-ID: <1485158090.03.0.319684080631.issue29335@psf.upfronthosting.co.za>


Gregory P. Smith added the comment:

thanks Ned, I was awaiting interesting buildbot results. :)

fixed in 2.7 and 3.5 onwards.  thanks for the report Zach.

not closing until I also apply the fix to the subprocess32 backport.

----------
resolution:  -> fixed
stage: patch review -> commit review

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29335>
_______________________________________

From report at bugs.python.org  Mon Jan 23 03:24:25 2017
From: report at bugs.python.org (Roundup Robot)
Date: Mon, 23 Jan 2017 08:24:25 +0000
Subject: [issue28999] Use Py_RETURN_NONE and like
In-Reply-To: <1481969387.72.0.775552115596.issue28999@psf.upfronthosting.co.za>
Message-ID: <20170123082422.111706.93131.F06F50DB@psf.io>


Roundup Robot added the comment:

New changeset 2c724f45f23f by Serhiy Storchaka in branch 'default':
Issue #28999: Use Py_RETURN_NONE, Py_RETURN_TRUE and Py_RETURN_FALSE wherever
https://hg.python.org/cpython/rev/2c724f45f23f

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28999>
_______________________________________

From report at bugs.python.org  Mon Jan 23 03:31:15 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Mon, 23 Jan 2017 08:31:15 +0000
Subject: [issue28999] Use Py_RETURN_NONE and like
In-Reply-To: <1481969387.72.0.775552115596.issue28999@psf.upfronthosting.co.za>
Message-ID: <1485160275.55.0.281885084433.issue28999@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

Thank you Raymond. I didn't expect to get an approval from you and were ready to withdraw the patch. I myself were just +0 on pushing these changes.

Omitted changes in Modules/xx*.c and added few changes which Coccinelle missed.

----------
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28999>
_______________________________________

From report at bugs.python.org  Mon Jan 23 03:33:53 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Mon, 23 Jan 2017 08:33:53 +0000
Subject: [issue29328] struct module should support variable-length strings
In-Reply-To: <1484850272.25.0.472309940141.issue29328@psf.upfronthosting.co.za>
Message-ID: <1485160433.7.0.866065729711.issue29328@psf.upfronthosting.co.za>


Raymond Hettinger added the comment:

FWIW, the existence of netstruct https://github.com/stendec/netstruct does at least show that a coherent proposal is possible (the docs provide an API, motivation and examples).

One question is whether this could be implemented in the stdlib struct module without major brain surgery to the existing code which is organized around an initial prepare_s() step whose output is a PyStructObject which includes a fixed s_size field.  The various parsing routines all depend on the s_size field.  Where this proposal would require too much of a total rewrite is likely only answerable by someone building a patch with tests and real-world examples.

Another question is whether supporting variable length input would be better served by providing an alternative API that is better designed for it.  For example, unpack(fmt, input_stream or iterator, limit=None) which would consume as many bytes as needed from the input stream with an optional limit.  This would be nicer than having to figure-out in advance how to extract an input string of the appropriate size.

A last question is whether this should remain outside the standard library.  With PyPI becoming so rich and easy to access, we're often deciding that code is better-off outside the standard library where it can flourish in a more fluid environment.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29328>
_______________________________________

From report at bugs.python.org  Mon Jan 23 03:46:39 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Mon, 23 Jan 2017 08:46:39 +0000
Subject: [issue29338] Output the text signature in the help of a class
In-Reply-To: <1484991756.99.0.970269270756.issue29338@psf.upfronthosting.co.za>
Message-ID: <1485161199.76.0.343174235129.issue29338@psf.upfronthosting.co.za>


Raymond Hettinger added the comment:

The patch looks fine and meets a real need.  I say go ahead and apply it if no other objections arise.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29338>
_______________________________________

From report at bugs.python.org  Mon Jan 23 03:56:54 2017
From: report at bugs.python.org (Christian Heimes)
Date: Mon, 23 Jan 2017 08:56:54 +0000
Subject: [issue29347] Python 2.7.8 is crashing while creating weakref for a
 given object.
In-Reply-To: <1485141841.25.0.0993319189306.issue29347@psf.upfronthosting.co.za>
Message-ID: <1485161814.87.0.875349965218.issue29347@psf.upfronthosting.co.za>


Christian Heimes added the comment:

Can you reproduce the issue with a more recent version of Python 2.7? 2.7.8 is pretty old.

----------
nosy: +christian.heimes

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29347>
_______________________________________

From report at bugs.python.org  Mon Jan 23 03:57:04 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Mon, 23 Jan 2017 08:57:04 +0000
Subject: [issue26729] Incorrect __text_signature__ for sorted
In-Reply-To: <1460310880.55.0.462687598027.issue26729@psf.upfronthosting.co.za>
Message-ID: <1485161824.05.0.156583684049.issue26729@psf.upfronthosting.co.za>


Changes by Raymond Hettinger <raymond.hettinger at gmail.com>:


----------
assignee: rhettinger -> serhiy.storchaka

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue26729>
_______________________________________

From report at bugs.python.org  Mon Jan 23 04:07:48 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Mon, 23 Jan 2017 09:07:48 +0000
Subject: [issue26729] Incorrect __text_signature__ for sorted
In-Reply-To: <1460310880.55.0.462687598027.issue26729@psf.upfronthosting.co.za>
Message-ID: <1485162468.86.0.766998533448.issue26729@psf.upfronthosting.co.za>


Raymond Hettinger added the comment:

Serhiy, do you want to apply this?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue26729>
_______________________________________

From report at bugs.python.org  Mon Jan 23 04:14:32 2017
From: report at bugs.python.org (Marc-Andre Lemburg)
Date: Mon, 23 Jan 2017 09:14:32 +0000
Subject: [issue28749] Fixed the documentation of the mapping codec APIs
In-Reply-To: <1485114168.37.0.947999423927.issue28749@psf.upfronthosting.co.za>
Message-ID: <ed92c452-82d3-7a5a-c016-3c48f2ebd87c@egenix.com>


Marc-Andre Lemburg added the comment:

>> The only part that is not correct is "single string characters".
>> This should read "single bytes" or "bytes strings of length 1".
>
> This is not correct. Decoding mappings map not bytes strings, but
integers.

Looking at the implementation, you're right. AFAIR, the first
incarnation of the charmap codec used single chars in Python 2.
I guess the documentation was never updated when the change was
made to use integers instead.

> And this is not the only incorrect part. Decoding mappings can map to
> multicharacter Unicode strings, not to single Unicode	 characters. Not
just
> None, but the integer 0xfffe and Unicode string '\ufffe' mean "undefined
> mapping".

Yes, this was added later on as well. Apparently the docs
were never updated.

> There are similar incorrectnesses about encoding mappings.

Ok, fair enough, let's remove the two paragraphs.

>> I also don't see where you copied the description. Without some
>> description of what "mappings" are in the context of the charmap
>> codec, it's not easy to understand what the purpose of these
>> APIs is. Please just fix the bytes wording instead of removing the
>> whole intro.
>
> Decoding mappings were desribed in the introduction and in the
description of
> PyUnicode_DecodeCharmap() (both are outdated and incomplete). I merged
and
> corrected descriptions and left it only in one place, since
> PyUnicode_DecodeCharmap() is the only function that needs this. Same for
> encoding mappings. Both decoding and encoding mappings do not have a
relation
> to PyUnicode_Translate(). The paragraph about a LookupError in the
> introduction was totally wrong. I left in the introduction only common
part.
> Other details are too different in decoding, encoding and translation
mappings.
>
>> >> Also, this wording needs to be corrected: "bytes (integers in the
range
>> >> from 0 to 255)". Bytes are not integers. I'd suggest to use the more
>> >> correct wording "bytes strings of length 1".>
>> > The word "bytes" means here not Python bytes object, but is used in
more
>> > common meaning: an integer in the range from 0 to 255.
>> That's confusing, since we use the term "bytes" as referring
>> to the bytes object in Python. Please use "integers in the range
>> 0-255".
>
> Okay, I'll remove the word "bytes" here. But how would you formulate the
> following sentence: "Unmapped bytes (ones which cause a
:exc:`LookupError`) as
> well as mapped to ``None``, ``0xFFFE`` or ``'\ufffe'`` are treated as
"undefined
> mapping" and cause an error."?

Better:

"""
If *mapping* is *NULL*, Latin-1 decoding will be applied.  Else
*mapping* must map bytes ordinals (integers in the range from 0 to 255)
to Unicode strings, integers (which are then interpreted as Unicode
ordinals) or ``None``. Unmapped data bytes - ones which cause a
:exc:`LookupError`, as well as ones which get mapped to ``None``,
``0xFFFE`` or ``'\ufffe'``, are treated as undefined mappings and cause
an error.
"""

>> Aside: The deprecation of PyUnicode_EncodeCharmap() also seems misplaced
>> in this context, since only the Py_UNICODE version of the API is
>> deprecated. The functionality still exists and is useful. An API
>> similar to the _PyUnicode_EncodeCharmap() API should be made publicly
>> available to accommodate for the deprecation, since the mentioned
>> PyUnicode_AsCharmapString() and PyUnicode_AsEncodedString()
>> APIs are not suitable as replacement. PyUnicode_AsCharmapString()
>> doesn't support error handling (strange, BTW) and
>> PyUnicode_AsEncodedString() has a completely unrelated meaning (no
>> idea why it's mentioned here at all).
>
> Only PyUnicode_EncodeCharmap() is deprecated,
PyUnicode_AsCharmapString() is
> not deprecated. I placed the deprecated function just after its
non-deprecated
> counerpart following the pattern for other deprecated functions. If
you prefer
> I'll move both deprecated functions (PyUnicode_EncodeCharmap and
> PyUnicode_TranslateCharmap) together at the end of this section.

No, I'd prefer this deprecation to be undone as long as we
don't have a proper alternative for the API.

Looking at the various deprecations for the Py_UNICODE APIs,
I find that the Unicode API symmetry was severely broken.
In the Python 2 API, we always have an PyUnicode_Encode...() and
corresponding PyUnicode_Decode...() API for every codec.

In Python 3, the encode APIs were apparently all deprecated
due to their use of Py_UNICODE and only the the much less useful
PyUnicode_As...String() APIs were left, which intentionally do not
have an error argument, because they were intended as quick
replacement for PyString_AsString() uses in Python 2.

> I don't know why PyUnicode_AsCharmapString() don't support the errors
> argument. I added PyUnicode_AsEncodedString() as a replacement
(issue19569)
> because this is the only public non-deprecated way to do a charmap
encoding
> with errors handling. There is no exact equivalent, but
> PyUnicode_AsCharmapString() and PyUnicode_AsEncodedString() cover
different
> areas of using PyUnicode_EncodeCharmap().

The only way around this is to have new APIs to reestablish
the previous encoding functionality (only based on
PyObject *unicode instead of Py_UNICODE) at the C API level.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28749>
_______________________________________

From mal at egenix.com  Mon Jan 23 04:14:28 2017
From: mal at egenix.com (M.-A. Lemburg)
Date: Mon, 23 Jan 2017 10:14:28 +0100
Subject: [issue28749] Fixed the documentation of the mapping codec APIs
In-Reply-To: <1485114168.37.0.947999423927.issue28749@psf.upfronthosting.co.za>
References: <1485114168.37.0.947999423927.issue28749@psf.upfronthosting.co.za>
Message-ID: <ed92c452-82d3-7a5a-c016-3c48f2ebd87c@egenix.com>

>> The only part that is not correct is "single string characters".
>> This should read "single bytes" or "bytes strings of length 1".
>
> This is not correct. Decoding mappings map not bytes strings, but
integers.

Looking at the implementation, you're right. AFAIR, the first
incarnation of the charmap codec used single chars in Python 2.
I guess the documentation was never updated when the change was
made to use integers instead.

> And this is not the only incorrect part. Decoding mappings can map to
> multicharacter Unicode strings, not to single Unicode	 characters. Not
just
> None, but the integer 0xfffe and Unicode string '\ufffe' mean "undefined
> mapping".

Yes, this was added later on as well. Apparently the docs
were never updated.

> There are similar incorrectnesses about encoding mappings.

Ok, fair enough, let's remove the two paragraphs.

>> I also don't see where you copied the description. Without some
>> description of what "mappings" are in the context of the charmap
>> codec, it's not easy to understand what the purpose of these
>> APIs is. Please just fix the bytes wording instead of removing the
>> whole intro.
>
> Decoding mappings were desribed in the introduction and in the
description of
> PyUnicode_DecodeCharmap() (both are outdated and incomplete). I merged
and
> corrected descriptions and left it only in one place, since
> PyUnicode_DecodeCharmap() is the only function that needs this. Same for
> encoding mappings. Both decoding and encoding mappings do not have a
relation
> to PyUnicode_Translate(). The paragraph about a LookupError in the
> introduction was totally wrong. I left in the introduction only common
part.
> Other details are too different in decoding, encoding and translation
mappings.
>
>> >> Also, this wording needs to be corrected: "bytes (integers in the
range
>> >> from 0 to 255)". Bytes are not integers. I'd suggest to use the more
>> >> correct wording "bytes strings of length 1".>
>> > The word "bytes" means here not Python bytes object, but is used in
more
>> > common meaning: an integer in the range from 0 to 255.
>> That's confusing, since we use the term "bytes" as referring
>> to the bytes object in Python. Please use "integers in the range
>> 0-255".
>
> Okay, I'll remove the word "bytes" here. But how would you formulate the
> following sentence: "Unmapped bytes (ones which cause a
:exc:`LookupError`) as
> well as mapped to ``None``, ``0xFFFE`` or ``'\ufffe'`` are treated as
"undefined
> mapping" and cause an error."?

Better:

"""
If *mapping* is *NULL*, Latin-1 decoding will be applied.  Else
*mapping* must map bytes ordinals (integers in the range from 0 to 255)
to Unicode strings, integers (which are then interpreted as Unicode
ordinals) or ``None``. Unmapped data bytes - ones which cause a
:exc:`LookupError`, as well as ones which get mapped to ``None``,
``0xFFFE`` or ``'\ufffe'``, are treated as undefined mappings and cause
an error.
"""

>> Aside: The deprecation of PyUnicode_EncodeCharmap() also seems misplaced
>> in this context, since only the Py_UNICODE version of the API is
>> deprecated. The functionality still exists and is useful. An API
>> similar to the _PyUnicode_EncodeCharmap() API should be made publicly
>> available to accommodate for the deprecation, since the mentioned
>> PyUnicode_AsCharmapString() and PyUnicode_AsEncodedString()
>> APIs are not suitable as replacement. PyUnicode_AsCharmapString()
>> doesn't support error handling (strange, BTW) and
>> PyUnicode_AsEncodedString() has a completely unrelated meaning (no
>> idea why it's mentioned here at all).
>
> Only PyUnicode_EncodeCharmap() is deprecated,
PyUnicode_AsCharmapString() is
> not deprecated. I placed the deprecated function just after its
non-deprecated
> counerpart following the pattern for other deprecated functions. If
you prefer
> I'll move both deprecated functions (PyUnicode_EncodeCharmap and
> PyUnicode_TranslateCharmap) together at the end of this section.

No, I'd prefer this deprecation to be undone as long as we
don't have a proper alternative for the API.

Looking at the various deprecations for the Py_UNICODE APIs,
I find that the Unicode API symmetry was severely broken.
In the Python 2 API, we always have an PyUnicode_Encode...() and
corresponding PyUnicode_Decode...() API for every codec.

In Python 3, the encode APIs were apparently all deprecated
due to their use of Py_UNICODE and only the the much less useful
PyUnicode_As...String() APIs were left, which intentionally do not
have an error argument, because they were intended as quick
replacement for PyString_AsString() uses in Python 2.

> I don't know why PyUnicode_AsCharmapString() don't support the errors
> argument. I added PyUnicode_AsEncodedString() as a replacement
(issue19569)
> because this is the only public non-deprecated way to do a charmap
encoding
> with errors handling. There is no exact equivalent, but
> PyUnicode_AsCharmapString() and PyUnicode_AsEncodedString() cover
different
> areas of using PyUnicode_EncodeCharmap().

The only way around this is to have new APIs to reestablish
the previous encoding functionality (only based on
PyObject *unicode instead of Py_UNICODE) at the C API level.

-- 
Marc-Andre Lemburg
eGenix.com

From report at bugs.python.org  Mon Jan 23 04:41:21 2017
From: report at bugs.python.org (Jim Fasarakis-Hilliard)
Date: Mon, 23 Jan 2017 09:41:21 +0000
Subject: [issue29349] Update old Python 2 code in
 Docs/tools/extensions/patchlevel.py
Message-ID: <1485164481.25.0.457345760711.issue29349@psf.upfronthosting.co.za>


New submission from Jim Fasarakis-Hilliard:

There's a hidden Python 2 print call in the script that only gets reached after you move the Doc/ folder outside the main CPython directory and run `make html`. 

Additionally, an obsolete way of assuring a file gets closed is used (changed to use `with` statement).

P.S: Tagged this with the Documentation component, maybe build is more applicable but I wasn't sure.

----------
assignee: docs at python
components: Documentation
files: patchlevel.patch
keywords: patch
messages: 286065
nosy: Jim Fasarakis-Hilliard, docs at python
priority: normal
severity: normal
status: open
title: Update old Python 2 code in Docs/tools/extensions/patchlevel.py
type: behavior
versions: Python 3.4, Python 3.5, Python 3.6, Python 3.7
Added file: http://bugs.python.org/file46387/patchlevel.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29349>
_______________________________________

From report at bugs.python.org  Mon Jan 23 04:57:15 2017
From: report at bugs.python.org (Jim Fasarakis-Hilliard)
Date: Mon, 23 Jan 2017 09:57:15 +0000
Subject: [issue29339] Interactive: Move to same indentation level as previous
 line
In-Reply-To: <1485039375.56.0.473197770315.issue29339@psf.upfronthosting.co.za>
Message-ID: <1485165435.71.0.613295341246.issue29339@psf.upfronthosting.co.za>


Jim Fasarakis-Hilliard added the comment:

I'm not exactly sure what you mean but, since this isn't a bug per se and is more of a subjective opinion on how the REPL should handle the indentation level, you should probably ask *first* on python-ideas to get input from other members. (See https://docs.python.org/devguide/faq.html#suggesting-changes)

----------
nosy: +Jim Fasarakis-Hilliard

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29339>
_______________________________________

From report at bugs.python.org  Mon Jan 23 05:00:54 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Mon, 23 Jan 2017 10:00:54 +0000
Subject: [issue27867] various issues due to misuse of PySlice_GetIndicesEx
In-Reply-To: <1472226685.15.0.242668202848.issue27867@psf.upfronthosting.co.za>
Message-ID: <1485165654.85.0.544484225443.issue27867@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

We can't just add API functions in maintained releases, because it will break the stable ABI. We can use them only when explicitly define the version of API.

Proposed patch for 3.6 and 3.7 adds public API functions PySlice_Unpack() and PySlice_AdjustIndices() and makes PySlice_GetIndicesEx() a macro if set Py_LIMITED_API to the version that supports new API. Otherwise PySlice_GetIndicesEx() becomes deprecated.

This doesn't break extensions compiled with older Python versions. Extensions compiled with new Python versions without limited API or with high API version are not compatible with older Python versions as expected, but have fixed the original issue. Compiling extensions with new Python versions with set low Py_LIMITED_API value will produce a deprecation warning.

Pay attention to names and signatures of new API. It would be hard to change it when it added.

I think this is the safest way. In 2.7 we should replace PySlice_GetIndicesEx() with a macro for internal use only if we want to fix an issue for builtins and preserve a binary compatibility.

----------
priority: normal -> high
Added file: http://bugs.python.org/file46388/slice_get_indices_3.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue27867>
_______________________________________

From report at bugs.python.org  Mon Jan 23 05:26:05 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Mon, 23 Jan 2017 10:26:05 +0000
Subject: [issue29350] Add support of multiple signatures
Message-ID: <1485167165.19.0.665694725089.issue29350@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka:

Some functions can be described by the single signature. See examples in msg285647. Selected examples:

  dict.pop(key)
  dict.pop(key, default)

  type(obj)
  type(name, bases, mapping)

  range(stop)
  range(start, stop, step=1)

  min(iterable, *, key=identity)
  min(iterable, *, default, key=identity)
  min(*args, key=identity)

I think the only way to resolve this problem is to add the support of multiple signatures in inspect, pydoc, Argument Clinic, etc.

----------
components: Library (Lib)
messages: 286068
nosy: larry, rhettinger, serhiy.storchaka, yselivanov
priority: normal
severity: normal
status: open
title: Add support of multiple signatures
type: enhancement
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29350>
_______________________________________

From report at bugs.python.org  Mon Jan 23 05:31:20 2017
From: report at bugs.python.org (Roundup Robot)
Date: Mon, 23 Jan 2017 10:31:20 +0000
Subject: [issue26729] Incorrect __text_signature__ for sorted
In-Reply-To: <1460310880.55.0.462687598027.issue26729@psf.upfronthosting.co.za>
Message-ID: <20170123103117.62837.2408.4960B007@psf.io>


Roundup Robot added the comment:

New changeset c0a9fb3e19b9 by Serhiy Storchaka in branch '3.5':
Issue #26729: Fixed __text_signature__ for sorted().
https://hg.python.org/cpython/rev/c0a9fb3e19b9

New changeset fcb19fb42058 by Serhiy Storchaka in branch '3.6':
Issue #26729: Fixed __text_signature__ for sorted().
https://hg.python.org/cpython/rev/fcb19fb42058

New changeset 4ad195d9e184 by Serhiy Storchaka in branch 'default':
Issue #26729: Fixed __text_signature__ for sorted().
https://hg.python.org/cpython/rev/4ad195d9e184

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue26729>
_______________________________________

From report at bugs.python.org  Mon Jan 23 05:31:57 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Mon, 23 Jan 2017 10:31:57 +0000
Subject: [issue26729] Incorrect __text_signature__ for sorted
In-Reply-To: <1460310880.55.0.462687598027.issue26729@psf.upfronthosting.co.za>
Message-ID: <1485167517.84.0.391690602923.issue26729@psf.upfronthosting.co.za>


Changes by Serhiy Storchaka <storchaka+cpython at gmail.com>:


----------
resolution:  -> fixed
stage: commit review -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue26729>
_______________________________________

From report at bugs.python.org  Mon Jan 23 05:37:19 2017
From: report at bugs.python.org (Roundup Robot)
Date: Mon, 23 Jan 2017 10:37:19 +0000
Subject: [issue29338] Output the text signature in the help of a class
In-Reply-To: <1484991756.99.0.970269270756.issue29338@psf.upfronthosting.co.za>
Message-ID: <20170123103716.6961.58685.1DD12F0E@psf.io>


Roundup Robot added the comment:

New changeset 3d5dcdf26fab by Serhiy Storchaka in branch 'default':
Issue #29338: The help of a builtin or extension class now includes the
https://hg.python.org/cpython/rev/3d5dcdf26fab

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29338>
_______________________________________

From report at bugs.python.org  Mon Jan 23 05:37:57 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Mon, 23 Jan 2017 10:37:57 +0000
Subject: [issue29338] Output the text signature in the help of a class
In-Reply-To: <1484991756.99.0.970269270756.issue29338@psf.upfronthosting.co.za>
Message-ID: <1485167877.05.0.478819431794.issue29338@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

Thanks Raymond.

----------
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29338>
_______________________________________

From report at bugs.python.org  Mon Jan 23 06:03:08 2017
From: report at bugs.python.org (STINNER Victor)
Date: Mon, 23 Jan 2017 11:03:08 +0000
Subject: [issue28651] Make objects with empty __slots__ GC types
In-Reply-To: <1478716305.57.0.745057898966.issue28651@psf.upfronthosting.co.za>
Message-ID: <1485169388.19.0.991777042575.issue28651@psf.upfronthosting.co.za>


STINNER Victor added the comment:

This issue was mentioned in another issue when a refleak was found:

http://bugs.python.org/issue24379#msg280456

----------
nosy: +haypo

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28651>
_______________________________________

From report at bugs.python.org  Mon Jan 23 06:04:03 2017
From: report at bugs.python.org (STINNER Victor)
Date: Mon, 23 Jan 2017 11:04:03 +0000
Subject: [issue24379] Add operator.subscript as a convenience for creating
 slices
In-Reply-To: <1433398024.28.0.0919512106527.issue24379@psf.upfronthosting.co.za>
Message-ID: <1485169443.97.0.803899673269.issue24379@psf.upfronthosting.co.za>


Changes by STINNER Victor <victor.stinner at gmail.com>:


----------
nosy: +haypo

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24379>
_______________________________________

From report at bugs.python.org  Mon Jan 23 06:14:38 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Mon, 23 Jan 2017 11:14:38 +0000
Subject: [issue29338] Output the text signature in the help of a class
In-Reply-To: <1484991756.99.0.970269270756.issue29338@psf.upfronthosting.co.za>
Message-ID: <1485170078.09.0.0346194279816.issue29338@psf.upfronthosting.co.za>


Xiang Zhang added the comment:

The buildbots are failing due to test_pydoc. :-(

----------
nosy: +xiang.zhang

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29338>
_______________________________________

From report at bugs.python.org  Mon Jan 23 06:15:19 2017
From: report at bugs.python.org (STINNER Victor)
Date: Mon, 23 Jan 2017 11:15:19 +0000
Subject: [issue18835] Add aligned memory variants to the suite of PyMem
 functions/macros
In-Reply-To: <1377484371.18.0.399621580043.issue18835@psf.upfronthosting.co.za>
Message-ID: <1485170119.69.0.420958623418.issue18835@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Antoine Pitrou: "Benchmarks and Intel's recommendation show that aligned allocation is actually important for AVX performance, and NumPy depends on CPython providing the right allocation APIs (for integration with tracemalloc): https://github.com/numpy/numpy/issues/5312"

I don't think that NumPy was ever fully integrated with tracemalloc.

Since Python 3.6, NumPy doesn't have to use Python memory allocators to benefit of tracemalloc debugger: I added a new C API to be able to manually track/untrack memory blocks which are not directly allocated by Python, see the issue #26530. I implemented this feature for NumPy, but since I never got any feedback from NumPy, I left the API private.

Moreover, I also added second feature to tracemalloc: it's now possible to track memory allocation in different address spaces. The feature was also designed for NumPy which can allocate memory in the GPU address space. See the issue #26588.

With these new tracemalloc features, I don't think that NumPy can still be used to request this feature in CPython core.

--

Raymond: "A principal use case would be PyObject pointers where we want to keep all or most of the data fields in the same cache line (i.e. the fields for list, tuple, dict, and set objects).

Deques would benefit from having the deque blocks aligned to 64byte boundaries and never crossing page boundaries.  Set entries would benefit from 32byte alignment."

Victor (me!): "Do you have an idea of performance benefit of memory alignment?"

Since Raymond never provided any evidence that a new aligned memory allocator would give a significant speedup, and there issue is inactive for 2 years, I close it.

See also the change 6e16b0045cf1, it seems like Raymond doesn't plan to use this feature anymore.

--

If someone wants this feature, we need good reasons to implement it.

----------
resolution:  -> rejected
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue18835>
_______________________________________

From report at bugs.python.org  Mon Jan 23 06:24:38 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Mon, 23 Jan 2017 11:24:38 +0000
Subject: [issue28749] Fixed the documentation of the mapping codec APIs
In-Reply-To: <1479637798.25.0.994184606714.issue28749@psf.upfronthosting.co.za>
Message-ID: <1485170678.71.0.366248206086.issue28749@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

"bytes ordinals" is good term. Thank you. Here is an updated patch.

> No, I'd prefer this deprecation to be undone as long as we
> don't have a proper alternative for the API.

This is different issue.

----------
Added file: http://bugs.python.org/file46389/docs-PyUnicode_Translate-4.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28749>
_______________________________________

From report at bugs.python.org  Mon Jan 23 06:30:18 2017
From: report at bugs.python.org (Roundup Robot)
Date: Mon, 23 Jan 2017 11:30:18 +0000
Subject: [issue26273] Expose TCP_CONGESTION and TCP_USER_TIMEOUT to the socket
 module
In-Reply-To: <1454489684.49.0.598051421532.issue26273@psf.upfronthosting.co.za>
Message-ID: <20170123113014.14323.84921.B6AAFD79@psf.io>


Roundup Robot added the comment:

New changeset ffb2534fcbf1 by Victor Stinner in branch '3.6':
Issue #26273: Document TCP_USER_TIMEOUT and TCP_CONGESTION
https://hg.python.org/cpython/rev/ffb2534fcbf1

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue26273>
_______________________________________

From report at bugs.python.org  Mon Jan 23 06:31:12 2017
From: report at bugs.python.org (STINNER Victor)
Date: Mon, 23 Jan 2017 11:31:12 +0000
Subject: [issue26273] Expose TCP_CONGESTION and TCP_USER_TIMEOUT to the socket
 module
In-Reply-To: <1454489684.49.0.598051421532.issue26273@psf.upfronthosting.co.za>
Message-ID: <1485171072.33.0.402216863717.issue26273@psf.upfronthosting.co.za>


STINNER Victor added the comment:

> Patch looks good to me.

Oh, I forgot socket_doc.patch! Thanks Serhiy for the reminder and Martin for the review :-) I just pushed the patch.

----------
resolution:  -> fixed
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue26273>
_______________________________________

From report at bugs.python.org  Mon Jan 23 07:02:53 2017
From: report at bugs.python.org (Roundup Robot)
Date: Mon, 23 Jan 2017 12:02:53 +0000
Subject: [issue29338] Output the text signature in the help of a class
In-Reply-To: <1484991756.99.0.970269270756.issue29338@psf.upfronthosting.co.za>
Message-ID: <20170123120249.111947.31673.DC1D3097@psf.io>


Roundup Robot added the comment:

New changeset cebc9c7ad195 by Serhiy Storchaka in branch 'default':
Issue #29338: Don't output an empty signature for class constructor.
https://hg.python.org/cpython/rev/cebc9c7ad195

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29338>
_______________________________________

From report at bugs.python.org  Mon Jan 23 07:03:50 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Mon, 23 Jan 2017 12:03:50 +0000
Subject: [issue29338] Output the text signature in the help of a class
In-Reply-To: <1484991756.99.0.970269270756.issue29338@psf.upfronthosting.co.za>
Message-ID: <1485173030.75.0.219301711871.issue29338@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

Oh, I forgot to run tests for such simple change! Thanks for reminder Xiang!

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29338>
_______________________________________

From report at bugs.python.org  Mon Jan 23 07:20:05 2017
From: report at bugs.python.org (desbma)
Date: Mon, 23 Jan 2017 12:20:05 +0000
Subject: [issue29212] Python 3.6 logging thread name regression with
 concurrent.future threads
In-Reply-To: <1483938963.09.0.315595404976.issue29212@psf.upfronthosting.co.za>
Message-ID: <1485174005.58.0.0506988017987.issue29212@psf.upfronthosting.co.za>


desbma added the comment:

I don't think using repr(self) as a fallback was intentional, that is why I wrote regression, but I may be wrong.

I agree that the previous default 'Thread-x' gives little information, but only the user can give a relevant name to describe what a thread is doing.
Using a new default like "ThreadPoolExecutor-worker_x" would give more information but at the same time it leaks internal information about how the thread was created in the code (which is not relevant when using the logging module), not about what it's doing.

Anyway as long as that repr string is replaced, I am happy :)

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29212>
_______________________________________

From report at bugs.python.org  Mon Jan 23 07:52:07 2017
From: report at bugs.python.org (INADA Naoki)
Date: Mon, 23 Jan 2017 12:52:07 +0000
Subject: [issue29212] Python 3.6 logging thread name regression with
 concurrent.future threads
In-Reply-To: <1483938963.09.0.315595404976.issue29212@psf.upfronthosting.co.za>
Message-ID: <1485175927.68.0.27893167525.issue29212@psf.upfronthosting.co.za>


INADA Naoki added the comment:

OK. I'll wait for another opinions.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29212>
_______________________________________

From report at bugs.python.org  Mon Jan 23 08:00:59 2017
From: report at bugs.python.org (INADA Naoki)
Date: Mon, 23 Jan 2017 13:00:59 +0000
Subject: [issue29336] merge tuples in module
In-Reply-To: <1484969445.33.0.406694538272.issue29336@psf.upfronthosting.co.za>
Message-ID: <1485176459.55.0.129370948037.issue29336@psf.upfronthosting.co.za>


INADA Naoki added the comment:

Thanks.  Your patch reduced memory consumption by 2%,
and number of tuples by 15%.

$ cat invtuple.py
import app
import sys
import traceback
import tracemalloc

print(tracemalloc.get_traced_memory())
allobj = sys.getobjects(0, tuple)
print(len(allobj))

$ PYTHONTRACEMALLOC=1 venv/bin/python invtuple.py
(51491189, 51506311)
87143

$ PYTHONTRACEMALLOC=1 venv-mt/bin/python  invtuple.py
(50551519, 50566641)
74308

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29336>
_______________________________________

From report at bugs.python.org  Mon Jan 23 08:25:03 2017
From: report at bugs.python.org (Markus Gerstel)
Date: Mon, 23 Jan 2017 13:25:03 +0000
Subject: [issue29351] absolute imports for logging
Message-ID: <1485177903.18.0.43823292649.issue29351@psf.upfronthosting.co.za>


New submission from Markus Gerstel:

Running 'import logging' causes at minimum 46 failing 'open' and 12 failing 'stat' calls because python looks for packages inside python/Lib/logging which will never be there, in particular: sys, os, time, cStringIO, traceback, warnings, weakref, collections, codecs, thread, threading, atexit.

The impact of this is limited when python is installed locally, but noticeable when run on a networked file system.


How to reproduce: 
run 
$ strace python -c "import logging;" 2>&1 | grep ENOENT | grep "\/logging\/"


How to fix:
Add 'from __future__ import absolute_import' to all files in the logging directory.
A relevant patch is attached.

----------
components: Library (Lib)
files: 0001-absolute-import.patch
keywords: patch
messages: 286083
nosy: mgerstel
priority: normal
severity: normal
status: open
title: absolute imports for logging
type: resource usage
versions: Python 2.7
Added file: http://bugs.python.org/file46390/0001-absolute-import.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29351>
_______________________________________

From report at bugs.python.org  Mon Jan 23 08:28:18 2017
From: report at bugs.python.org (Ma Lin)
Date: Mon, 23 Jan 2017 13:28:18 +0000
Subject: [issue28518] execute("begin immediate") throwing OperationalError
In-Reply-To: <1477307118.31.0.70785963389.issue28518@psf.upfronthosting.co.za>
Message-ID: <1485178098.29.0.989354482255.issue28518@psf.upfronthosting.co.za>


Ma Lin added the comment:

If the answer is (no, no) , the behavior strictly follows the doc changes in commit 284676cf2ac8.
Anyway, I'm not a deep user of SQLite, I can't give further advices. :(

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28518>
_______________________________________

From report at bugs.python.org  Mon Jan 23 08:37:06 2017
From: report at bugs.python.org (kristall)
Date: Mon, 23 Jan 2017 13:37:06 +0000
Subject: [issue29152] unittest subTest does not call addFailure
In-Reply-To: <1483534655.28.0.0626841658388.issue29152@psf.upfronthosting.co.za>
Message-ID: <1485178626.94.0.137882651151.issue29152@psf.upfronthosting.co.za>


kristall added the comment:

Thanks for the quick 1st response. Since I put that part of my script on hold, I'd like to ask if it is possible to estimate when Antoine will have a look at this? Thanks in advance.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29152>
_______________________________________

From report at bugs.python.org  Mon Jan 23 08:38:28 2017
From: report at bugs.python.org (Ma Lin)
Date: Mon, 23 Jan 2017 13:38:28 +0000
Subject: [issue29325] pysqlite: Evaluate removal of sqlite3_stmt_readonly
In-Reply-To: <1484843450.51.0.0079411502064.issue29325@psf.upfronthosting.co.za>
Message-ID: <1485178708.92.0.367724983914.issue29325@psf.upfronthosting.co.za>


Ma Lin added the comment:

Just remind, there is a chance to remove sqlite3_stmt_readonly(). :)

----------
nosy: +berker.peksag, palaviv

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29325>
_______________________________________

From report at bugs.python.org  Mon Jan 23 09:10:53 2017
From: report at bugs.python.org (Berker Peksag)
Date: Mon, 23 Jan 2017 14:10:53 +0000
Subject: [issue28518] execute("begin immediate") throwing OperationalError
In-Reply-To: <1477307118.31.0.70785963389.issue28518@psf.upfronthosting.co.za>
Message-ID: <1485180653.66.0.0433354160045.issue28518@psf.upfronthosting.co.za>


Berker Peksag added the comment:

> 1. Should we add the VACUUM with a explicit commit? Maybe there should
> be an implicit commit before VACUUM?

VACUUM is often an expensive operation so I think people should need to explicitly handle it anyway.

> 2. Should a SELECT start a transaction? I think it should according to
> PEP 249. There is a open issue on the case (#9924). Should we just
> change this on this patch?

Let's discuss that in issue 9924 :) Last time I look at it there was some backward compatibility concern so we might able to fix it only in 3.7.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28518>
_______________________________________

From report at bugs.python.org  Mon Jan 23 09:50:18 2017
From: report at bugs.python.org (Michael Felt)
Date: Mon, 23 Jan 2017 14:50:18 +0000
Subject: [issue18235] _sysconfigdata.py wrong on AIX installations
In-Reply-To: <1371429201.94.0.372851990361.issue18235@psf.upfronthosting.co.za>
Message-ID: <1485183018.9.0.275013371412.issue18235@psf.upfronthosting.co.za>


Michael Felt added the comment:

This is "only" for Python-2.7 (for now). The others will be tested as I am able.

Working with the patch submitted 2013-10-19 (aka https://bugs.python.org/file32229/issue18235.patch)

A) Without/before the patch:

root at x064:[/data/prj/python/Python-2.7.13.0]grep LDSHARED ./build/lib.aix-5.3-2.7/_sysconfigdata.py
 'BLDSHARED': 'Modules/ld_so_aix xlc_r -bI:Modules/python.exp',
 'LDSHARED': 'Modules/ld_so_aix xlc_r -bI:Modules/python.exp',

B) Apply the patch:
diff -r 0a26ef834a49 Lib/sysconfig.py
--- a/Lib/sysconfig.py	Sat Oct 19 14:24:44 2013 +0200
+++ b/Lib/sysconfig.py	Sat Oct 19 11:46:10 2013 -0700
@@ -368,7 +368,7 @@
     # -- these paths are relative to the Python source, but when installed
     # the scripts are in another directory.
     if _PYTHON_BUILD:
-        vars['LDSHARED'] = vars['BLDSHARED']
+        vars['BLDSHARED'] = vars['LDSHARED']

After the patch:

root at x064:[/data/prj/python/Python-2.7.13.0]grep LDSHARED ./build/lib.aix-5.3-2.7/_sysconfigdata.py
 'BLDSHARED': '/opt/lib/python2.7/config/ld_so_aix xlc_r -bI:/opt/lib/python2.7/config/python.exp',
 'LDSHARED': '/opt/lib/python2.7/config/ld_so_aix xlc_r -bI:/opt/lib/python2.7/config/python.exp',

Which makes me think - maybe NOTHING is needed - as 'BLDSHARED' should be Modules/ld_so_aix,
while  'LDSHARED' should be '/opt/lib/python2.7/config/ld_so_aix

Rather than actual deletes: commented out to (after patch was applied)

C: if: statement and assignment - commented out
diff -ru Python-2.7.13/Lib/sysconfig.py Python-2.7.13.0/Lib/sysconfig.py
--- Python-2.7.13/Lib/sysconfig.py      2016-12-17 20:05:06 +0000
+++ Python-2.7.13.0/Lib/sysconfig.py    2017-01-23 14:39:31 +0000
@@ -310,8 +310,8 @@
     # On AIX, there are wrong paths to the linker scripts in the Makefile
     # -- these paths are relative to the Python source, but when installed
     # the scripts are in another directory.
-    if _PYTHON_BUILD:
-        vars['LDSHARED'] = vars['BLDSHARED']
+#    if _PYTHON_BUILD:
+#        vars['BLDSHARED'] = vars['LDSHARED']

root at x064:[/data/prj/python/Python-2.7.13.0]grep LDSHARED ./build/lib.aix-5.3-2.7/_sysconfigdata.py
 'BLDSHARED': 'Modules/ld_so_aix xlc_r -bI:Modules/python.exp',
 'LDSHARED': '/opt/lib/python2.7/config/ld_so_aix xlc_r -bI:/opt/lib/python2.7/config/python.exp',
root at x064:[/data/prj/python/Python-2.7.13.0]

So, perhaps the best patch 'today', versus what may have been best in 2013 - is to remove the 5 lines starting with # On AIX, ...

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue18235>
_______________________________________

From report at bugs.python.org  Mon Jan 23 10:00:58 2017
From: report at bugs.python.org (Akira Li)
Date: Mon, 23 Jan 2017 15:00:58 +0000
Subject: [issue28876] bool of large range raises OverflowError
In-Reply-To: <1480930662.45.0.497603219066.issue28876@psf.upfronthosting.co.za>
Message-ID: <1485183658.86.0.0381798348831.issue28876@psf.upfronthosting.co.za>


Akira Li added the comment:

I've updated the patch to use 4-space indent (pep-7).

I've added space around "=" (pep-7); unlike the usual
"dict(designator=value)" -- no space around "=" for keyword argument
(pep-8).

----------
Added file: http://bugs.python.org/file46391/range_bool-c99-designated-initializers-indent.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28876>
_______________________________________

From report at bugs.python.org  Mon Jan 23 10:03:23 2017
From: report at bugs.python.org (12345 67890)
Date: Mon, 23 Jan 2017 15:03:23 +0000
Subject: [issue29339] Interactive: Move to same indentation level as previous
 line
In-Reply-To: <1485039375.56.0.473197770315.issue29339@psf.upfronthosting.co.za>
Message-ID: <1485183803.35.0.920853372691.issue29339@psf.upfronthosting.co.za>


12345 67890 added the comment:

I did not mark this as a bug. I marked it as an "enhancement" and I said that the priority was low.

"I'm not exactly sure what you mean"

In an IDE when developing in python, each time you press enter, the cursor is set to the same indentation level as the previous line so that you don't need to type spaces on every new line to return to the same indentation level. I am merely suggesting that the interactive interpreter do the same thing. 

"is more of a subjective opinion"

You are of course correct that this is a subjective opinion. But it's not just my subjective opinion. As I said, it's what nearly every IDE does including (maybe?) the one that you're using.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29339>
_______________________________________

From report at bugs.python.org  Mon Jan 23 10:06:51 2017
From: report at bugs.python.org (Dries Desmet)
Date: Mon, 23 Jan 2017 15:06:51 +0000
Subject: [issue29351] absolute imports for logging
In-Reply-To: <1485177903.18.0.43823292649.issue29351@psf.upfronthosting.co.za>
Message-ID: <1485184011.19.0.0294673039549.issue29351@psf.upfronthosting.co.za>


Changes by Dries Desmet <dries at urga.be>:


----------
nosy: +dries_desmet

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29351>
_______________________________________

From report at bugs.python.org  Mon Jan 23 10:10:59 2017
From: report at bugs.python.org (Markus Gerstel)
Date: Mon, 23 Jan 2017 15:10:59 +0000
Subject: [issue29351] absolute imports for logging
In-Reply-To: <1485177903.18.0.43823292649.issue29351@psf.upfronthosting.co.za>
Message-ID: <1485184259.18.0.621897686456.issue29351@psf.upfronthosting.co.za>


Changes by Markus Gerstel <markus.gerstel at diamond.ac.uk>:


----------
nosy: +vinay.sajip

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29351>
_______________________________________

From report at bugs.python.org  Mon Jan 23 10:16:59 2017
From: report at bugs.python.org (Eli Collins)
Date: Mon, 23 Jan 2017 15:16:59 +0000
Subject: [issue29346] datetime.utcfromtimestamp() returns strange result for
 very large values
In-Reply-To: <1485107254.51.0.0185006280427.issue29346@psf.upfronthosting.co.za>
Message-ID: <1485184619.51.0.655148615509.issue29346@psf.upfronthosting.co.za>


Eli Collins added the comment:

My apologies, I think Alexander is right, this is a duplicate of #29100

I'd seen the first message in that issue talking about a core dump, and thought this was different problem; but I see the scope of the issue broadened to cover general bounds issues in later messages. 

Reviewing the fix & the tests from that issue's patches, it looks like exactly this case.

----------
resolution:  -> duplicate

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29346>
_______________________________________

From report at bugs.python.org  Mon Jan 23 11:15:21 2017
From: report at bugs.python.org (Mark Dickinson)
Date: Mon, 23 Jan 2017 16:15:21 +0000
Subject: [issue28876] bool of large range raises OverflowError
In-Reply-To: <1480930662.45.0.497603219066.issue28876@psf.upfronthosting.co.za>
Message-ID: <1485188121.67.0.747636193533.issue28876@psf.upfronthosting.co.za>


Mark Dickinson added the comment:

Latest patch LGTM too.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28876>
_______________________________________

From report at bugs.python.org  Mon Jan 23 11:17:26 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Mon, 23 Jan 2017 16:17:26 +0000
Subject: [issue20185] Derby #17: Convert 49 sites to Argument Clinic across 13
 files
In-Reply-To: <1389140539.55.0.106381625615.issue20185@psf.upfronthosting.co.za>
Message-ID: <1485188246.65.0.220643195592.issue20185@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

Updated patch for longobject.c addresses Martin's comments.

----------
Added file: http://bugs.python.org/file46392/clinic_longobject_v4.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue20185>
_______________________________________

From report at bugs.python.org  Mon Jan 23 11:29:56 2017
From: report at bugs.python.org (Alex Gaynor)
Date: Mon, 23 Jan 2017 16:29:56 +0000
Subject: [issue29136] Add OP_NO_TLSv1_3
In-Reply-To: <1483391244.76.0.29431388133.issue29136@psf.upfronthosting.co.za>
Message-ID: <1485188996.8.0.820497242997.issue29136@psf.upfronthosting.co.za>


Alex Gaynor added the comment:

We can easily just add `TLS13:...` at the from of our ciphersuite list and it'll be ok though right? (Note to self, do the same in urllib3, twisted, requests, god only knows what else)

----------
nosy: +alex

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29136>
_______________________________________

From report at bugs.python.org  Mon Jan 23 11:30:08 2017
From: report at bugs.python.org (Vinay Sajip)
Date: Mon, 23 Jan 2017 16:30:08 +0000
Subject: [issue29351] absolute imports for logging
In-Reply-To: <1485177903.18.0.43823292649.issue29351@psf.upfronthosting.co.za>
Message-ID: <1485189008.84.0.412619076249.issue29351@psf.upfronthosting.co.za>


Vinay Sajip added the comment:

There's nothing particular about logging here, is there? Any of the 2.7 stdlib modules will exhibit similar behaviour, since that __future__ import is not in general defined in the stdlib modules, other than in the test suite.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29351>
_______________________________________

From report at bugs.python.org  Mon Jan 23 11:35:46 2017
From: report at bugs.python.org (Christian Heimes)
Date: Mon, 23 Jan 2017 16:35:46 +0000
Subject: [issue29136] Add OP_NO_TLSv1_3
In-Reply-To: <1483391244.76.0.29431388133.issue29136@psf.upfronthosting.co.za>
Message-ID: <1485189346.04.0.441871667088.issue29136@psf.upfronthosting.co.za>


Christian Heimes added the comment:

No, of course it does not work (yet):

$ LD_LIBRARY_PATH=. apps/openssl ciphers TLS13
Error in cipher list
140546693477888:error:1410D0B9:SSL routines:SSL_CTX_set_cipher_list:no cipher match:ssl/ssl_lib.c:2100:

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29136>
_______________________________________

From report at bugs.python.org  Mon Jan 23 11:47:54 2017
From: report at bugs.python.org (Markus Gerstel)
Date: Mon, 23 Jan 2017 16:47:54 +0000
Subject: [issue29351] absolute imports for logging
In-Reply-To: <1485177903.18.0.43823292649.issue29351@psf.upfronthosting.co.za>
Message-ID: <1485190074.47.0.778554119551.issue29351@psf.upfronthosting.co.za>


Markus Gerstel added the comment:

Yes, this is indeed the same for other stdlib modules, too. Logging is just the first one that came to attention in our investigations.

I haven't prepared any other patches yet though, because your answer could easily be "No, we cannot consider these changes under any circumstances for 2.7 because $reason".

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29351>
_______________________________________

From report at bugs.python.org  Mon Jan 23 11:49:16 2017
From: report at bugs.python.org (Akira Li)
Date: Mon, 23 Jan 2017 16:49:16 +0000
Subject: [issue29352] provide the authorative source for s[i:j] negative slice
 indices (<-len(s)) behavior for standard sequences
Message-ID: <1485190156.24.0.631194950251.issue29352@psf.upfronthosting.co.za>


New submission from Akira Li:

I've failed to find where the behavior for negative indices in s[i:j]
expression (i, j < -len(s)) for standard sequences (str, list, etc) is
formally defined.

The observed behavior implemented in PySlice_GetIndicesEx(): If "len(s)
+ i" or "len(s) + j" is negative, use 0. [1] I don't see it in the docs.

        if (*start < 0) *start += length;
        if (*start < 0) *start = (*step < 0) ? -1 : 0;
        ...
        if (*stop < 0) *stop += length;
        if (*stop < 0) *stop = (*step < 0) ? -1 : 0;

The tutorial mentions [2]:

> out of range slice indexes are handled gracefully when used for
> slicing"

slice.indices() documentation says [3]:

> Missing or out-of-bounds indices are handled in a manner consistent
> with regular slices.

Neither define it explicitly.

The behavior for the upper boundary is defined explicitly [4]:

> If *i* or *j* is greater than ``len(s)``, use ``len(s)``

I've added the documentation patch that defines the behavior for the
lower boundary too.

[1] Objects/sliceobject.c
[2] Doc/tutorial/introduction.rst
[3] Doc/reference/datamodel.rst
[4] Doc/library/stdtypes.rst

----------
assignee: docs at python
components: Documentation
files: docs-negative-slice-indices.patch
keywords: patch
messages: 286098
nosy: akira, docs at python
priority: normal
severity: normal
status: open
title: provide the authorative source for s[i:j] negative slice indices (<-len(s)) behavior for standard sequences
versions: Python 3.5, Python 3.6, Python 3.7
Added file: http://bugs.python.org/file46393/docs-negative-slice-indices.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29352>
_______________________________________

From report at bugs.python.org  Mon Jan 23 11:52:20 2017
From: report at bugs.python.org (Zach Riggle)
Date: Mon, 23 Jan 2017 16:52:20 +0000
Subject: [issue29335] subprocess module does not check WIFSTOPPED on SIGCHLD
In-Reply-To: <1484945288.47.0.935636853008.issue29335@psf.upfronthosting.co.za>
Message-ID: <1485190340.8.0.741533834585.issue29335@psf.upfronthosting.co.za>


Zach Riggle added the comment:

Of note, there's no need to actually cause a SIGSEGV to generate the signal.

The tests might be more clear to replace:

    libc.printf(ctypes.c_char_p(0xdeadbeef))

with

    os.kill(os.getpid(), signal.SIGSEGV)

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29335>
_______________________________________

From report at bugs.python.org  Mon Jan 23 11:56:47 2017
From: report at bugs.python.org (STINNER Victor)
Date: Mon, 23 Jan 2017 16:56:47 +0000
Subject: [issue29335] subprocess module does not check WIFSTOPPED on SIGCHLD
In-Reply-To: <1484945288.47.0.935636853008.issue29335@psf.upfronthosting.co.za>
Message-ID: <1485190607.34.0.818881350505.issue29335@psf.upfronthosting.co.za>


Changes by STINNER Victor <victor.stinner at gmail.com>:


----------
nosy: +haypo

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29335>
_______________________________________

From report at bugs.python.org  Mon Jan 23 11:57:09 2017
From: report at bugs.python.org (STINNER Victor)
Date: Mon, 23 Jan 2017 16:57:09 +0000
Subject: [issue29335] subprocess module does not check WIFSTOPPED on SIGCHLD
In-Reply-To: <1484945288.47.0.935636853008.issue29335@psf.upfronthosting.co.za>
Message-ID: <1485190629.54.0.444807945983.issue29335@psf.upfronthosting.co.za>


STINNER Victor added the comment:

If you want crashes, look at the portable faulthandler._sigsegv() :-)

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29335>
_______________________________________

From report at bugs.python.org  Mon Jan 23 12:09:18 2017
From: report at bugs.python.org (Yury Selivanov)
Date: Mon, 23 Jan 2017 17:09:18 +0000
Subject: [issue29350] Add support of multiple signatures
In-Reply-To: <1485167165.19.0.665694725089.issue29350@psf.upfronthosting.co.za>
Message-ID: <1485191358.35.0.961129746686.issue29350@psf.upfronthosting.co.za>


Yury Selivanov added the comment:

Signature object provides methods like .bind(), which will be hard to define if a function has many signatures.  Also, inspect.signature currently returns one Signature object, that shouldn't be changed.

Wouldn't it be easier instead of this:
  type(obj)
  type(name, bases, mapping)

do this:
  type(obj_or_name, bases=None, mapping=None)

And explain what really is going on in the docstring?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29350>
_______________________________________

From report at bugs.python.org  Mon Jan 23 12:21:54 2017
From: report at bugs.python.org (Zach Riggle)
Date: Mon, 23 Jan 2017 17:21:54 +0000
Subject: [issue29335] subprocess module does not check WIFSTOPPED on SIGCHLD
In-Reply-To: <1484945288.47.0.935636853008.issue29335@psf.upfronthosting.co.za>
Message-ID: <1485192114.83.0.660129914526.issue29335@psf.upfronthosting.co.za>


Zach Riggle added the comment:

Neat, though that's not in the standard library.

The current logic for getting a handle to libc could also be simplified via ctypes.util.find_library (https://docs.python.org/3/library/ctypes.html#finding-shared-libraries).

Darwin:

    >>> import ctypes.util
    >>> ctypes.util.find_library('c')
    '/usr/lib/libc.dylib'
    
Linux:

    >>> import ctypes.util
    >>> ctypes.util.find_library('c')
    'libc.so.6'

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29335>
_______________________________________

From report at bugs.python.org  Mon Jan 23 12:26:46 2017
From: report at bugs.python.org (Vinay Sajip)
Date: Mon, 23 Jan 2017 17:26:46 +0000
Subject: [issue29351] absolute imports for logging
In-Reply-To: <1485177903.18.0.43823292649.issue29351@psf.upfronthosting.co.za>
Message-ID: <1485192406.82.0.118917299932.issue29351@psf.upfronthosting.co.za>


Vinay Sajip added the comment:

> No, we cannot consider these changes under any circumstances for 2.7 because ...

Well, it would change import behaviour in a backwards-incompatible way, wouldn't it?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29351>
_______________________________________

From report at bugs.python.org  Mon Jan 23 12:51:01 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Mon, 23 Jan 2017 17:51:01 +0000
Subject: [issue29350] Add support of multiple signatures
In-Reply-To: <1485167165.19.0.665694725089.issue29350@psf.upfronthosting.co.za>
Message-ID: <1485193861.48.0.388675050631.issue29350@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

It can be do, but I don't think it is worth. Making bases and mapping optional arguments does not make much sense to me. And there is no a value that can be used as a default value for the second parameter in dict.pop().

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29350>
_______________________________________

From report at bugs.python.org  Mon Jan 23 13:19:44 2017
From: report at bugs.python.org (Ethan Furman)
Date: Mon, 23 Jan 2017 18:19:44 +0000
Subject: [issue29167] Race condition in enum.py:_decompose()
In-Reply-To: <1483613666.47.0.301459916652.issue29167@psf.upfronthosting.co.za>
Message-ID: <1485195584.75.0.506561717906.issue29167@psf.upfronthosting.co.za>


Ethan Furman added the comment:

Fixed the race condition for both the RuntimeError and for getting duplicate composite members.

----------
keywords: +patch
stage: test needed -> patch review
Added file: http://bugs.python.org/file46394/issue29167.stoneleaf.01.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29167>
_______________________________________

From report at bugs.python.org  Mon Jan 23 13:42:50 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Mon, 23 Jan 2017 18:42:50 +0000
Subject: [issue26491] Defer DECREFs until enum object is in a consistent state
 for re-entrancy
In-Reply-To: <1457264726.34.0.955181545994.issue26491@psf.upfronthosting.co.za>
Message-ID: <1485196970.45.0.770120498217.issue26491@psf.upfronthosting.co.za>


Changes by Serhiy Storchaka <storchaka+cpython at gmail.com>:


----------
assignee:  -> rhettinger
status: pending -> open

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue26491>
_______________________________________

From report at bugs.python.org  Mon Jan 23 13:58:25 2017
From: report at bugs.python.org (brokenenglish)
Date: Mon, 23 Jan 2017 18:58:25 +0000
Subject: [issue29353] Incorrect handling of HTTP response with "Content-Type:
 message/rfc822" header
Message-ID: <1485197905.85.0.319696153354.issue29353@psf.upfronthosting.co.za>


New submission from brokenenglish:

Hello.

I found a bug that causes incorrect handling of some values of "Content-Type" header. When you retrieve any URL with "Content-Type: message/rfc822" header, additional payload is added to HTTPMessage. In some cases it causes annoing warnings. Here is a code from packages requests and urllib3 that can cause warning output:
1. Checking and raising excpection in urllib3: https://github.com/shazow/urllib3/blob/0fb5e083b2adf7618db8c26e8e50206de09dd845/urllib3/util/response.py#L61-L66
2. The same code packaged in requests: https://github.com/kennethreitz/requests/blob/362da46e9a46da6e86e1907f03014384ab210151/requests/packages/urllib3/util/response.py#L61-L66
3. Logging the error to output: https://github.com/kennethreitz/requests/blob/362da46e9a46da6e86e1907f03014384ab210151/requests/packages/urllib3/connectionpool.py#L402-L407

The issue arises from the class email.feedparser.FeedParser that handles HTTP and email headers in the same way. So when it handles headers with the message with content-type "message/*" and some other, method _parsegen() tries to parse it like a message with another message (see comments to this condition: https://hg.python.org/cpython/file/tip/Lib/email/feedparser.py#l295). If headers-only argument is set to True, we can avoid some redundant checks on HTTP headers (see this condition: https://hg.python.org/cpython/file/tip/Lib/email/feedparser.py#l244).

I read the comment above (https://hg.python.org/cpython/file/tip/Lib/email/feedparser.py#l241) but I am a newbie and I don't see any other solution to my problem for now. I can rewrite the patch if you show me a better way to fix this issue.

A patch with the unit test is atached.

----------
files: rfc822_httpmessage_payload.patch
keywords: patch
messages: 286106
nosy: brokenenglish
priority: normal
severity: normal
status: open
title: Incorrect handling of HTTP response with "Content-Type: message/rfc822" header
versions: Python 3.3, Python 3.4, Python 3.5, Python 3.6, Python 3.7
Added file: http://bugs.python.org/file46395/rfc822_httpmessage_payload.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29353>
_______________________________________

From report at bugs.python.org  Mon Jan 23 14:03:43 2017
From: report at bugs.python.org (Brett Cannon)
Date: Mon, 23 Jan 2017 19:03:43 +0000
Subject: [issue29337] BytesWarning at compile time
In-Reply-To: <1484982586.78.0.422468246503.issue29337@psf.upfronthosting.co.za>
Message-ID: <1485198223.79.0.321694681368.issue29337@psf.upfronthosting.co.za>


Brett Cannon added the comment:

LGTM

----------
nosy: +brett.cannon

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29337>
_______________________________________

From report at bugs.python.org  Mon Jan 23 14:07:17 2017
From: report at bugs.python.org (Brett Cannon)
Date: Mon, 23 Jan 2017 19:07:17 +0000
Subject: [issue29339] Interactive: Move to same indentation level as previous
 line
In-Reply-To: <1485039375.56.0.473197770315.issue29339@psf.upfronthosting.co.za>
Message-ID: <1485198437.84.0.920212699484.issue29339@psf.upfronthosting.co.za>


Brett Cannon added the comment:

Posting an idea directly to the issue tracker is fine as long as it's done in a reasonable fashion (i.e. not rudely).

As for the specific idea, I assume this is for the REPL as used in a shell and not in IDLE. If you want to propose a patch that can work cross-platform in both a Windows and UNIX shell where dedenting works consistently then it might be considered.

----------
nosy: +brett.cannon

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29339>
_______________________________________

From report at bugs.python.org  Mon Jan 23 14:22:25 2017
From: report at bugs.python.org (12345 67890)
Date: Mon, 23 Jan 2017 19:22:25 +0000
Subject: [issue29339] Interactive: Move to same indentation level as previous
 line
In-Reply-To: <1485039375.56.0.473197770315.issue29339@psf.upfronthosting.co.za>
Message-ID: <1485199345.55.0.121786735523.issue29339@psf.upfronthosting.co.za>


12345 67890 added the comment:

I hope I haven't been rude; the idea here was to see what the feeling of the core developers was about this idea.

As for the patch, that will probably come after the Python workflow has been fixed because doing it then will save everyone time and as I have said, this is low priority.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29339>
_______________________________________

From report at bugs.python.org  Mon Jan 23 14:44:41 2017
From: report at bugs.python.org (R. David Murray)
Date: Mon, 23 Jan 2017 19:44:41 +0000
Subject: [issue29353] Incorrect handling of HTTP response with "Content-Type:
 message/rfc822" header
In-Reply-To: <1485197905.85.0.319696153354.issue29353@psf.upfronthosting.co.za>
Message-ID: <1485200681.49.0.645997038968.issue29353@psf.upfronthosting.co.za>


R. David Murray added the comment:

IMO http *should* be using headersonly=True, so while I haven't looked into this issue the solution seems plausible to me.  I'm not entirely sure why it is considered a backward-compatibility hack, but I don't see any likelyhood that the headersonly API will go way.

----------
nosy: +r.david.murray

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29353>
_______________________________________

From report at bugs.python.org  Mon Jan 23 14:56:52 2017
From: report at bugs.python.org (Brett Cannon)
Date: Mon, 23 Jan 2017 19:56:52 +0000
Subject: [issue29339] Interactive: Move to same indentation level as previous
 line
In-Reply-To: <1485039375.56.0.473197770315.issue29339@psf.upfronthosting.co.za>
Message-ID: <1485201412.74.0.328440217886.issue29339@psf.upfronthosting.co.za>


Brett Cannon added the comment:

No, I don't think you were rude, 12345 67890. I was just giving the standard explanation as some people are not pleasant and you never know who might read this issue later and benefit from the reminder.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29339>
_______________________________________

From report at bugs.python.org  Mon Jan 23 15:32:55 2017
From: report at bugs.python.org (Terry J. Reedy)
Date: Mon, 23 Jan 2017 20:32:55 +0000
Subject: [issue29339] Interactive: Move to same indentation level as previous
 line
In-Reply-To: <1485039375.56.0.473197770315.issue29339@psf.upfronthosting.co.za>
Message-ID: <1485203575.23.0.952536877138.issue29339@psf.upfronthosting.co.za>


Terry J. Reedy added the comment:

Adding at least 'dumb indentation' (repeat last indent) to the REPL would make it at least a bit more usable.  (This still would not be a good as an IDE, like IDLE, that does smarter indentation, such as adding an additional indent after a compound statement header ends with ':'.)

The simplest patch would be to slice and echo the initial whitespace (spaces and tabs) of each line that needs to be followed with a secondary prompt.  What I don't know is whether, on every console, a printed tab always has the same effect as a tab entered by a user.
Testing would be needed.

----------
nosy: +terry.reedy

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29339>
_______________________________________

From report at bugs.python.org  Mon Jan 23 15:47:32 2017
From: report at bugs.python.org (Jim Fasarakis-Hilliard)
Date: Mon, 23 Jan 2017 20:47:32 +0000
Subject: [issue29339] Interactive: Move to same indentation level as previous
 line
In-Reply-To: <1485039375.56.0.473197770315.issue29339@psf.upfronthosting.co.za>
Message-ID: <1485204452.02.0.636836967559.issue29339@psf.upfronthosting.co.za>


Jim Fasarakis-Hilliard added the comment:

Thanks for the standard explanation, Brett. I was just following the devguide too strictly and assumed python-ideas is the first place one should go :-).

As for the idea, it seems others wish/wished it too (first paragraph: https://docs.python.org/3/tutorial/interactive.html#alternatives-to-the-interactive-interpreter)

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29339>
_______________________________________

From report at bugs.python.org  Mon Jan 23 15:56:00 2017
From: report at bugs.python.org (Martin Panter)
Date: Mon, 23 Jan 2017 20:56:00 +0000
Subject: [issue18235] _sysconfigdata.py wrong on AIX installations
In-Reply-To: <1371429201.94.0.372851990361.issue18235@psf.upfronthosting.co.za>
Message-ID: <1485204960.82.0.00844984329765.issue18235@psf.upfronthosting.co.za>


Martin Panter added the comment:

If the assignment is completely removed, won?t this break the test when run from the source or build tree (as opposed to when installed)? Or at least make the situation worse: the AIX buildbot is already failing test_distutils, but at least it is looking for Modules/ld_so_aix locally rather than in the yet-to-be-installed location.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue18235>
_______________________________________

From report at bugs.python.org  Mon Jan 23 16:01:02 2017
From: report at bugs.python.org (David Edelsohn)
Date: Mon, 23 Jan 2017 21:01:02 +0000
Subject: [issue18235] _sysconfigdata.py wrong on AIX installations
In-Reply-To: <1371429201.94.0.372851990361.issue18235@psf.upfronthosting.co.za>
Message-ID: <1485205262.79.0.940367063704.issue18235@psf.upfronthosting.co.za>


David Edelsohn added the comment:

I completely agree with Martin's concern.  As I expressed before, this needs to work in three contexts:

1) Building modules in the tree during the build process.
2) In-tree testing of build module feature (test_distutils).
3) Building and installing modules with an installed version of Python.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue18235>
_______________________________________

From report at bugs.python.org  Mon Jan 23 16:06:16 2017
From: report at bugs.python.org (Saida Dhanavath)
Date: Mon, 23 Jan 2017 21:06:16 +0000
Subject: [issue29347] Python 2.7.8 is crashing while creating weakref for a
 given object.
In-Reply-To: <1485141841.25.0.0993319189306.issue29347@psf.upfronthosting.co.za>
Message-ID: <1485205576.59.0.872693603002.issue29347@psf.upfronthosting.co.za>


Saida Dhanavath added the comment:

Hi Christian Heimes,


PFA. I have written a some code to simulate and test PyWeakReference struct instantion and then hit segfault based on flag passed-in to C code. Here I am trying to execute some of the operations from new_weakref and dealloc_weakref of Objects/weakrefobject.c to show that new isntance of PyWeakReference is not initialized properly. Have also checked latest 3.6 source and there is no difference in alloc and dealloc routines of 2.7.8 and 3.6.0. Have run test code on 2.7.8, 2.7.12+, 3.4m and 3.5m interpreters and got segfault in all runs.

----------
Added file: http://bugs.python.org/file46396/crash_in_weakref.zip

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29347>
_______________________________________

From report at bugs.python.org  Mon Jan 23 16:12:35 2017
From: report at bugs.python.org (Martin Panter)
Date: Mon, 23 Jan 2017 21:12:35 +0000
Subject: [issue29152] unittest subTest does not call addFailure
In-Reply-To: <1483534655.28.0.0626841658388.issue29152@psf.upfronthosting.co.za>
Message-ID: <1485205955.42.0.529215464194.issue29152@psf.upfronthosting.co.za>


Martin Panter added the comment:

It is not obvious what the effect of not calling addFailure() is, but perhaps this is related to Issue 25894? Failure and error statuses are not immediately reported from subtests.

----------
nosy: +martin.panter

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29152>
_______________________________________

From report at bugs.python.org  Mon Jan 23 16:30:20 2017
From: report at bugs.python.org (Terry J. Reedy)
Date: Mon, 23 Jan 2017 21:30:20 +0000
Subject: [issue29339] Interactive: Move to same indentation level as previous
 line
In-Reply-To: <1485039375.56.0.473197770315.issue29339@psf.upfronthosting.co.za>
Message-ID: <1485207020.36.0.675331322453.issue29339@psf.upfronthosting.co.za>


Terry J. Reedy added the comment:

The point I was trying to get at above is that simply printing a prompt and making the use enter the entire line should work on every console, while anything fancier may not be so reliable.  Thinking more, I realize that my patch outline is incomplete. After 'line = input(prompt + indent)', line will not include the indent. A printed indent will have to be added to the input received from the user.  (In IDLE's Shell and editors, the indents that IDLE insert()s into a text widget are indistinguishable from those types by a user and *are* included in the user input that IDLE reads.)  An associated issue is that cross-platform automated tests would be difficult to impossible.

More experiment reveals the fatal problem: in REPL mode, python reads stdin and writes to stdout and stderr.  In the Windows console, and I am sure others, printed output cannot be deleted.  In particular, printed input spaces, such as the one at the end of '>>> ' cannot be deleted.  So dedenting, as in the following example, would not be possible.  One cannot input() a negative string.

if possible:
    print('I like it')
    write_patch()
    test()
else:
    print('too bad')
    reject_idea()

So unless I am wrong, the idea must be rejected.

----------
stage:  -> needs patch
versions: +Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29339>
_______________________________________

From report at bugs.python.org  Mon Jan 23 16:50:38 2017
From: report at bugs.python.org (Elias Vanderstuyft)
Date: Mon, 23 Jan 2017 21:50:38 +0000
Subject: [issue29354] Python 2.7.12: pydoc.help(lambda (a,
 ): lambda x: a) raises IndexError
Message-ID: <1485208238.9.0.161996393795.issue29354@psf.upfronthosting.co.za>


New submission from Elias Vanderstuyft:

The following might be either a bug in the pydoc module or in the inspect module:

$ python
Python 2.7.12 (default, Nov 19 2016, 06:48:10) 
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import pydoc
>>> pydoc.help(lambda (a,): lambda x: a)    # this fails
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/pydoc.py", line 1797, in __call__
    self.help(request)
  File "/usr/lib/python2.7/pydoc.py", line 1844, in help
    else: doc(request, 'Help on %s:')
  File "/usr/lib/python2.7/pydoc.py", line 1581, in doc
    pager(render_doc(thing, title, forceload))
  File "/usr/lib/python2.7/pydoc.py", line 1576, in render_doc
    return title % desc + '\n\n' + text.document(object, name)
  File "/usr/lib/python2.7/pydoc.py", line 363, in document
    if inspect.isroutine(object): return self.docroutine(*args)
  File "/usr/lib/python2.7/pydoc.py", line 1321, in docroutine
    args, varargs, varkw, defaults = inspect.getargspec(object)
  File "/usr/lib/python2.7/inspect.py", line 817, in getargspec
    args, varargs, varkw = getargs(func.func_code)
  File "/usr/lib/python2.7/inspect.py", line 791, in getargs
    args[i] = stack[0]
IndexError: list index out of range
>>> pydoc.help(lambda (a,): lambda x: x)    # this succeeds

>>>

----------
components: Library (Lib)
messages: 286119
nosy: Elias Vanderstuyft
priority: normal
severity: normal
status: open
title: Python 2.7.12: pydoc.help(lambda (a,): lambda x: a) raises IndexError
type: behavior
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29354>
_______________________________________

From report at bugs.python.org  Mon Jan 23 16:56:27 2017
From: report at bugs.python.org (Michael Felt)
Date: Mon, 23 Jan 2017 21:56:27 +0000
Subject: [issue18235] _sysconfigdata.py wrong on AIX installations
In-Reply-To: <1371429201.94.0.372851990361.issue18235@psf.upfronthosting.co.za>
Message-ID: <1485208587.88.0.424877988805.issue18235@psf.upfronthosting.co.za>


Michael Felt added the comment:

You guys are the experts. I can only comment on what I see. IMHO: the file _sysconfigdata.py is more accurate with nothing in it.

I am clearly confused by whatever process this is. If you believe it is more accurate to have the BLD variable 'inaccurate' in this file - and that we just need to "believe" that it has the correct value during the build (which, starting with version 2.7.13 seems to be the case, as I am able to build in a different directory - and could not do this with version 2.7.10 (first time I tried using separate directories).

In short, I have no expert opinion on this. My naive view is that both variables being accurate in the file is more accurate. In any case, the patch is needed for a working _sysconfigdata.py and integration with something such as "pip build ...". Only after I learned about 'pip' did I ever run into this, i.e., only people who are trying to install something extra have issues here. I have been 'patching' _sysconfigdata.py manually so that things work normally.

I hope my feedback helps you move forward on this.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue18235>
_______________________________________

From report at bugs.python.org  Mon Jan 23 17:03:48 2017
From: report at bugs.python.org (=?utf-8?q?Charles-Fran=C3=A7ois_Natali?=)
Date: Mon, 23 Jan 2017 22:03:48 +0000
Subject: [issue29343] sock.close() raises OSError EBADF when socket's fd is
 closed
In-Reply-To: <1485092861.97.0.0667849204017.issue29343@psf.upfronthosting.co.za>
Message-ID: <CAH_1eM11j-cbRjQB-CGhX1kvrrM4-QvNUnwwu-WFJQK_YgXk2Q@mail.gmail.com>


Charles-Fran?ois Natali added the comment:

FWIW I agree with Antoine and Martin: ignoring EBADF is a bad idea,
quite dangerous.
The man page probably says this to highlight that users shouldn't
*retry* close():
"""
       Retrying the close() after a failure return is the wrong thing to do,
       since this may cause a reused file descriptor from another thread to
       be closed.  This can occur because the Linux kernel always releases
       the file descriptor early in the close operation, freeing it for
       reuse; the steps that may return an error, such as flushing data to
       the filesystem or device, occur only later in the close operation.

"""

(Including on EINTR).

In short, I think that the change is a good idea: Socket.close() is
guaranteed to be idempotent, so this error can only happen in case of
invalid usage.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29343>
_______________________________________

From report at bugs.python.org  Mon Jan 23 17:26:51 2017
From: report at bugs.python.org (Aviv Palivoda)
Date: Mon, 23 Jan 2017 22:26:51 +0000
Subject: [issue28518] execute("begin immediate") throwing OperationalError
In-Reply-To: <1477307118.31.0.70785963389.issue28518@psf.upfronthosting.co.za>
Message-ID: <1485210411.3.0.843676949129.issue28518@psf.upfronthosting.co.za>


Aviv Palivoda added the comment:

Removed opening a transaction on select. I will argue for that in issue 9924 after this is resolved.

----------
Added file: http://bugs.python.org/file46397/sqlite-ddl-dml-3.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28518>
_______________________________________

From report at bugs.python.org  Mon Jan 23 17:31:00 2017
From: report at bugs.python.org (Aviv Palivoda)
Date: Mon, 23 Jan 2017 22:31:00 +0000
Subject: [issue29228] sqlite3 OperationalError on changing into WAL
 transaction mode
In-Reply-To: <1484072366.39.0.529572648914.issue29228@psf.upfronthosting.co.za>
Message-ID: <1485210660.72.0.112876607906.issue29228@psf.upfronthosting.co.za>


Aviv Palivoda added the comment:

There is also a issue with the docs here. Look at issue 29121.

----------
nosy: +palaviv

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29228>
_______________________________________

From report at bugs.python.org  Mon Jan 23 17:41:21 2017
From: report at bugs.python.org (12345 67890)
Date: Mon, 23 Jan 2017 22:41:21 +0000
Subject: [issue29339] Interactive: Move to same indentation level as previous
 line
In-Reply-To: <1485039375.56.0.473197770315.issue29339@psf.upfronthosting.co.za>
Message-ID: <1485211281.67.0.522170760727.issue29339@psf.upfronthosting.co.za>


12345 67890 added the comment:

It probably won't be trivial, but it definitely is possible. Suplemon has the exact functionality needed and it's only dependency is CPython. The only question is: how much code is this going to add?

[1]: https://github.com/richrd/suplemon

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29339>
_______________________________________

From report at bugs.python.org  Mon Jan 23 17:46:19 2017
From: report at bugs.python.org (12345 67890)
Date: Mon, 23 Jan 2017 22:46:19 +0000
Subject: [issue29339] Interactive: Move to same indentation level as previous
 line
In-Reply-To: <1485039375.56.0.473197770315.issue29339@psf.upfronthosting.co.za>
Message-ID: <1485211579.09.0.201553162865.issue29339@psf.upfronthosting.co.za>


12345 67890 added the comment:

In addition, windows allows direct console control via the win32 API and the code for that isn't too lengthy.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29339>
_______________________________________

From report at bugs.python.org  Mon Jan 23 18:56:05 2017
From: report at bugs.python.org (Martin Panter)
Date: Mon, 23 Jan 2017 23:56:05 +0000
Subject: [issue29353] Incorrect handling of HTTP response with "Content-Type:
 message/rfc822" header
In-Reply-To: <1485197905.85.0.319696153354.issue29353@psf.upfronthosting.co.za>
Message-ID: <1485215765.76.0.806298331348.issue29353@psf.upfronthosting.co.za>


Martin Panter added the comment:

There is an inconsistency when parsing with headersonly=True. According to the documentation, get_payload() with message/rfc822 should should return a list of Message objects, not a string. But using headersonly=True produces a non-multipart Message object:

>>> m = Parser().parsestr("Content-Type: message/rfc822\r\n\r\n", headersonly=True)
>>> m.get_content_type()
'message/rfc822'
>>> m.is_multipart()  # Doc says True
False
>>> m.get_payload()  # Doc says list of Message objects
''

Related to this, setting headersonly=True can also cause a internal inconsistency. Maybe this is why it was called a ?hack?:

>>> Parser().parsestr("Content-Type: message/delivery-status\r\nInvalid line\r\n\r\n", headersonly=True).as_string()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.5/email/message.py", line 159, in as_string
    g.flatten(self, unixfrom=unixfrom)
  File "/usr/lib/python3.5/email/generator.py", line 115, in flatten
    self._write(msg)
  File "/usr/lib/python3.5/email/generator.py", line 181, in _write
    self._dispatch(msg)
  File "/usr/lib/python3.5/email/generator.py", line 214, in _dispatch
    meth(msg)
  File "/usr/lib/python3.5/email/generator.py", line 331, in _handle_message_delivery_status
    g.flatten(part, unixfrom=False, linesep=self._NL)
  File "/usr/lib/python3.5/email/generator.py", line 106, in flatten
    old_msg_policy = msg.policy
AttributeError: 'str' object has no attribute 'policy'

I think it may be best only change get_payload() to return a string in the next Python version (3.7), with appropriate documentation updates. For existing Python versions, perhaps urllib3 could check if the list returned by get_payload() only has trivial empty Message objects (no header fields and only empty payloads themselves).

If we agree that only a feature change for 3.7 is appropriate, there are other problems with the current parsing of HTTP headers that could also be looked at:

* Only a blank line should end a header section (Issue 24363, Issue 26686)
* ?From? line should be a defect
* Use ?email? package?s HTTP parsing policy
* Don?t assume Latin-1 encoding (Issue 27716)
* Avoid double-handling (header lines are parsed in http.client, then joined together and parsed again in email.feedparser)

----------
components: +email
nosy: +barry, martin.panter
versions:  -Python 3.3, Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29353>
_______________________________________

From report at bugs.python.org  Mon Jan 23 19:49:45 2017
From: report at bugs.python.org (Martin Panter)
Date: Tue, 24 Jan 2017 00:49:45 +0000
Subject: [issue21272] use _sysconfigdata to itinialize distutils.sysconfig
In-Reply-To: <1397686328.42.0.129070062942.issue21272@psf.upfronthosting.co.za>
Message-ID: <1485218985.15.0.00687616276878.issue21272@psf.upfronthosting.co.za>


Changes by Martin Panter <vadmium+py at gmail.com>:


----------
versions: +Python 3.6 -Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue21272>
_______________________________________

From report at bugs.python.org  Mon Jan 23 21:45:08 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Tue, 24 Jan 2017 02:45:08 +0000
Subject: [issue29338] Output the text signature in the help of a class
In-Reply-To: <1484991756.99.0.970269270756.issue29338@psf.upfronthosting.co.za>
Message-ID: <1485225908.45.0.902291239728.issue29338@psf.upfronthosting.co.za>


Xiang Zhang added the comment:

Things doesn't come to an end. :-( The enum test suite also gets a related test and make the buildbot fail:

FAIL: test_pydoc (test.test_enum.TestStdLib)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\buildarea\3.x.bolen-windows10\build\lib\test\test_enum.py", line 2409, in test_pydoc
    self.assertEqual(result, expected_text)
AssertionError: 'Help[68 chars]n |  Color(value, names=None, *, module=None, [896 chars]ing.' != 'Help[68 chars]n |  An enumeration.\n |  \n |  Method resolut[809 chars]ing.'
  Help on class Color in module test.test_enum:
  
  class Color(enum.Enum)
-  |  Color(value, names=None, *, module=None, qualname=None, type=None, start=1)
-  |  
   |  An enumeration.
   |  
   |  Method resolution order:
   |      Color
   |      enum.Enum
   |      builtins.object
   |  
   |  Data and other attributes defined here:
   |  
   |  blue = <Color.blue: 3>
   |  
   |  green = <Color.green: 2>
   |  
   |  red = <Color.red: 1>
   |  
   |  ----------------------------------------------------------------------
   |  Data descriptors inherited from enum.Enum:
   |  
   |  name
   |      The name of the Enum member.
   |  
   |  value
   |      The value of the Enum member.
   |  
   |  ----------------------------------------------------------------------
   |  Data descriptors inherited from enum.EnumMeta:
   |  
   |  __members__
   |      Returns a mapping of member name->value.
   |      
   |      This mapping lists all enum members, including aliases. Note that this
   |      is a read-only view of the internal mapping.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29338>
_______________________________________

From report at bugs.python.org  Mon Jan 23 22:26:17 2017
From: report at bugs.python.org (Martin Panter)
Date: Tue, 24 Jan 2017 03:26:17 +0000
Subject: [issue12067] Doc: remove errors about mixed-type comparisons.
In-Reply-To: <1305237573.86.0.646542413513.issue12067@psf.upfronthosting.co.za>
Message-ID: <1485228377.51.0.237127263201.issue12067@psf.upfronthosting.co.za>


Martin Panter added the comment:

Updated patch for 2.7, which I plan to commit soon. Corresponding Py 3 patch coming soon.

----------
Added file: http://bugs.python.org/file46398/expressions-py2.7_v17.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue12067>
_______________________________________

From report at bugs.python.org  Mon Jan 23 22:45:07 2017
From: report at bugs.python.org (Emily Morehouse)
Date: Tue, 24 Jan 2017 03:45:07 +0000
Subject: [issue29354] Python 2.7.12: pydoc.help(lambda (a,
 ): lambda x: a) raises IndexError
In-Reply-To: <1485208238.9.0.161996393795.issue29354@psf.upfronthosting.co.za>
Message-ID: <1485229507.35.0.5145413916.issue29354@psf.upfronthosting.co.za>


Emily Morehouse added the comment:

Aspiring contributor here!

I believe I have narrowed this error down to the inspect module, specifically in the for loop beginning on line 759. 

There's a bit of "acrobatics" (to quote the code comments) that I'm trying to sift through, I've been using the getargspec method from the inspect module as well as print statements in the inspect module to narrow down the error. It definitely doesn't seem like the previously mentioned for loop is functioning properly, as the first item in the stack is being referenced when it is empty. Additionally, I'm utilizing the dis module to help make sense of the full code object to compare against the behavior in the for loop.

Yury, I've respectfully included you here as you're listed as the expert for the inspect module. I would really like to submit a patch for this, but may need a nudge in the right direction. I will update here with my progress.

----------
nosy: +emilyemorehouse, yselivanov

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29354>
_______________________________________

From report at bugs.python.org  Mon Jan 23 23:06:28 2017
From: report at bugs.python.org (Martin Panter)
Date: Tue, 24 Jan 2017 04:06:28 +0000
Subject: [issue12067] Doc: remove errors about mixed-type comparisons.
In-Reply-To: <1305237573.86.0.646542413513.issue12067@psf.upfronthosting.co.za>
Message-ID: <1485230788.6.0.55001417113.issue12067@psf.upfronthosting.co.za>


Changes by Martin Panter <vadmium+py at gmail.com>:


Added file: http://bugs.python.org/file46399/expressions-py3.7_v17.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue12067>
_______________________________________

From report at bugs.python.org  Mon Jan 23 23:38:13 2017
From: report at bugs.python.org (Ma Lin)
Date: Tue, 24 Jan 2017 04:38:13 +0000
Subject: [issue29355] sqlite3: remove sqlite3_stmt_readonly()
Message-ID: <1485232693.83.0.291296040478.issue29355@psf.upfronthosting.co.za>


New submission from Ma Lin:

In CPython 3.6.0, sqlite3 module uses sqlite3_stmt_readonly(), however this function is only available on SQLite 3.7.4+ (release on 2010-12-07).
Then CPython 3.6.0 can not be compiled on RHEL6 (with SQLite 3.6.x), complaints: issue29098, issue29325.

sqlite3_stmt_readonly() was introduced in 284676cf2ac8, it was used twice [1][2], but it seems that we can avoid using this function in both of them.

[1] https://hg.python.org/cpython/file/3.6/Modules/_sqlite/cursor.c#l517
With palaviv's patches of issue28518, we can eliminate sqlite3_stmt_readonly() in here.

[2] https://hg.python.org/cpython/file/3.6/Modules/_sqlite/cursor.c#l612
We can use the old way in here, the old way is even better IMO.

This issue depends on issue28518.

----------
components: Library (Lib)
messages: 286130
nosy: Ma Lin, berker.peksag, palaviv
priority: normal
severity: normal
status: open
title: sqlite3: remove sqlite3_stmt_readonly()
type: enhancement
versions: Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29355>
_______________________________________

From report at bugs.python.org  Mon Jan 23 23:38:37 2017
From: report at bugs.python.org (Yury Selivanov)
Date: Tue, 24 Jan 2017 04:38:37 +0000
Subject: [issue29354] Python 2.7.12: pydoc.help(lambda (a,
 ): lambda x: a) raises IndexError
In-Reply-To: <1485208238.9.0.161996393795.issue29354@psf.upfronthosting.co.za>
Message-ID: <1485232717.32.0.880816366258.issue29354@psf.upfronthosting.co.za>


Yury Selivanov added the comment:

Hi Emily!  Python 2.7 is in a maintenance-only mode at this point and mostly receives security-related patches.  The problem with accepting other kind of patches is that we want 2.7 to be super stable.

You can continue working on this bug anyways, once the patch is ready we'll have to ask Benjamin Peterson (Python 2.7 release manager) to review it.

I suspect the bug is that inspect.getargspec doesn't fully support tuple arguments unpacking (it's a rather obscure feature that we removed from Python 3).

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29354>
_______________________________________

From report at bugs.python.org  Mon Jan 23 23:45:29 2017
From: report at bugs.python.org (Yury Selivanov)
Date: Tue, 24 Jan 2017 04:45:29 +0000
Subject: [issue29343] sock.close() raises OSError EBADF when socket's fd is
 closed
In-Reply-To: <1485087117.22.0.587639771074.issue29343@psf.upfronthosting.co.za>
Message-ID: <1485233129.4.0.379575677363.issue29343@psf.upfronthosting.co.za>


Yury Selivanov added the comment:

Christian, how did you find this (i.e. what module/package is broken because of this)?  I'm still not entirely sure that raising EBADF in socket.close() is a good thing.

----------
nosy: +yselivanov

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29343>
_______________________________________

From report at bugs.python.org  Mon Jan 23 23:50:14 2017
From: report at bugs.python.org (Saida Dhanavath)
Date: Tue, 24 Jan 2017 04:50:14 +0000
Subject: [issue29347] Python 2.7.8 is crashing while creating weakref for a
 given object.
In-Reply-To: <1485141841.25.0.0993319189306.issue29347@psf.upfronthosting.co.za>
Message-ID: <1485233414.46.0.354654502983.issue29347@psf.upfronthosting.co.za>


Saida Dhanavath added the comment:

Hi Christian Heimes,


Please ignore typos in the previous post. I have written some code to simulate and test PyWeakReference struct instantiation and then hit segfault based on the flag passed to C code. Here I am trying to execute some of the operations from new_weakref and dealloc_weakref of Objects/weakrefobject.c to show that new instance of PyWeakReference is not initialized properly and results in segfault. Have also checked python 3.6.0 source and I do not see any change in weakref alloc and dealloc routines of 2.7.8 and 3.6.0 versions. Have run test code on 2.7.8, 2.7.12+, 3.4m and 3.5m interpreters and got segfault in all runs.


Please find the sample output as given below.

ubuntu at ubuntu1610saida:~/weakref$ make build PYVERSION=2.7
swig -python weakref_crash.i
gcc -w -fpic -c weakref_crash.c weakref_crash_wrap.c -I/usr/include/python2.7
gcc -shared weakref_crash.o weakref_crash_wrap.o -o _pyweakref_crash.so -L/usr/lib/python2.7 -lpython2.7

ubuntu at ubuntu1610saida:~/weakref$ python test.py 0 0
2.7.12+ (default, Sep 17 2016, 12:08:02) 
[GCC 6.2.0 20160914]
0
0

ubuntu at ubuntu1610saida:~/weakref$ python test.py 0 1
2.7.12+ (default, Sep 17 2016, 12:08:02) 
[GCC 6.2.0 20160914]
0
0

ubuntu at ubuntu1610saida:~/weakref$ python test.py 1 0
2.7.12+ (default, Sep 17 2016, 12:08:02) 
[GCC 6.2.0 20160914]
1010101
1010101

ubuntu at ubuntu1610saida:~/weakref$ python test.py 1 1
2.7.12+ (default, Sep 17 2016, 12:08:02) 
[GCC 6.2.0 20160914]
1010101
1010101
Segmentation fault (core dumped)

ubuntu at ubuntu1610saida:~/weakref$ make clean
rm -f *.so *.o *crash.py *.pyc *crash_wrap.c

ubuntu at ubuntu1610saida:~/weakref$ make build PYVERSION=3.5m
swig -python weakref_crash.i
gcc -w -fpic -c weakref_crash.c weakref_crash_wrap.c -I/usr/include/python3.5m
gcc -shared weakref_crash.o weakref_crash_wrap.o -o _pyweakref_crash.so -L/usr/lib/python3.5m -lpython3.5m
ubuntu at ubuntu1610saida:~/weakref$ python3.5m test.py 0 0
3.5.2+ (default, Sep 22 2016, 12:18:14) 
[GCC 6.2.0 20160927]
0
0
ubuntu at ubuntu1610saida:~/weakref$ python3.5m test.py 0 1
3.5.2+ (default, Sep 22 2016, 12:18:14) 
[GCC 6.2.0 20160927]
0
0
ubuntu at ubuntu1610saida:~/weakref$ python3.5m test.py 1 0
3.5.2+ (default, Sep 22 2016, 12:18:14) 
[GCC 6.2.0 20160927]
1010101
101
ubuntu at ubuntu1610saida:~/weakref$ python3.5m test.py 1 1
3.5.2+ (default, Sep 22 2016, 12:18:14) 
[GCC 6.2.0 20160927]
1010101
101
Segmentation fault (core dumped)

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29347>
_______________________________________

From report at bugs.python.org  Mon Jan 23 23:52:52 2017
From: report at bugs.python.org (=?utf-8?q?=C3=81d=C3=A1m_Szieberth?=)
Date: Tue, 24 Jan 2017 04:52:52 +0000
Subject: [issue29356] TypeError by typing for Ellipsis
Message-ID: <1485233572.17.0.570161775222.issue29356@psf.upfronthosting.co.za>


New submission from ?d?m Szieberth:

I would expect Ellipsis the same behavior as None in type hints instead of:

TypeError: Union[arg, ...]: each arg must be a type. Got Ellipsis.

... would be much nicer than type(...) there.

----------
messages: 286134
nosy: ?d?m Szieberth
priority: normal
severity: normal
status: open
title: TypeError by typing for Ellipsis
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29356>
_______________________________________

From report at bugs.python.org  Mon Jan 23 23:57:15 2017
From: report at bugs.python.org (Roundup Robot)
Date: Tue, 24 Jan 2017 04:57:15 +0000
Subject: [issue29273] test___all__ alters utf8 locale setting
In-Reply-To: <1484378754.56.0.202012710496.issue29273@psf.upfronthosting.co.za>
Message-ID: <20170124045711.19405.68028.43B2E727@psf.io>


Roundup Robot added the comment:

New changeset 154757703c98 by Martin Panter in branch '3.5':
Issue #29273: Remove unneeded workaround to restore locale
https://hg.python.org/cpython/rev/154757703c98

New changeset 146f7ec82f93 by Martin Panter in branch '3.6':
Issues #29273: Merge test___all__ from 3.5
https://hg.python.org/cpython/rev/146f7ec82f93

New changeset 1615898492a6 by Martin Panter in branch 'default':
Issues #29273: Merge test___all__ from 3.6
https://hg.python.org/cpython/rev/1615898492a6

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29273>
_______________________________________

From report at bugs.python.org  Tue Jan 24 00:02:41 2017
From: report at bugs.python.org (Martin Panter)
Date: Tue, 24 Jan 2017 05:02:41 +0000
Subject: [issue29349] Update old Python 2 code in
 Docs/tools/extensions/patchlevel.py
In-Reply-To: <1485164481.25.0.457345760711.issue29349@psf.upfronthosting.co.za>
Message-ID: <1485234161.88.0.776298433417.issue29349@psf.upfronthosting.co.za>


Martin Panter added the comment:

Is it okay to only fix this in 3.5+? 3.4 only gets security fixes now.

Either way, the ?with? statement changes is not a bug fix and should only go into 3.7.

----------
nosy: +martin.panter
stage:  -> commit review

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29349>
_______________________________________

From report at bugs.python.org  Tue Jan 24 00:13:07 2017
From: report at bugs.python.org (Martin Panter)
Date: Tue, 24 Jan 2017 05:13:07 +0000
Subject: [issue28556] typing.py upgrades
In-Reply-To: <1477756009.46.0.442279756181.issue28556@psf.upfronthosting.co.za>
Message-ID: <1485234787.54.0.0119629568035.issue28556@psf.upfronthosting.co.za>


Martin Panter added the comment:

ERROR: test_annotation_usage_with_methods (test.test_typing.NamedTupleTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/proj/python/cpython/Lib/test/test_typing.py", line 2058, in test_annotation_usage_with_methods
    self.assertEquals(XMeth(1).double(), 2)
  File "/home/proj/python/cpython/Lib/unittest/case.py", line 1311, in deprecated_func
    DeprecationWarning, 2)
DeprecationWarning: Please use assertEqual instead.

----------
nosy: +martin.panter

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28556>
_______________________________________

From report at bugs.python.org  Tue Jan 24 00:27:42 2017
From: report at bugs.python.org (Martin Panter)
Date: Tue, 24 Jan 2017 05:27:42 +0000
Subject: [issue24363] httplib fails to handle semivalid HTTP headers
In-Reply-To: <1433257563.79.0.772954216205.issue24363@psf.upfronthosting.co.za>
Message-ID: <1485235662.29.0.230124545928.issue24363@psf.upfronthosting.co.za>


Martin Panter added the comment:

Just a minor update with an extra get_payload() test I missed before

----------
versions: +Python 3.7
Added file: http://bugs.python.org/file46400/policy-flag.v2.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24363>
_______________________________________

From report at bugs.python.org  Tue Jan 24 00:40:14 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Tue, 24 Jan 2017 05:40:14 +0000
Subject: [issue29338] Output the text signature in the help of a class
In-Reply-To: <1484991756.99.0.970269270756.issue29338@psf.upfronthosting.co.za>
Message-ID: <1485236414.89.0.546117792422.issue29338@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

It is easy to fix the test by adding missed lines. But I'm not sure that output the (correct) signature of enum classes makes the help better.

    Color(value, names=None, *, module=None, qualname=None, type=None, start=1)

Ethan, what are your thoughts?

----------
nosy: +ethan.furman
resolution: fixed -> 
stage: resolved -> 
status: closed -> open

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29338>
_______________________________________

From report at bugs.python.org  Tue Jan 24 01:26:51 2017
From: report at bugs.python.org (Nick Coghlan)
Date: Tue, 24 Jan 2017 06:26:51 +0000
Subject: [issue29338] Output the text signature in the help of a class
In-Reply-To: <CADiSq7c1gf-DYy9gE_xHmpzTWQVMxzWCY8pxUh=2wRJn8dey_Q@mail.gmail.com>
Message-ID: <CADiSq7crRophAFxwX6yhMw6g5_rgZ=0X1STpWpuE-Q6AdXBPTg@mail.gmail.com>


Nick Coghlan added the comment:

Enum's interaction with the help subsystem has always been somewhat
fragile: http://bugs.python.org/issue18693

In this case, I think a reasonable quick fix would be to add the new text
to the expected output for enum instances (allowing this issue to be closed
again), and then file a separate issue about offering a way to omit that
text for types that discourage or don't allow dynamic creation of new
instances.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29338>
_______________________________________

From report at bugs.python.org  Tue Jan 24 01:26:51 2017
From: report at bugs.python.org (Emily Morehouse)
Date: Tue, 24 Jan 2017 06:26:51 +0000
Subject: [issue29341] Missing accepting path-like object in docstrings of os
 module functions
In-Reply-To: <1485064061.59.0.597221589705.issue29341@psf.upfronthosting.co.za>
Message-ID: <1485239211.2.0.368814326879.issue29341@psf.upfronthosting.co.za>


Emily Morehouse added the comment:

I see that path-like objects are indeed mentioned in the documentation (Doc/library/os.rst), simply stating "Changed in version 3.6: Supports a path-like object." Other methods, such as os.chroot, also mention such a change.

Comparing the docs mentioned above to the docstrings in Modules/clinic/posixmodule.c.h (and Modules/clinic/posixmodule.c for that matter), there's a clear disparity between the detail in the docs vs brevity in the docstrings (specifically in reference to os.chroot). 

Therefore, my question is: how detailed should the docstrings be and how closely should they match Doc/library/os.rst? I can certainly update the docstrings accordingly.

----------
nosy: +emilyemorehouse

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29341>
_______________________________________

From report at bugs.python.org  Tue Jan 24 01:30:04 2017
From: report at bugs.python.org (Roundup Robot)
Date: Tue, 24 Jan 2017 06:30:04 +0000
Subject: [issue29189] Broken indentation in FancyURLopener documentation
In-Reply-To: <1483768195.47.0.481112354487.issue29189@psf.upfronthosting.co.za>
Message-ID: <20170124063000.24986.40394.FD669311@psf.io>


Roundup Robot added the comment:

New changeset fb2885f9b4dd by Martin Panter in branch '2.7':
Issue #29189: Fix indentation in RST markup
https://hg.python.org/cpython/rev/fb2885f9b4dd

New changeset 75e341d79c99 by Martin Panter in branch '3.5':
Issue #29189: Fix indentation in RST markup
https://hg.python.org/cpython/rev/75e341d79c99

New changeset f0854405c830 by Martin Panter in branch '3.6':
Issues #29189: Merge indentation fixes from 3.5
https://hg.python.org/cpython/rev/f0854405c830

New changeset d654b7c63b48 by Martin Panter in branch 'default':
Issues #29189: Merge indentation fixes from 3.6
https://hg.python.org/cpython/rev/d654b7c63b48

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29189>
_______________________________________

From report at bugs.python.org  Tue Jan 24 01:33:10 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Tue, 24 Jan 2017 06:33:10 +0000
Subject: [issue29341] Missing accepting path-like object in docstrings of os
 module functions
In-Reply-To: <1485064061.59.0.597221589705.issue29341@psf.upfronthosting.co.za>
Message-ID: <1485239590.65.0.744446830393.issue29341@psf.upfronthosting.co.za>


Xiang Zhang added the comment:

I don't mean to sync the docstring and the documentation totally. But just like os.chown, there are some functions explicitly mention the acceptable types of path parameter in their docstrings. I think since they already mention the types, then make the list complete. For example, I think for os.chown make the list "can be string, bytes, path-like object or open-file-descriptor int" is enough.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29341>
_______________________________________

From report at bugs.python.org  Tue Jan 24 01:34:09 2017
From: report at bugs.python.org (Emily Morehouse)
Date: Tue, 24 Jan 2017 06:34:09 +0000
Subject: [issue29354] Python 2.7.12: pydoc.help(lambda (a,
 ): lambda x: a) raises IndexError
In-Reply-To: <1485208238.9.0.161996393795.issue29354@psf.upfronthosting.co.za>
Message-ID: <1485239649.62.0.809130960706.issue29354@psf.upfronthosting.co.za>


Emily Morehouse added the comment:

Yury, thanks for the encouragement to continue on this patch. I think it will be a good exercise to dive a bit deeper into Python's bytecode and put some knowledge to use.

I believe that tuple argument unpacking is handled appropriately, but there is an even further edge case where a closure is introduced. In the following, inspect.getargspec works for the first bit of code, but the second fails, as 'a' is referenced outside of scope.

>>> dis.dis(lambda (a,): lambda x: x)
  1           0 LOAD_FAST                0 (.0)
              3 UNPACK_SEQUENCE          1
              6 STORE_FAST               1 (a)
              9 LOAD_CONST               1 (<code object <lambda> at 0x10087a130, file "<stdin>", line 1>)
             12 MAKE_FUNCTION            0
             15 RETURN_VALUE
>>> dis.dis(lambda (a,): lambda x: a)
  1           0 LOAD_FAST                0 (.0)
              3 UNPACK_SEQUENCE          1
              6 STORE_DEREF              0 (a)
              9 LOAD_CLOSURE             0 (a)
             12 BUILD_TUPLE              1
             15 LOAD_CONST               1 (<code object <lambda> at 0x10087a930, file "<stdin>", line 1>)
             18 MAKE_CLOSURE             0
             21 RETURN_VALUE

I'll keep poking at this and see where I get.

-- EM

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29354>
_______________________________________

From report at bugs.python.org  Tue Jan 24 01:34:35 2017
From: report at bugs.python.org (Emily Morehouse)
Date: Tue, 24 Jan 2017 06:34:35 +0000
Subject: [issue29339] Interactive: Move to same indentation level as previous
 line
In-Reply-To: <1485039375.56.0.473197770315.issue29339@psf.upfronthosting.co.za>
Message-ID: <1485239675.88.0.523879866682.issue29339@psf.upfronthosting.co.za>


Changes by Emily Morehouse <emily at cuttlesoft.com>:


----------
nosy: +emilyemorehouse

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29339>
_______________________________________

From report at bugs.python.org  Tue Jan 24 01:49:32 2017
From: report at bugs.python.org (Senthil Kumaran)
Date: Tue, 24 Jan 2017 06:49:32 +0000
Subject: [issue29182] Remove the warning in urllib docs that it doesn't do
 certificate validate by default.
In-Reply-To: <1483722117.81.0.319313681571.issue29182@psf.upfronthosting.co.za>
Message-ID: <1485240572.59.0.7827794696.issue29182@psf.upfronthosting.co.za>


Senthil Kumaran added the comment:

Here is the updated patch that addresses the comment.

----------
Added file: http://bugs.python.org/file46401/issue29182_docs_fix.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29182>
_______________________________________

From report at bugs.python.org  Tue Jan 24 01:49:54 2017
From: report at bugs.python.org (Martin Panter)
Date: Tue, 24 Jan 2017 06:49:54 +0000
Subject: [issue29273] test___all__ alters utf8 locale setting
In-Reply-To: <1484378754.56.0.202012710496.issue29273@psf.upfronthosting.co.za>
Message-ID: <1485240594.92.0.670437881406.issue29273@psf.upfronthosting.co.za>


Changes by Martin Panter <vadmium+py at gmail.com>:


----------
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29273>
_______________________________________

From report at bugs.python.org  Tue Jan 24 02:06:41 2017
From: report at bugs.python.org (Roundup Robot)
Date: Tue, 24 Jan 2017 07:06:41 +0000
Subject: [issue29338] Output the text signature in the help of a class
In-Reply-To: <1484991756.99.0.970269270756.issue29338@psf.upfronthosting.co.za>
Message-ID: <20170124070637.43087.58803.837EAD80@psf.io>


Roundup Robot added the comment:

New changeset b74a6a7d4389 by Serhiy Storchaka in branch 'default':
Issue #29338: Fix test_enum.
https://hg.python.org/cpython/rev/b74a6a7d4389

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29338>
_______________________________________

From report at bugs.python.org  Tue Jan 24 02:09:14 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Tue, 24 Jan 2017 07:09:14 +0000
Subject: [issue29338] Output the text signature in the help of a class
In-Reply-To: <1484991756.99.0.970269270756.issue29338@psf.upfronthosting.co.za>
Message-ID: <1485241754.02.0.277983615681.issue29338@psf.upfronthosting.co.za>


Changes by Serhiy Storchaka <storchaka+cpython at gmail.com>:


----------
resolution:  -> fixed
stage:  -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29338>
_______________________________________

From report at bugs.python.org  Tue Jan 24 02:14:54 2017
From: report at bugs.python.org (Elazar Gershuni)
Date: Tue, 24 Jan 2017 07:14:54 +0000
Subject: [issue29357] New NamedTuple syntax silently ignores method definitions
Message-ID: <1485242094.09.0.593682639878.issue29357@psf.upfronthosting.co.za>


New submission from Elazar Gershuni:

The following does not work as expected:
```
from typing import NamedTuple

class A(NamedTuple):
    a: int

    def __repr__(self):
        return 'some A'

    def spam(self):
        print('spam!')

>>> a = A(5)
>>> repr(a)  # should be 'some A'
'A(a=5)'
>>> a.spam()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'A' object has no attribute 'spam'
```

----------
components: Library (Lib)
messages: 286147
nosy: elazar, gvanrossum, rhettinger
priority: normal
severity: normal
status: open
title: New NamedTuple syntax silently ignores method definitions
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29357>
_______________________________________

From report at bugs.python.org  Tue Jan 24 02:44:14 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Tue, 24 Jan 2017 07:44:14 +0000
Subject: [issue29354] Python 2.7.12: pydoc.help(lambda (a,
 ): lambda x: a) raises IndexError
In-Reply-To: <1485208238.9.0.161996393795.issue29354@psf.upfronthosting.co.za>
Message-ID: <1485243854.46.0.91374338418.issue29354@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

Proposed patch fixes the issue.

----------
keywords: +patch
nosy: +serhiy.storchaka
stage:  -> patch review
Added file: http://bugs.python.org/file46402/inspect_argtuple_unpack_cellvars.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29354>
_______________________________________

From report at bugs.python.org  Tue Jan 24 03:00:29 2017
From: report at bugs.python.org (Ivan Levkivskyi)
Date: Tue, 24 Jan 2017 08:00:29 +0000
Subject: [issue28556] typing.py upgrades
In-Reply-To: <1477756009.46.0.442279756181.issue28556@psf.upfronthosting.co.za>
Message-ID: <1485244829.34.0.890545111263.issue28556@psf.upfronthosting.co.za>


Ivan Levkivskyi added the comment:

Thank you Martin!
Fixed this upstream https://github.com/python/typing/pull/365

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28556>
_______________________________________

From report at bugs.python.org  Tue Jan 24 03:21:07 2017
From: report at bugs.python.org (INADA Naoki)
Date: Tue, 24 Jan 2017 08:21:07 +0000
Subject: [issue29337] BytesWarning at compile time
In-Reply-To: <1484982586.78.0.422468246503.issue29337@psf.upfronthosting.co.za>
Message-ID: <1485246067.59.0.0941619637987.issue29337@psf.upfronthosting.co.za>


INADA Naoki added the comment:

LGTM

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29337>
_______________________________________

From report at bugs.python.org  Tue Jan 24 03:25:29 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 24 Jan 2017 08:25:29 +0000
Subject: [issue29358] Add tp_fastnew and tp_fastinit to PyTypeObject,
 15-20% faster object instanciation
Message-ID: <1485246329.89.0.254567368952.issue29358@psf.upfronthosting.co.za>


New submission from STINNER Victor:

After #29259 "Add tp_fastcall to PyTypeObject: support FASTCALL calling convention for all callable objects", the two last slots which still use the (args: tuple, kwargs: dict) calling convention are tp_new and tp_init, two major slots to instanciate objects.

I implemented tp_fastnew/tp_fastinit on top of the issue #29259 pull request (tp_fastcall). The implementation is a WIP, it's just complete enough to start benchmarking the code.

Example of benchmarks on the two types currently optimized in my WIP fast_init branch, list and _asyncio.Future:
---
haypo at smithers$ ./python -m perf timeit -s 'List=list' 'List()' --duplicate=100 --compare-to=../default-ref/python 
Median +- std dev: [ref] 81.9 ns +- 0.2 ns -> [fast_init] 69.3 ns +- 0.4 ns: 1.18x faster (-15%)

haypo at smithers$ ./python -m perf timeit -s 'List=list' 'List((1,2,3))' --duplicate=100 --compare-to=../default-ref/python 
Median +- std dev: [ref] 137 ns +- 6 ns -> [fast_init] 107 ns +- 0 ns: 1.28x faster (-22%)

haypo at smithers$ ./python -m perf timeit -s 'import _asyncio, asyncio; Future=_asyncio.Future; loop=asyncio.get_event_loop()' 'Future(loop=loop)' --compare-to=../default-ref/python
Median +- std dev: [ref] 411 ns +- 20 ns -> [fast_init] 355 ns +- 18 ns: 1.16x faster (-14%)
---

The speedup of tp_fastnew + tp_fastinit is between 1.16x faster and 1.28x faster. The question is now if it is worth it.

Warning: The code is not fully optimized and is likely to have subtle bugs. The pull request is not ready for a review, but you may take a look if you would like to have an idea of the required changes. The most tricky changes are made in typeobject.c to support backward compatibility (call tp_new if tp_fastnew is not set) and stable API (support Python 3.6 PyTypeObject without the two slots).

Note: tp_fastnew and tp_fastinit slots are the expected end of my large FASTCALL optimization project.

----------
components: Interpreter Core
messages: 286151
nosy: haypo, inada.naoki, rhettinger, serhiy.storchaka, yselivanov
priority: normal
pull_requests: 23
severity: normal
status: open
title: Add tp_fastnew and tp_fastinit to PyTypeObject, 15-20% faster object instanciation
type: performance
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29358>
_______________________________________

From report at bugs.python.org  Tue Jan 24 03:32:59 2017
From: report at bugs.python.org (Aivar Annamaa)
Date: Tue, 24 Jan 2017 08:32:59 +0000
Subject: [issue29359] Deprecate string concatenation without plus
Message-ID: <1485246779.15.0.108891177308.issue29359@psf.upfronthosting.co.za>


New submission from Aivar Annamaa:

How quickly will you notice the bug here:

for sequence in ["Command",
                 "MagicCommand",
                 "Open",
                 "Save",
                 "SaveAs",
                 "NewFile",
                 "EditorTextCreated"
                 "ShellCommand",
                 "ShellInput",
                 "ShowView",
                 "HideView",
                 "TextInsert",
                 "TextDelete",
                 ]:
    do_something(sequence)

? 

Given the title of the issue, its probably not so hard, but usually a list literal is not the first thing to suspect.

Similar problem of missing comma may occur easily with multiline calls.

If concatenation without plus was deprecated, these mistakes would be so easy to notice.

----------
messages: 286152
nosy: Aivar.Annamaa
priority: normal
severity: normal
status: open
title: Deprecate string concatenation without plus
type: enhancement
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29359>
_______________________________________

From report at bugs.python.org  Tue Jan 24 03:37:57 2017
From: report at bugs.python.org (Aivar Annamaa)
Date: Tue, 24 Jan 2017 08:37:57 +0000
Subject: [issue29359] Deprecate string concatenation without plus
In-Reply-To: <1485246779.15.0.108891177308.issue29359@psf.upfronthosting.co.za>
Message-ID: <1485247077.69.0.749274481042.issue29359@psf.upfronthosting.co.za>


Changes by Aivar Annamaa <aivar.annamaa at gmail.com>:


----------
components: +Interpreter Core

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29359>
_______________________________________

From report at bugs.python.org  Tue Jan 24 03:41:54 2017
From: report at bugs.python.org (Ivan Levkivskyi)
Date: Tue, 24 Jan 2017 08:41:54 +0000
Subject: [issue29357] New NamedTuple syntax silently ignores method definitions
In-Reply-To: <1485242094.09.0.593682639878.issue29357@psf.upfronthosting.co.za>
Message-ID: <1485247314.78.0.806205583432.issue29357@psf.upfronthosting.co.za>


Ivan Levkivskyi added the comment:

This has been already reported in https://github.com/python/typing/issues/352 and fixed in https://hg.python.org/cpython/rev/f100619e7137 and https://github.com/python/typing/pull/364

Now adding new methods works but overwriting existing special attributes raises AttributeError:

class A(NamedTuple):
    x: int
    def spam(self):  # this works
        ...
    def _fields(self):  # this is an error (and also for __repr__ etc)

If you think that overwriting all special attributes should be allowed (or only some of them) then we could discuss this at python/typing tracker.

----------
nosy: +levkivskyi
resolution:  -> duplicate
stage:  -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29357>
_______________________________________

From report at bugs.python.org  Tue Jan 24 03:45:01 2017
From: report at bugs.python.org (INADA Naoki)
Date: Tue, 24 Jan 2017 08:45:01 +0000
Subject: [issue29336] merge tuples in module
In-Reply-To: <1484969445.33.0.406694538272.issue29336@psf.upfronthosting.co.za>
Message-ID: <1485247501.07.0.862727283844.issue29336@psf.upfronthosting.co.za>


INADA Naoki added the comment:

merge-constants.patch LGTM

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29336>
_______________________________________

From report at bugs.python.org  Tue Jan 24 03:45:58 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Tue, 24 Jan 2017 08:45:58 +0000
Subject: [issue29359] Deprecate string concatenation without plus
In-Reply-To: <1485246779.15.0.108891177308.issue29359@psf.upfronthosting.co.za>
Message-ID: <1485247558.22.0.104921487372.issue29359@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

See PEP 3126.

----------
nosy: +serhiy.storchaka

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29359>
_______________________________________

From report at bugs.python.org  Tue Jan 24 03:48:29 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Tue, 24 Jan 2017 08:48:29 +0000
Subject: [issue29359] Deprecate string concatenation without plus
In-Reply-To: <1485246779.15.0.108891177308.issue29359@psf.upfronthosting.co.za>
Message-ID: <1485247709.88.0.217148577588.issue29359@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

See also the topic "Implicit string literal concatenation considered harmful?" on Python-Ideas:
https://mail.python.org/pipermail/python-ideas/2013-May/020527.html

If you want to revive this issue, it is better to do this on the Python-Ideas mailing list.

----------
resolution:  -> rejected
stage:  -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29359>
_______________________________________

From report at bugs.python.org  Tue Jan 24 03:49:44 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 24 Jan 2017 08:49:44 +0000
Subject: [issue29360] Don't check if all keys are strings in _PyStack_AsDict()
Message-ID: <1485247784.31.0.540630884546.issue29360@psf.upfronthosting.co.za>


New submission from STINNER Victor:

While running the test suite on issues #29259 (tp_fastcall) and #29358 (tp_fastnew and tp_fastinit), a few test failed on the following assertion of _PyStack_AsDict():

   assert(PyUnicode_CheckExact(key));

For example, test_dict calls dict(**{1: 2}) which must raise a TypeError. I'm not sure who is responsible to check if all keys are strings: _PyStack_AsDict(), PyArg_ParseXXX() or the final function?


The check is already implemented in dict_init(), dict_update_common() calls PyArg_ValidateKeywordArguments():

>>> dict(**{1:2})
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: keyword arguments must be strings


In Python 3.7, PyArg_ParseTupleAndKeywords(), _PyArg_ParseTupleAndKeywordsFast() and _PyArg_ParseStackAndKeywords() and PyArg_ValidateKeywordArguments() raise an error if a key of keyword argumens is not a string:

    if (!PyUnicode_Check(key)) {
        PyErr_SetString(PyExc_TypeError,
                        "keywords must be strings");
        return cleanreturn(0, &freelist);
    }


IMHO the CALL_FUNCTION_EX instruction and the FASTCALL calling convention must not check if all keys are string: the check must only be done in the function (which can be a type constructor like dict_init()), for performance. Almost all functions use one the PyArg_ParseXXX() function. Only very special cases like dict_init() use more specific code to handle keyword arguments.

By the way, _PyObject_FastCallKeywords() already contains the following comment:

    /* kwnames must only contains str strings, no subclass, and all keys must
       be unique: these checks are implemented in Python/ceval.c and
       _PyArg_ParseStackAndKeywords(). */


Note: Technically, I know that it's possible to put non-string keys in a dictionary and in a type dictionary. PyPy (and other Python implementations?) don't support non-string in type dictionary. Do you know use cases where func(**kwargs) must accept non-string keys?


At the end, this long issue is a simple patch replacing an assertion with a comment in _PyStack_AsDict(): see attached patch ;-)

----------
components: Interpreter Core
files: pystack_asdict.patch
keywords: patch
messages: 286157
nosy: haypo
priority: normal
severity: normal
status: open
title: Don't check if all keys are strings in _PyStack_AsDict()
versions: Python 3.7
Added file: http://bugs.python.org/file46403/pystack_asdict.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29360>
_______________________________________

From report at bugs.python.org  Tue Jan 24 03:49:51 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 24 Jan 2017 08:49:51 +0000
Subject: [issue29360] Don't check if all keys are strings in _PyStack_AsDict()
In-Reply-To: <1485247784.31.0.540630884546.issue29360@psf.upfronthosting.co.za>
Message-ID: <1485247791.45.0.998616577541.issue29360@psf.upfronthosting.co.za>


Changes by STINNER Victor <victor.stinner at gmail.com>:


----------
nosy: +inada.naoki, serhiy.storchaka

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29360>
_______________________________________

From report at bugs.python.org  Tue Jan 24 04:02:13 2017
From: report at bugs.python.org (INADA Naoki)
Date: Tue, 24 Jan 2017 09:02:13 +0000
Subject: [issue29360] Don't check if all keys are strings in _PyStack_AsDict()
In-Reply-To: <1485247784.31.0.540630884546.issue29360@psf.upfronthosting.co.za>
Message-ID: <1485248533.74.0.0259235317051.issue29360@psf.upfronthosting.co.za>


INADA Naoki added the comment:

patch LGTM.

Python 2 supported `dict(**{1:3})`.  But I don't know there are any functions
supporting non-string keyword argument.

> PyPy (and other Python implementations?) don't support non-string in type dictionary.

Wow! I thought it's Python's language.
If we can prohibit non string name in all namespace, there might be possible
optimization.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29360>
_______________________________________

From report at bugs.python.org  Tue Jan 24 04:06:31 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Tue, 24 Jan 2017 09:06:31 +0000
Subject: [issue29360] Don't check if all keys are strings in _PyStack_AsDict()
In-Reply-To: <1485247784.31.0.540630884546.issue29360@psf.upfronthosting.co.za>
Message-ID: <1485248791.98.0.982574743002.issue29360@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

The assertion was valid while all keywords did came from a constant keywords tuple in CALL_FUNCTION_KW. But if the FASTCALL protocol is extended for var-keyword arguments, keywords can be arbitrary and the assertion can fail. The assertion on the next line can fail too.

Calling _PyStack_AsDict() with non-string or non-unique keywords means that the var-keyword argument was first unpacked to arrays of keyword names and values and then converted back to a dict. Seems something is done non-efficiently.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29360>
_______________________________________

From report at bugs.python.org  Tue Jan 24 04:17:12 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Tue, 24 Jan 2017 09:17:12 +0000
Subject: [issue29336] merge tuples in module
In-Reply-To: <1484969445.33.0.406694538272.issue29336@psf.upfronthosting.co.za>
Message-ID: <1485249432.78.0.237260073834.issue29336@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

merge-constants.patch is rather a proof of concept. I think it may be more efficient after removing unneeded folded constants (issue28813). But this can be done better with the AST optimizer (issue1346238, issue11549). It would be worth also to merge nested constants. This is similar to interning string constants (see in Objects/codeobject.c).

The benefit is small and we should check that merging constants doesn't have too large cost (CPU time or temporally consumed memory) at compile time.

I think we should first implement the AST optimizer, and then try to combine merging constants with interning string constants.

----------
priority: normal -> low

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29336>
_______________________________________

From report at bugs.python.org  Tue Jan 24 04:24:35 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 24 Jan 2017 09:24:35 +0000
Subject: [issue29358] Add tp_fastnew and tp_fastinit to PyTypeObject,
 15-20% faster object instanciation
In-Reply-To: <1485246329.89.0.254567368952.issue29358@psf.upfronthosting.co.za>
Message-ID: <1485249875.01.0.673117616857.issue29358@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Serhiy Storchaka: "Calling _PyStack_AsDict() with non-string or non-unique keywords means that the var-keyword argument was first unpacked to arrays of keyword names and values and then converted back to a dict. Seems something is done non-efficiently." (msg286159 of the issue #29360)

Python code from test_dict:

   dict(**invalid)

Bytecode:

    LOAD_GLOBAL              1 (dict)
    BUILD_TUPLE              0
    LOAD_FAST                1 (invalid)
    CALL_FUNCTION_EX         1

Call stack:

* _PyEval_EvalFrameDefault()
* do_call_core()
* PyObject_Call()
* _Py_RawFastCallDict() -- conversion from dict to stack+kwnames
* type_call()
* call_init()
* _PyStack_AsDict() -- convertsion from stack+kwnames to dict

Oh right, there are two conversions using a temporary FASTCALL format for (keyword) arguments.

This code was not upstream yet, it comes from the pull request of this issue.

Maybe we need two flavors of type_call(): type_call(args: tuple, kwargs: tuple) if tp_fastinit isn't set, type_fastcall(stack, nargs, kwnames) (FASTCALL) if tp_fastinit is set.

But it seems that the logic should be implemented in PyObject_Call() and _PyObject_FastCallDict(), it cannot be implemented in type_call(), it's too late.

For best performances (avoid any kind of conversion and avoid any temporary object), we should implement something like that.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29358>
_______________________________________

From report at bugs.python.org  Tue Jan 24 04:41:41 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 24 Jan 2017 09:41:41 +0000
Subject: [issue29360] Don't check if all keys are strings in _PyStack_AsDict()
In-Reply-To: <1485247784.31.0.540630884546.issue29360@psf.upfronthosting.co.za>
Message-ID: <1485250901.08.0.174268797972.issue29360@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Serhiy Storchaka: "The assertion was valid while all keywords did came from a constant keywords tuple in CALL_FUNCTION_KW. But if the FASTCALL protocol is extended for var-keyword arguments, keywords can be arbitrary and the assertion can fail. The assertion on the next line can fail too."

Oh right, I didn't notice that. Since it's an issue of the tp_fastnew/tp_fastinit patch, I replied there:
http://bugs.python.org/issue29358#msg286161

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29360>
_______________________________________

From report at bugs.python.org  Tue Jan 24 04:42:35 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 24 Jan 2017 09:42:35 +0000
Subject: [issue29360] Don't check if all keys are strings in _PyStack_AsDict()
In-Reply-To: <1485247784.31.0.540630884546.issue29360@psf.upfronthosting.co.za>
Message-ID: <1485250955.42.0.234625059413.issue29360@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Serhiy Storchaka: "The assertion was valid while all keywords did came from a constant keywords tuple in CALL_FUNCTION_KW. But if the FASTCALL protocol is extended for var-keyword arguments, keywords can be arbitrary and the assertion can fail. The assertion on the next line can fail too."

The second _PyStack_AsDict() assertion is:

   assert(PyDict_GetItem(kwdict, key) == NULL);

This assertion fails if kwnames (tuple) argument of _PyStack_AsDict() contains duplicated keys, or at least two keys which have the same hash value and are equal.

If someone pass kwnames with duplicates keys on purpose (for whatever) reasons, the assertion fails. I'm not sure that the assertion makes sense since it's a feature of Python to be able to replace dictionary values:

>>> def func(**kw): print(kw)
... 
>>> func(**{'x': 1, 'x': 2})
{'x': 2}
>>> {'x': 1, 'x': 2}
{'x': 2}

Are you suggesting to remove "assert(PyDict_GetItem(kwdict, key) == NULL);" too?


Note: In Python 3.6, CALL_FUNCTION_EX already uses FASTCALL for C functions using METH_FASTCALL. In this case, _PyCFunction_FastCallDict() uses _PyStack_UnpackDict(). For example, open() already uses METH_FASTCALL in Python 3.6. In there an issue with _PyStack_UnpackDict()? (I don't think so.)

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29360>
_______________________________________

From report at bugs.python.org  Tue Jan 24 04:45:59 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 24 Jan 2017 09:45:59 +0000
Subject: [issue29360] Don't check if all keys are strings in _PyStack_AsDict()
In-Reply-To: <1485247784.31.0.540630884546.issue29360@psf.upfronthosting.co.za>
Message-ID: <1485251159.8.0.59800653283.issue29360@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Victor> PyPy (and other Python implementations?) don't support non-string in type dictionary.
INADA Naoki> Wow! I thought it's Python's language. If we can prohibit non string name in all namespace, there might be possible optimization.

That's a different topic :-) It would be a backward incompatible change. Well... I don't think that anyone relies on this strange feature :-) IMHO it's more a side effect of the implementation, not a deliberate feature.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29360>
_______________________________________

From report at bugs.python.org  Tue Jan 24 04:48:17 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 24 Jan 2017 09:48:17 +0000
Subject: [issue29336] merge tuples in module
In-Reply-To: <1484969445.33.0.406694538272.issue29336@psf.upfronthosting.co.za>
Message-ID: <1485251297.96.0.279902592314.issue29336@psf.upfronthosting.co.za>


STINNER Victor added the comment:

> But this can be done better with the AST optimizer (issue1346238, issue11549).

There are also the PEP 511 (Code transformers) which was created to implement *external* AST optimizers, and the FAT Python which implements an AST optimizer.

Hum, after "-o no_annotation" discussed on the mailing list, maybe it's time to look again at this PEP? ;-)

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29336>
_______________________________________

From report at bugs.python.org  Tue Jan 24 04:52:50 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Tue, 24 Jan 2017 09:52:50 +0000
Subject: [issue29360] Don't check if all keys are strings in _PyStack_AsDict()
In-Reply-To: <1485247784.31.0.540630884546.issue29360@psf.upfronthosting.co.za>
Message-ID: <1485251570.58.0.873745691948.issue29360@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

We should either ensure that _PyStack_AsDict() is called only by CALL_FUNCTION_KW, and not by other way, or remove both assertions. I prefer the second option. They were here only for self-testing.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29360>
_______________________________________

From report at bugs.python.org  Tue Jan 24 04:56:12 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 24 Jan 2017 09:56:12 +0000
Subject: [issue29336] merge tuples in module
In-Reply-To: <1484969445.33.0.406694538272.issue29336@psf.upfronthosting.co.za>
Message-ID: <1485251772.91.0.148731279986.issue29336@psf.upfronthosting.co.za>


STINNER Victor added the comment:

> Thanks.  Your patch reduced memory consumption by 2%,

merge-constants.patch looks simple enought, but I'm not really impressed by such result. Is 2% worth it?

Since code objects loaded by import are likely for stay for the whole lifetime of a process, I would be interested to experiment interning all constant objects (the tuple of objects, but also each object of these tuples, so support "nested interning") of all code objets (consts, names, varnames, freevars, cellvars) in the marshal module.

To intern constants, we need to generate an unique key to not merge 1 int and 1.0 float. The risk of such global intern dict is to increase the memory usage of the unique keys uses more memory than the decreased caused by the removal of duplicate objects.

More generally, would it be possible to share co_consts (None,) tuples between code objects of two different modules? Or is it already the case with merge-constants.patch?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29336>
_______________________________________

From report at bugs.python.org  Tue Jan 24 04:56:41 2017
From: report at bugs.python.org (Reto Cavelti)
Date: Tue, 24 Jan 2017 09:56:41 +0000
Subject: [issue29361] bug in pyserial: serialposix.py
Message-ID: <1485251801.63.0.344501122232.issue29361@psf.upfronthosting.co.za>


New submission from Reto Cavelti:

bug in pyserial: serialposix.py

line 50:
        # set custom divisor
        buf[6] = buf[7] / baudrate

TypeError: integer argument expected, got float

fix: do cast: buf[6] = int(buf[7] / baudrate)

----------
components: Library (Lib)
messages: 286168
nosy: rc
priority: normal
severity: normal
status: open
title: bug in pyserial: serialposix.py
type: crash
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29361>
_______________________________________

From report at bugs.python.org  Tue Jan 24 04:57:27 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Tue, 24 Jan 2017 09:57:27 +0000
Subject: [issue29360] Don't check if all keys are strings in _PyStack_AsDict()
In-Reply-To: <1485247784.31.0.540630884546.issue29360@psf.upfronthosting.co.za>
Message-ID: <1485251847.26.0.951878886676.issue29360@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

> func(**{'x': 1, 'x': 2})

In this example the dictionary contains only one item. You need more complex example for making the assertion failing. Keys with non-constant hashes or equality, so first they are different, and later they become equal.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29360>
_______________________________________

From report at bugs.python.org  Tue Jan 24 04:57:54 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 24 Jan 2017 09:57:54 +0000
Subject: [issue29336] merge tuples in module
In-Reply-To: <1484969445.33.0.406694538272.issue29336@psf.upfronthosting.co.za>
Message-ID: <1485251874.12.0.435281289748.issue29336@psf.upfronthosting.co.za>


STINNER Victor added the comment:

I don't know what is the most efficient technic to reduce the memory usage of module, but at least I can say imports are the top #1 memory consumer. Look at tracemalloc examples:

https://docs.python.org/dev/library/tracemalloc.html#display-the-top-10

"We can see that Python loaded 4855 KiB data (bytecode and constants) from modules"

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29336>
_______________________________________

From report at bugs.python.org  Tue Jan 24 05:00:50 2017
From: report at bugs.python.org (Berker Peksag)
Date: Tue, 24 Jan 2017 10:00:50 +0000
Subject: [issue29361] bug in pyserial: serialposix.py
In-Reply-To: <1485251801.63.0.344501122232.issue29361@psf.upfronthosting.co.za>
Message-ID: <1485252050.57.0.0586796982578.issue29361@psf.upfronthosting.co.za>


Berker Peksag added the comment:

Thanks for the report, but pyserial is not part of the Python stdlib. Please open your bug report on https://github.com/pyserial/pyserial.

----------
nosy: +berker.peksag
resolution:  -> third party
stage:  -> resolved
status: open -> closed
type: crash -> behavior

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29361>
_______________________________________

From report at bugs.python.org  Tue Jan 24 05:22:07 2017
From: report at bugs.python.org (Marc-Andre Lemburg)
Date: Tue, 24 Jan 2017 10:22:07 +0000
Subject: [issue18235] _sysconfigdata.py wrong on AIX installations
In-Reply-To: <1371429201.94.0.372851990361.issue18235@psf.upfronthosting.co.za>
Message-ID: <1485253327.92.0.142922317881.issue18235@psf.upfronthosting.co.za>


Marc-Andre Lemburg added the comment:

Reading the ticket, it seems that there is some confusion about what LDSHARED and BLDSHARED are used for. BLDSHARED is used to override the LDSHARED value when building libpython and the shared modules (via setup.py).

LDSHARED is what is meant to be used for building shared modules after installation.

This distinction is not being followed by all targets in the Makefile, but is needed in cases where special build tools are necessary, as is the case on AIX.

For the latter, BLDSHARED has to point to the source tree version of those build tools, while LDSHARED has to point to the installed version of these.

On AIX, BLDSHARED should therefore point to the ./Modules/ld_so_aix (or better: the absolute dir of the source tree), while LDSHARED needs to point to the installation target for ld_so_aix, e.g. /usr/local/lib/python2.7/config/ld_so_aix.

So the fix which was already applied to 3.4 is correct for 2.7 as well. The missing part is the fix to the configure script, since in Python 2.7, this still uses relative paths:

                BLDSHARED="Modules/ld_so_aix \$(CC) -bI:Modules/python.exp"
                LDSHARED="\$(BINLIBDEST)/config/ld_so_aix \$(CC) -bI:\$(BINLIBDEST)/config/python.exp"

instead of the ones from 3.4:

                BLDSHARED="\$(srcdir)/Modules/ld_so_aix \$(CC) -bI:\$(srcdir)/Modules/python.exp"
                LDSHARED="\$(BINLIBDEST)/config/ld_so_aix \$(CC) -bI:\$(BINLIBDEST)/config/python.exp"

----------
nosy: +lemburg

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue18235>
_______________________________________

From report at bugs.python.org  Tue Jan 24 05:28:44 2017
From: report at bugs.python.org (Marc-Andre Lemburg)
Date: Tue, 24 Jan 2017 10:28:44 +0000
Subject: [issue18235] _sysconfigdata.py wrong on AIX installations
In-Reply-To: <1371429201.94.0.372851990361.issue18235@psf.upfronthosting.co.za>
Message-ID: <1485253724.26.0.710798196147.issue18235@psf.upfronthosting.co.za>


Marc-Andre Lemburg added the comment:

Hmm, looking at the patch again:

diff -r a089a8b1f93d Lib/sysconfig.py
--- a/Lib/sysconfig.py  Fri Jun 21 18:37:02 2013 -0400
+++ b/Lib/sysconfig.py  Fri Jun 21 22:33:15 2013 -0700
@@ -368,7 +368,7 @@
     # -- these paths are relative to the Python source, but when installed
     # the scripts are in another directory.
     if _PYTHON_BUILD:
-        vars['LDSHARED'] = vars['BLDSHARED']
+        vars['BLDSHARED'] = vars['LDSHARED']
 
     # There's a chicken-and-egg situation on OS X with regards to the
     # _sysconfigdata module after the changes introduced by #15298:

I think that with the configure fix, the special case for AIX can be dropped altogether. It just polishes over the bug in configure, turning a relative path into an absolute one.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue18235>
_______________________________________

From report at bugs.python.org  Tue Jan 24 05:32:24 2017
From: report at bugs.python.org (Marc-Andre Lemburg)
Date: Tue, 24 Jan 2017 10:32:24 +0000
Subject: [issue18235] _sysconfigdata.py wrong on AIX installations
In-Reply-To: <1371429201.94.0.372851990361.issue18235@psf.upfronthosting.co.za>
Message-ID: <1485253944.17.0.710300032691.issue18235@psf.upfronthosting.co.za>


Marc-Andre Lemburg added the comment:

BTW: Does the ticket still apply to 3.5+ ? From reading the ticket, it seems that the problem is already fixed for those versions.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue18235>
_______________________________________

From report at bugs.python.org  Tue Jan 24 05:33:38 2017
From: report at bugs.python.org (Antoine Pitrou)
Date: Tue, 24 Jan 2017 10:33:38 +0000
Subject: [issue18235] _sysconfigdata.py wrong on AIX installations
In-Reply-To: <1371429201.94.0.372851990361.issue18235@psf.upfronthosting.co.za>
Message-ID: <1485254018.7.0.404075853812.issue18235@psf.upfronthosting.co.za>


Changes by Antoine Pitrou <pitrou at free.fr>:


----------
nosy:  -pitrou

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue18235>
_______________________________________

From report at bugs.python.org  Tue Jan 24 06:18:15 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 24 Jan 2017 11:18:15 +0000
Subject: [issue29360] Don't check if all keys are strings in _PyStack_AsDict()
In-Reply-To: <1485247784.31.0.540630884546.issue29360@psf.upfronthosting.co.za>
Message-ID: <1485256695.73.0.136946276018.issue29360@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Attached pystack_asdict-2.patch removes the two assertions and rewrites _PyStack_AsDict() documentation to describe better how it handles non-string and duplicated keys (don't check keys type and merge duplicated keys).


> We should either ensure that _PyStack_AsDict() is called only by CALL_FUNCTION_KW, and not by other way, or remove both assertions. I prefer the second option. They were here only for self-testing.

CALL_FUNTION_EX doesn't use _PyStack_AsDict() (at least, currently, I should carefully review my tp_fast{new,init,call} patches).

When I wrote _PyStack_AsDict(), I added the GetItem==NULL assertion because I didn't know how to handle this corner case.

For best performances, I'm in favor of not calling GetItem in _PyStack_AsDict() and so merge duplicated keys. But before removing the assertion, I wanted to make sure that it doesn't break the Python semantics.

The problem is that I'm unable to find any concrete tcase in Python 3.6 nor Python 3.7 where it would be possible to call _PyStack_AsDict() with duplicate keys. See my analysis below.

I agree to remove both assertion. I just checked, _PyStack_AsDict() already explicitly explain that keys must be strings and must be unique, but are not checked. My patch now also mentions that duplicated keys are merged.

--

(1) When a Python is called by _PyFunction_FastCallDict(), a TypeError is raised if an argument is passed as a positional argument and a keyword argument. Example: "def func(x): pass; func(1, x=2)".

(2) If for some reasons, two keys of keyword arguments are equal (same hash value and are equals) but go into "**kwargs", the last value is kept: _PyEval_EvalCodeWithName() calls PyDict_SetItem() which replaces existing key to the new value. I failed to find an example here.

(3) Calling other functions using kwargs dictionary from a FASTCALL function converts stack+kwnames arguments to kwargs dictionary using _PyStack_AsDict(). _PyCFunction_FastCallKeywords() implements that for the METH_VARARGS|METH_KEYWORDS calling convention. For example, round(3.14, ndigits=1) calls _PyCFunction_FastCallKeywords() with kwnames=('ndigits',) and args=[3.14, 1], _PyStack_AsDict() creates {'ndigits': 1} dictionary.


The tricky question is how (3) handles duplicated keys in dictionary.


"round(3.14, ndigits=1)" uses CALL_FUNCTION_KW which creates kwnames=('ndigits,') constant tuple. The compiler raises a SyntaxError if a function is called with a duplicate key. Without modifying a code object, I don't see any obvious way to pass duplicate keys in kwnames to CALL_FUNCTION_KW.

CALL_FUNCTION_EX expects a dictionary on the stack. For METH_VARARGS|METH_KEYWORDS C functions, the dictionary is passed through, there is not conversion.

If CALL_FUNCTION_EX gets a mapping which is not exactly the dict type, PyDict_New() + PyDict_Update() is used to convert the mapping into a regular dict. This function merges duplicated keys.


Without modifying code objects nor writing buggy C code using _PyObject_FastCallKeywords(), I'm not sure that it's possible to pass duplicated keys to _PyStack_AsDict(). It seems that the issue is more theorical, no?

----------
Added file: http://bugs.python.org/file46404/pystack_asdict-2.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29360>
_______________________________________

From report at bugs.python.org  Tue Jan 24 06:18:41 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 24 Jan 2017 11:18:41 +0000
Subject: [issue29360] _PyStack_AsDict(): Don't check if all keys are strings
 nor if keys are unique
In-Reply-To: <1485247784.31.0.540630884546.issue29360@psf.upfronthosting.co.za>
Message-ID: <1485256721.39.0.0618208309073.issue29360@psf.upfronthosting.co.za>


Changes by STINNER Victor <victor.stinner at gmail.com>:


----------
title: Don't check if all keys are strings in _PyStack_AsDict() -> _PyStack_AsDict(): Don't check if all keys are strings nor if keys are unique

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29360>
_______________________________________

From report at bugs.python.org  Tue Jan 24 06:27:07 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Tue, 24 Jan 2017 11:27:07 +0000
Subject: [issue29360] _PyStack_AsDict(): Don't check if all keys are strings
 nor if keys are unique
In-Reply-To: <1485247784.31.0.540630884546.issue29360@psf.upfronthosting.co.za>
Message-ID: <1485257227.78.0.387059380922.issue29360@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

Try following keys:

class Key:
    def __eq__(self, other):
        return self is other or random.getrandbits(1)
    def __hash__(self):
        return random.getrandbits(1)

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29360>
_______________________________________

From report at bugs.python.org  Tue Jan 24 06:42:16 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 24 Jan 2017 11:42:16 +0000
Subject: [issue29360] _PyStack_AsDict(): Don't check if all keys are strings
 nor if keys are unique
In-Reply-To: <1485247784.31.0.540630884546.issue29360@psf.upfronthosting.co.za>
Message-ID: <1485258136.95.0.107405776982.issue29360@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Serhiy Storchaka: "Try following keys: (...)"

To which function am I supposed to pass such keys? I was unable to find a simple way how to pass arbitrary keys to _PyStack_AsDict().

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29360>
_______________________________________

From report at bugs.python.org  Tue Jan 24 06:59:55 2017
From: report at bugs.python.org (Jim Fasarakis-Hilliard)
Date: Tue, 24 Jan 2017 11:59:55 +0000
Subject: [issue29349] Update old Python 2 code in
 Docs/tools/extensions/patchlevel.py
In-Reply-To: <1485164481.25.0.457345760711.issue29349@psf.upfronthosting.co.za>
Message-ID: <1485259195.77.0.683967058655.issue29349@psf.upfronthosting.co.za>


Jim Fasarakis-Hilliard added the comment:

I'm breaking these to separate files to make it easier to apply. I also noticed that other files in `Doc/tools/extensions/` use old constructs so I'm not sure about the *with*.

I'm guessing that either it should be changed in other files too or, since it's working fine, leave it as is.

----------
Added file: http://bugs.python.org/file46405/patchlevel_print.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29349>
_______________________________________

From report at bugs.python.org  Tue Jan 24 07:00:03 2017
From: report at bugs.python.org (Jim Fasarakis-Hilliard)
Date: Tue, 24 Jan 2017 12:00:03 +0000
Subject: [issue29349] Update old Python 2 code in
 Docs/tools/extensions/patchlevel.py
In-Reply-To: <1485164481.25.0.457345760711.issue29349@psf.upfronthosting.co.za>
Message-ID: <1485259203.3.0.611531854745.issue29349@psf.upfronthosting.co.za>


Changes by Jim Fasarakis-Hilliard <d.f.hilliard at gmail.com>:


Added file: http://bugs.python.org/file46406/patchlevel_with.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29349>
_______________________________________

From report at bugs.python.org  Tue Jan 24 07:01:50 2017
From: report at bugs.python.org (Martin Panter)
Date: Tue, 24 Jan 2017 12:01:50 +0000
Subject: [issue18235] _sysconfigdata.py wrong on AIX installations
In-Reply-To: <1371429201.94.0.372851990361.issue18235@psf.upfronthosting.co.za>
Message-ID: <1485259310.65.0.0359963943471.issue18235@psf.upfronthosting.co.za>


Martin Panter added the comment:

I don?t run AIX, but my understanding is there are three distinct branches (2.7, 3.5, and 3.6+). Some of the following is guessed, so please correct me if I am wrong:

Python 2.7:
_sysconfigdata.py incorrectly created with LDSHARED = Modules/ld_so_aix.
Result: Installed distutils fails (unless you?re in the build tree).
Fix: Apply David?s issue18235.patch + Michael H?s msg219888 fix.

Python 3.5:
David?s issue18235.patch already applied.
_sysconfigdata.py incorrectly created with BLDSHARED = $(LIBPL)/ld_so_aix, i.e. the installed location.
Distutils does not refer to _sysconfigdata.py.
Result: Distutils unaffected, but running from the build tree, sysconfig.get_config_var("LDSHARED") would incorrectly use the installed location.
Fix: Apply Michael H?s msg219888.

Python 3.6+:
Distutils does use _sysconfigdata*.py, thanks to revision 3fa8aebed636.
Result: Distutils shouldn?t work unless Python is already installed. Since the 3.6 and 3.x buildbots pass test_distutils, does that mean that both those versions of Python are already installed?
Fix: Should also be fixed by merging msg219888 from 3.5.

Marc-Andre, regarding configure.ac and absolute vs relative paths, I suggest you open a separate bug. That is an orthogonal issue, and this bug is already too complicated. But also keep in mind Issue 18235. You want the build tree, not the source tree.

Also, if you mean to drop the assignment in sysconfig, I think that will break in-tree usage of sysconfig (3.5+) and distutils (3.6+), unless Python happens to also be installed.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue18235>
_______________________________________

From report at bugs.python.org  Tue Jan 24 07:02:08 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Tue, 24 Jan 2017 12:02:08 +0000
Subject: [issue29360] _PyStack_AsDict(): Don't check if all keys are strings
 nor if keys are unique
In-Reply-To: <1485247784.31.0.540630884546.issue29360@psf.upfronthosting.co.za>
Message-ID: <1485259328.81.0.280857542047.issue29360@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

I don't know. To that that is crashed when pass non-string keys.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29360>
_______________________________________

From report at bugs.python.org  Tue Jan 24 07:08:08 2017
From: report at bugs.python.org (Martin Panter)
Date: Tue, 24 Jan 2017 12:08:08 +0000
Subject: [issue18235] _sysconfigdata.py wrong on AIX installations
In-Reply-To: <1371429201.94.0.372851990361.issue18235@psf.upfronthosting.co.za>
Message-ID: <1485259688.94.0.631316323378.issue18235@psf.upfronthosting.co.za>


Martin Panter added the comment:

Sorry I meant Issue 10656. I recently committed a fix regarding out-of-tree builds and ld_so_aix, and it sounds like you were thinking of reverting that.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue18235>
_______________________________________

From report at bugs.python.org  Tue Jan 24 08:15:16 2017
From: report at bugs.python.org (Vinay Sajip)
Date: Tue, 24 Jan 2017 13:15:16 +0000
Subject: [issue29308] venv should match virtualenv VIRTUAL_ENV_DISABLE_PROMPT
 config
In-Reply-To: <1484750377.25.0.809506276992.issue29308@psf.upfronthosting.co.za>
Message-ID: <1485263716.98.0.198797828785.issue29308@psf.upfronthosting.co.za>


Vinay Sajip added the comment:

Other scripts (activate, activate.csh, activate.csh) support this environment variable already. This patch updates Activate.ps1 to respect the variable - can you confirm if this change works for you?

----------
assignee:  -> vinay.sajip
keywords: +patch
stage:  -> patch review
Added file: http://bugs.python.org/file46407/fix-29308-01.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29308>
_______________________________________

From report at bugs.python.org  Tue Jan 24 08:33:42 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 24 Jan 2017 13:33:42 +0000
Subject: [issue29360] _PyStack_AsDict(): Don't check if all keys are strings
 nor if keys are unique
In-Reply-To: <1485247784.31.0.540630884546.issue29360@psf.upfronthosting.co.za>
Message-ID: <1485264822.92.0.330252162864.issue29360@psf.upfronthosting.co.za>


STINNER Victor added the comment:

> I don't know. To that that is crashed when pass non-string keys.

Oh ok. I found a way to create a dictionary with equal keys and pass it to dict(**kw) in my fast_init branch (I already removed the PyUnicode_Check() asssertion in this branch):

haypo at selma$ ./python bug.py 
call with: {'a': 1, 'b': 2}
python: Objects/abstract.c:2471: _PyStack_AsDict: Assertion `PyDict_GetItem(kwdict, key) == NULL' failed.
Aborted


So yes, with special objects, the PyDict_GetItem()==NULL assertion fails. With pystack_asdict-2.patch, the duplicated (equal) keys are merged as expected:

$ ./python bug.py 
call with: {'a': 1, 'b': 2}
got: {'a': 2}

Again, there is also a bug in my fast_init branch, I should avoid double conversion dict=>kwnames=>dict to call a type.

----------
Added file: http://bugs.python.org/file46408/bug.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29360>
_______________________________________

From report at bugs.python.org  Tue Jan 24 08:39:20 2017
From: report at bugs.python.org (Michael Felt)
Date: Tue, 24 Jan 2017 13:39:20 +0000
Subject: [issue18235] _sysconfigdata.py wrong on AIX installations
In-Reply-To: <1371429201.94.0.372851990361.issue18235@psf.upfronthosting.co.za>
Message-ID: <1485265160.44.0.56314666222.issue18235@psf.upfronthosting.co.za>


Michael Felt added the comment:

re: msg286115, 

1) Building modules in the tree during the build process.
** if "in the tree" means building outside of source - such as:
../src/python-X.Y.Z/configure ...
make

** the tarball for Python-2.7.13 is building from ../src/Python-2.7.13/configure where as ../src/Python-2.7.12/configure and the earlier (ones I tried) were not.

** However, 'make distclean' is not working. 
   Further details in a later issue - this bit about make distclean id merely FYI. 

2) In-tree testing of build module feature (test_distutils).

** Is this as simple as (from the 'build directory':
make test 

** shall check this later, make test in general seems to be working - is test_distutils a specific test you want verified?
3) Building and installing modules with an installed version of Python.
** FYI: this will work with the 5 lines deleted - as I mentioned above.
   This is how I manually patched _sysconfigdata.py in Python-2.7.12 when I ran into this.

Now reading the other comments from last night...

re: 286172 and 286173

Short response:
it looks as if patches elsewhere in the 'auto' config files has taken care of this issue - at least for Python-2.7

I shall download and check the latest Python-3.{4|5|6} and report back.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue18235>
_______________________________________

From report at bugs.python.org  Tue Jan 24 08:39:41 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Tue, 24 Jan 2017 13:39:41 +0000
Subject: [issue29360] _PyStack_AsDict(): Don't check if all keys are strings
 nor if keys are unique
In-Reply-To: <1485247784.31.0.540630884546.issue29360@psf.upfronthosting.co.za>
Message-ID: <1485265181.28.0.557847652214.issue29360@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

pystack_asdict-2.patch LGTM.

----------
assignee:  -> haypo
stage:  -> commit review

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29360>
_______________________________________

From report at bugs.python.org  Tue Jan 24 09:09:14 2017
From: report at bugs.python.org (Roundup Robot)
Date: Tue, 24 Jan 2017 14:09:14 +0000
Subject: [issue29360] _PyStack_AsDict(): Don't check if all keys are strings
 nor if keys are unique
In-Reply-To: <1485247784.31.0.540630884546.issue29360@psf.upfronthosting.co.za>
Message-ID: <20170124140910.43181.31368.51D7277B@psf.io>


Roundup Robot added the comment:

New changeset 2d2210b36b25 by Victor Stinner in branch 'default':
Issue #29360: _PyStack_AsDict() doesn't check kwnames
https://hg.python.org/cpython/rev/2d2210b36b25

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29360>
_______________________________________

From report at bugs.python.org  Tue Jan 24 09:10:46 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 24 Jan 2017 14:10:46 +0000
Subject: [issue29360] _PyStack_AsDict(): Don't check if all keys are strings
 nor if keys are unique
In-Reply-To: <1485247784.31.0.540630884546.issue29360@psf.upfronthosting.co.za>
Message-ID: <1485267046.49.0.113903824395.issue29360@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Thanks Serhiy and Naoki. I really wanted more eyes on this weird issue :-)

It's also a reminder that we should be careful when converting keyword arguments, especially when duplicates are not allowed.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29360>
_______________________________________

From report at bugs.python.org  Tue Jan 24 09:23:57 2017
From: report at bugs.python.org (Michael Felt)
Date: Tue, 24 Jan 2017 14:23:57 +0000
Subject: [issue18235] _sysconfigdata.py wrong on AIX installations
In-Reply-To: <1371429201.94.0.372851990361.issue18235@psf.upfronthosting.co.za>
Message-ID: <1485267837.8.0.749734641429.issue18235@psf.upfronthosting.co.za>


Michael Felt added the comment:

As far as issue10656 and this issue are concerned:

Python-3.4 is out of context (but 3.4.6 was just released) - and does not work with 'out of tree' builds. 

The other versions: 2.7.13, 3.5.3 and 3.6.0 do build out-of-tree.
Note 3.5.3 and 3.6.0 use a different name for sysconfigdata.py

Notes:

FYI: regarding build and src in different directories

Not working...

unable to execute '../src/Python-3.4.6/Modules/ld_so_aix': No such file or directory
unable to execute '../src/Python-3.4.6/Modules/ld_so_aix': No such file or directory
 many many times

Concludes with:
ImportError: No module named 'time'
make: 1254-004 The error code from the last command is 1.
make: 1254-005 Ignored error code 1 from last command.
../src/Python-3.4.6/install-sh: ../src/Python-3.4.6/Modules/ld_so_aix does not exist
make: 1254-004 The error code from the last command is 1.

root at x064:[/data/prj/python/Python-3.4.6]find . -name _sysconfigdata.py -ls -exec grep LDSHARED {} \;
48049316   19 -rw-r-----  1 root      felt         19209 Jan 24 13:55 ./build/lib.aix-5.3-3.4/_sysconfigdata.py
 'BLDSHARED': '/opt/lib/python3.4/config/ld_so_aix xlc_r '
 'LDSHARED': '/opt/lib/python3.4/config/ld_so_aix xlc_r '

+++++++ Working ++++++++

Working: 2.7.13 -- Note: the 'lines' in sysconfig.py are removed:
root at x064:[/data/prj/python/Python-2.7.13.0]find . -name _sysconfigdata\*.py -ls -exec grep LDSHARED {} \;
48038426   16 -rw-r-----  1 root      felt         15807 Jan 24 13:36 ./build/lib.aix-5.3-2.7/_sysconfigdata.py
 'BLDSHARED': 'Modules/ld_so_aix xlc_r -bI:Modules/python.exp',
 'LDSHARED': '/opt/lib/python2.7/config/ld_so_aix xlc_r -bI:/opt/lib/python2.7/config/python.exp',

Working: 3.5.3
48049196   21 -rw-r-----  1 root      felt         20640 Jan 24 13:55 ./build/lib.aix-5.3-3.5/_sysconfigdata.py
 'BLDSHARED': '/opt/lib/python3.5/config-3.5m/ld_so_aix xlc_r '
 'LDSHARED': '/opt/lib/python3.5/config-3.5m/ld_so_aix xlc_r '

Working: 3.6.0 -- Note new file name!!
root at x064:[/data/prj/python/Python-3.6.0]find . -name _sysconfigdata\*.py -ls -exec grep LDSHARED {} \;
47794146   20 -rw-r-----  1 root      felt         20394 Jan 24 13:49 ./build/lib.aix-5.3-3.6/_sysconfigdata_m_aix5_.py
 'BLDSHARED': 'Modules/ld_so_aix xlc_r -bI:Modules/python.exp',
 'LDSHARED': '/opt/lib/python3.6/config-3.6m/ld_so_aix xlc_r '

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue18235>
_______________________________________

From report at bugs.python.org  Tue Jan 24 09:29:29 2017
From: report at bugs.python.org (Jack Bennett)
Date: Tue, 24 Jan 2017 14:29:29 +0000
Subject: [issue29308] venv should match virtualenv VIRTUAL_ENV_DISABLE_PROMPT
 config
In-Reply-To: <1485263716.98.0.198797828785.issue29308@psf.upfronthosting.co.za>
Message-ID: <CAPtmCpdyxfZBUT+1HciDksJXvyyfKV0n_M6YoB1V1mE2b61VsA@mail.gmail.com>


Jack Bennett added the comment:

That works fine here.

Thank you

On 24 January 2017 at 13:15, Vinay Sajip <report at bugs.python.org> wrote:

>
> Vinay Sajip added the comment:
>
> Other scripts (activate, activate.csh, activate.csh) support this
> environment variable already. This patch updates Activate.ps1 to respect
> the variable - can you confirm if this change works for you?
>
> ----------
> assignee:  -> vinay.sajip
> keywords: +patch
> stage:  -> patch review
> Added file: http://bugs.python.org/file46407/fix-29308-01.diff
>
> _______________________________________
> Python tracker <report at bugs.python.org>
> <http://bugs.python.org/issue29308>
> _______________________________________
>

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29308>
_______________________________________

From report at bugs.python.org  Tue Jan 24 10:12:17 2017
From: report at bugs.python.org (Rob Reilink)
Date: Tue, 24 Jan 2017 15:12:17 +0000
Subject: [issue19500] Add client-side SSL session resumption
In-Reply-To: <1383624130.64.0.185541289049.issue19500@psf.upfronthosting.co.za>
Message-ID: <1485270737.62.0.809952021045.issue19500@psf.upfronthosting.co.za>


Rob Reilink added the comment:

With this code in place, ftplib should / could also be updated to support session resumption. This would fix bugs with connections to FTP servers that require session resumption [1], [2]

In ftplib.FTP_TLS.ntransfercmd, just add a reference to the current session in the wrap_socket call (maybe make this an option to do session resumption or not; I don't know if it could break something)

Proposed patch is attached.

[1] http://stackoverflow.com/questions/14659154/ftpes-session-reuse-required
[2] https://forum.filezilla-project.org/viewtopic.php?t=36903

----------
nosy: +robr
Added file: http://bugs.python.org/file46409/ftplib_session.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue19500>
_______________________________________

From report at bugs.python.org  Tue Jan 24 10:23:26 2017
From: report at bugs.python.org (Marc-Andre Lemburg)
Date: Tue, 24 Jan 2017 15:23:26 +0000
Subject: [issue18235] _sysconfigdata.py wrong on AIX installations
In-Reply-To: <1485267837.8.0.749734641429.issue18235@psf.upfronthosting.co.za>
Message-ID: <c5bd1207-86c9-1ba5-5ae2-c7e83708c2a7@egenix.com>


Marc-Andre Lemburg added the comment:

On 24.01.2017 15:23, Michael Felt wrote:
> As far as issue10656 and this issue are concerned:
> 
> Python-3.4 is out of context (but 3.4.6 was just released) - and does not work with 'out of tree' builds. 
> 
> The other versions: 2.7.13, 3.5.3 and 3.6.0 do build out-of-tree.
> Note 3.5.3 and 3.6.0 use a different name for sysconfigdata.py

I'm not sure what you mean with "out of tree". When building
Python, you are supposed to use the source code directory
as work dir.

> Notes:
> 
> FYI: regarding build and src in different directories
> 
> Not working...
> 
> unable to execute '../src/Python-3.4.6/Modules/ld_so_aix': No such file or directory
> unable to execute '../src/Python-3.4.6/Modules/ld_so_aix': No such file or directory
>  many many times
> 
> Concludes with:
> ImportError: No module named 'time'
> make: 1254-004 The error code from the last command is 1.
> make: 1254-005 Ignored error code 1 from last command.
> ../src/Python-3.4.6/install-sh: ../src/Python-3.4.6/Modules/ld_so_aix does not exist
> make: 1254-004 The error code from the last command is 1.
> 
> root at x064:[/data/prj/python/Python-3.4.6]find . -name _sysconfigdata.py -ls -exec grep LDSHARED {} \;
> 48049316   19 -rw-r-----  1 root      felt         19209 Jan 24 13:55 ./build/lib.aix-5.3-3.4/_sysconfigdata.py
>  'BLDSHARED': '/opt/lib/python3.4/config/ld_so_aix xlc_r '
>  'LDSHARED': '/opt/lib/python3.4/config/ld_so_aix xlc_r '

This is strange: where does the "../src/Python-3.4.6/Modules/ld_so_aix"
originate if the paths in the sysconfig file are absolute ?

> +++++++ Working ++++++++
> 
> Working: 2.7.13 -- Note: the 'lines' in sysconfig.py are removed:
> root at x064:[/data/prj/python/Python-2.7.13.0]find . -name _sysconfigdata\*.py -ls -exec grep LDSHARED {} \;
> 48038426   16 -rw-r-----  1 root      felt         15807 Jan 24 13:36 ./build/lib.aix-5.3-2.7/_sysconfigdata.py
>  'BLDSHARED': 'Modules/ld_so_aix xlc_r -bI:Modules/python.exp',
>  'LDSHARED': '/opt/lib/python2.7/config/ld_so_aix xlc_r -bI:/opt/lib/python2.7/config/python.exp',

BLDSHARED should be fixed to use an absolute path, I guess.

> Working: 3.5.3
> 48049196   21 -rw-r-----  1 root      felt         20640 Jan 24 13:55 ./build/lib.aix-5.3-3.5/_sysconfigdata.py
>  'BLDSHARED': '/opt/lib/python3.5/config-3.5m/ld_so_aix xlc_r '
>  'LDSHARED': '/opt/lib/python3.5/config-3.5m/ld_so_aix xlc_r '

Here BLDSHARED is wrong, but shouldn't do any harm since
it's only used for building Python itself, not for
extension modules.

> Working: 3.6.0 -- Note new file name!!
> root at x064:[/data/prj/python/Python-3.6.0]find . -name _sysconfigdata\*.py -ls -exec grep LDSHARED {} \;
> 47794146   20 -rw-r-----  1 root      felt         20394 Jan 24 13:49 ./build/lib.aix-5.3-3.6/_sysconfigdata_m_aix5_.py
>  'BLDSHARED': 'Modules/ld_so_aix xlc_r -bI:Modules/python.exp',
>  'LDSHARED': '/opt/lib/python3.6/config-3.6m/ld_so_aix xlc_r '

This looks odd and like a regression. Again, the path
should be absolute and point to the build tree, not the
installation tree.

PS: The BLDSHARED setting in sysconfig is not really relevant.
It's only there because we have it in the Makefile.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue18235>
_______________________________________

From report at bugs.python.org  Tue Jan 24 10:24:20 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 24 Jan 2017 15:24:20 +0000
Subject: [issue29360] _PyStack_AsDict(): Don't check if all keys are strings
 nor if keys are unique
In-Reply-To: <1485247784.31.0.540630884546.issue29360@psf.upfronthosting.co.za>
Message-ID: <1485271460.22.0.816890752505.issue29360@psf.upfronthosting.co.za>


Changes by STINNER Victor <victor.stinner at gmail.com>:


----------
resolution:  -> fixed
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29360>
_______________________________________

From report at bugs.python.org  Tue Jan 24 10:27:48 2017
From: report at bugs.python.org (Michael Felt)
Date: Tue, 24 Jan 2017 15:27:48 +0000
Subject: [issue18235] _sysconfigdata.py wrong on AIX installations
In-Reply-To: <1371429201.94.0.372851990361.issue18235@psf.upfronthosting.co.za>
Message-ID: <1485271668.79.0.901846530849.issue18235@psf.upfronthosting.co.za>


Michael Felt added the comment:

I am back at this: (note: short answer to msg277745 - No. does not work for me)

As ... expressed before, this needs to work in three contexts:

2) In-tree testing of build module feature (test_distutils).

Below are results of 'in tree' and 'out of tree' testing. in/out seems to have no (more) impact.
++++
msg219888 also has a patch - that moves the code back to where it originally was: _init_posix

But - same issue - ld_so_aix is not found. (notice the "src" in string of 'threading.py')
[112/401/2] test_distutils
xlc_r: 1501-218 (S) file _configtest.i contains an incorrect file suffix
unable to execute '/opt/lib/python2.7/config/ld_so_aix': No such file or directory
Warning -- os.environ was modified by test_distutils
test test_distutils failed -- multiple errors occurred; run in verbose mode for details
[113/401/3] test_dl
test_dl skipped -- No module named dl
[114/401/3] test_docxmlrpc
/data/prj/python/src/Python-2.7.13/Lib/threading.py:846: DeprecationWarning: sys.exc_clear() not supported in 3.x; use except clauses
  self.__exc_clear()
[115/401/3] test_dumbdbm
root at x064:[/data/prj/python/Python-2.7.13]


Now, back to 'in tree' with patch from msg219888 (no "src" in path)

root at x064:[/data/prj/python/Python-2.7.13]find . -name _sysconfigdata\*.py -ls -exec grep LDSHARED {} \;
48056876   16 -rw-r-----  1 root      1000        15362 Jan 24 15:07 ./build/lib.aix-5.3-2.7/_sysconfigdata.py
 'BLDSHARED': 'Modules/ld_so_aix xlc_r -bI:Modules/python.exp',
 'LDSHARED': '/opt/lib/python2.7/config/ld_so_aix xlc_r -bI:/opt/lib/python2.7/config/python.exp',
[112/401/2] test_distutils
xlc_r: 1501-218 (S) file _configtest.i contains an incorrect file suffix
unable to execute '/opt/lib/python2.7/config/ld_so_aix': No such file or directory
Warning -- os.environ was modified by test_distutils
test test_distutils failed -- multiple errors occurred; run in verbose mode for details
[113/401/3] test_dl
test_dl skipped -- No module named dl
[114/401/3] test_docxmlrpc
/data/prj/python/Python-2.7.13/Lib/threading.py:846: DeprecationWarning: sys.exc_clear() not supported in 3.x; use except clauses
  self.__exc_clear()

Conclusion:
** As the code is - that is removed means the environment variable in sysconfigdata.py is correct,
   I would suggest the patch is to remove the 6 lines there.
   Further, the addition of the code below (per msg219888) has no effect, including it is not needed.

The .dist file is from the source tarball from last december for 2.7.13

michael at x071:[/data/prj/python/src]diff -u  Python-2.7.13/Lib/sysconfig.py.dist>
--- Python-2.7.13/Lib/sysconfig.py.dist 2016-12-17 20:05:06 +0000
+++ Python-2.7.13/Lib/sysconfig.py      2017-01-24 14:56:20 +0000
@@ -307,12 +307,6 @@
             msg = msg + " (%s)" % e.strerror
         raise IOError(msg)

-    # On AIX, there are wrong paths to the linker scripts in the Makefile
-    # -- these paths are relative to the Python source, but when installed
-    # the scripts are in another directory.
-    if _PYTHON_BUILD:
-        vars['LDSHARED'] = vars['BLDSHARED']
-
     # There's a chicken-and-egg situation on OS X with regards to the
     # _sysconfigdata module after the changes introduced by #15298:
     # get_config_vars() is called by get_platform() as part of the
@@ -355,6 +349,11 @@
     # _sysconfigdata is generated at build time, see _generate_posix_vars()
     from _sysconfigdata import build_time_vars
     vars.update(build_time_vars)
+    # On AIX, there are wrong paths to the linker scripts in the Makefile
+    # -- these paths are relative to the Python source, but when installed
+    # the scripts are in another directory.
+    if _PYTHON_BUILD:
+        vars['LDSHARED'] = vars['BLDSHARED']

 def _init_non_posix(vars):
     """Initialize the module as appropriate for NT"""

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue18235>
_______________________________________

From report at bugs.python.org  Tue Jan 24 10:35:04 2017
From: report at bugs.python.org (Michael Felt)
Date: Tue, 24 Jan 2017 15:35:04 +0000
Subject: [issue18235] _sysconfigdata.py wrong on AIX installations
In-Reply-To: <1371429201.94.0.372851990361.issue18235@psf.upfronthosting.co.za>
Message-ID: <1485272104.04.0.38375087911.issue18235@psf.upfronthosting.co.za>


Michael Felt added the comment:

OK - so details on the failing tests - for the record only.

However, I would appreciate some 'expert' feedback on what is expected to be happening in this test:

======================================================================
FAIL: test_sysconfig_compiler_vars (distutils.tests.test_sysconfig.SysconfigTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/data/prj/python/Python-2.7.13/Lib/distutils/tests/test_sysconfig.py", line 102, in test_sysconfig_compiler_vars
    self.assertEqual(global_sysconfig.get_config_var('LDSHARED'), sysconfig.get_config_var('LDSHARED'))
AssertionError: 'Modules/ld_so_aix xlc_r -bI:Modules/python.exp' != '/opt/lib/python2.7/config/ld_so_aix xlc_r -bI:/opt/lib/python2.7/config/python.exp'

----------------------------------------------------------------------

BULK details...

======================================================================
ERROR: test_record_extensions (distutils.tests.test_install.InstallTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/data/prj/python/Python-2.7.13/Lib/distutils/tests/test_install.py", line 216, in test_record_extensions
    cmd.run()
  File "/data/prj/python/Python-2.7.13/Lib/distutils/command/install.py", line 563, in run
    self.run_command('build')
  File "/data/prj/python/Python-2.7.13/Lib/distutils/cmd.py", line 326, in run_command
    self.distribution.run_command(command)
  File "/data/prj/python/Python-2.7.13/Lib/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/data/prj/python/Python-2.7.13/Lib/distutils/command/build.py", line 127, in run
    self.run_command(cmd_name)
  File "/data/prj/python/Python-2.7.13/Lib/distutils/cmd.py", line 326, in run_command
    self.distribution.run_command(command)
  File "/data/prj/python/Python-2.7.13/Lib/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/data/prj/python/Python-2.7.13/Lib/distutils/command/build_ext.py", line 340, in run
    self.build_extensions()
  File "/data/prj/python/Python-2.7.13/Lib/distutils/command/build_ext.py", line 449, in build_extensions
    self.build_extension(ext)
  File "/data/prj/python/Python-2.7.13/Lib/distutils/command/build_ext.py", line 531, in build_extension
    target_lang=language)
  File "/data/prj/python/Python-2.7.13/Lib/distutils/ccompiler.py", line 691, in link_shared_object
    extra_preargs, extra_postargs, build_temp, target_lang)
  File "/data/prj/python/Python-2.7.13/Lib/distutils/unixccompiler.py", line 202, in link
    raise LinkError, msg
LinkError: command '/opt/lib/python2.7/config/ld_so_aix' failed with exit status 1

======================================================================
ERROR: test_search_cpp (distutils.tests.test_config_cmd.ConfigTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/data/prj/python/Python-2.7.13/Lib/distutils/tests/test_config_cmd.py", line 46, in test_search_cpp
    match = cmd.search_cpp(pattern='xxx', body='/* xxx */')
  File "/data/prj/python/Python-2.7.13/Lib/distutils/command/config.py", line 206, in search_cpp
    src, out = self._preprocess(body, headers, include_dirs, lang)
  File "/data/prj/python/Python-2.7.13/Lib/distutils/command/config.py", line 129, in _preprocess
    self.compiler.preprocess(src, out, include_dirs=include_dirs)
  File "/data/prj/python/Python-2.7.13/Lib/distutils/unixccompiler.py", line 113, in preprocess
    raise CompileError, msg
CompileError: command 'xlc_r' failed with exit status 40

======================================================================
ERROR: test_build_ext (distutils.tests.test_build_ext.BuildExtTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/data/prj/python/Python-2.7.13/Lib/distutils/tests/test_build_ext.py", line 63, in test_build_ext
    cmd.run()
  File "/data/prj/python/Python-2.7.13/Lib/distutils/command/build_ext.py", line 340, in run
    self.build_extensions()
  File "/data/prj/python/Python-2.7.13/Lib/distutils/command/build_ext.py", line 449, in build_extensions
    self.build_extension(ext)
  File "/data/prj/python/Python-2.7.13/Lib/distutils/command/build_ext.py", line 531, in build_extension
    target_lang=language)
  File "/data/prj/python/Python-2.7.13/Lib/distutils/ccompiler.py", line 691, in link_shared_object
    extra_preargs, extra_postargs, build_temp, target_lang)
  File "/data/prj/python/Python-2.7.13/Lib/distutils/unixccompiler.py", line 202, in link
    raise LinkError, msg
LinkError: command '/opt/lib/python2.7/config/ld_so_aix' failed with exit status 1

======================================================================
ERROR: test_get_outputs (distutils.tests.test_build_ext.BuildExtTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/data/prj/python/Python-2.7.13/Lib/distutils/tests/test_build_ext.py", line 292, in test_get_outputs
    cmd.run()
  File "/data/prj/python/Python-2.7.13/Lib/distutils/command/build_ext.py", line 340, in run
    self.build_extensions()
  File "/data/prj/python/Python-2.7.13/Lib/distutils/command/build_ext.py", line 449, in build_extensions
    self.build_extension(ext)
  File "/data/prj/python/Python-2.7.13/Lib/distutils/command/build_ext.py", line 531, in build_extension
    target_lang=language)
  File "/data/prj/python/Python-2.7.13/Lib/distutils/ccompiler.py", line 691, in link_shared_object
    extra_preargs, extra_postargs, build_temp, target_lang)
  File "/data/prj/python/Python-2.7.13/Lib/distutils/unixccompiler.py", line 202, in link
    raise LinkError, msg
LinkError: command '/opt/lib/python2.7/config/ld_so_aix' failed with exit status 1

======================================================================
FAIL: test_sysconfig_compiler_vars (distutils.tests.test_sysconfig.SysconfigTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/data/prj/python/Python-2.7.13/Lib/distutils/tests/test_sysconfig.py", line 102, in test_sysconfig_compiler_vars
    self.assertEqual(global_sysconfig.get_config_var('LDSHARED'), sysconfig.get_config_var('LDSHARED'))
AssertionError: 'Modules/ld_so_aix xlc_r -bI:Modules/python.exp' != '/opt/lib/python2.7/config/ld_so_aix xlc_r -bI:/opt/lib/python2.7/config/python.exp'

----------------------------------------------------------------------
Ran 194 tests in 3.254s

FAILED (failures=1, errors=4, skipped=21)
Traceback (most recent call last):
  File "test_distutils.py", line 18, in <module>
    test_main()
  File "test_distutils.py", line 13, in test_main
    test_support.run_unittest(distutils.tests.test_suite())
  File "/data/prj/python/Python-2.7.13/Lib/test/test_support.py", line 1494, in run_unittest
    _run_suite(suite)
  File "/data/prj/python/Python-2.7.13/Lib/test/test_support.py", line 1477, in _run_suite
    raise TestFailed(err)
test.test_support.TestFailed: multiple errors occurred
root at x064:[/data/prj/python/Python-2.7.13/Lib/test]

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue18235>
_______________________________________

From report at bugs.python.org  Tue Jan 24 10:35:21 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 24 Jan 2017 15:35:21 +0000
Subject: [issue29362] regrtest: don't fail immediately if a child does crash
Message-ID: <1485272121.72.0.342410390691.issue29362@psf.upfronthosting.co.za>


New submission from STINNER Victor:

When regrtest is used with -jN, regrtest spawn child processes to run tests. If a child crashs, regrtest master process immedialtely fails. It would prefer to be able to continue to run following tests. It's not because a single test crashed that all tests would crash.

Example of output when a crash occurs:
-----------------
(...)
0:02:45 [118/404/3] test_bytes crashed -- running: test_multiprocessing_spawn (67 sec), test_io (54 sec), test_subprocess (76 sec)
python: Objects/abstract.c:2186: _Py_CheckFunctionResult: Assertion `(callable != NULL) ^ (where != NULL)' failed.
Fatal Python error: Aborted

Current thread 0x00007f9210f11440 (most recent call first):
  File "/home/haypo/prog/python/git_cpython/Lib/unittest/case.py", line 177 in handle
  File "/home/haypo/prog/python/git_cpython/Lib/unittest/case.py", line 728 in assertRaises
  File "/home/haypo/prog/python/git_cpython/Lib/test/test_bytes.py", line 636 in test_maketrans
  (...)
  File "/home/haypo/prog/python/git_cpython/Lib/runpy.py", line 85 in _run_code
  File "/home/haypo/prog/python/git_cpython/Lib/runpy.py", line 193 in _run_module_as_main

Traceback (most recent call last):
  File "/home/haypo/prog/python/git_cpython/Lib/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  (...)
  File "/home/haypo/prog/python/git_cpython/Lib/test/libregrtest/main.py", line 411, in run_tests
    run_tests_multiprocess(self)
  File "/home/haypo/prog/python/git_cpython/Lib/test/libregrtest/runtest_mp.py", line 221, in run_tests_multiprocess
    raise Exception(msg)
Exception: Child error on test_bytes: Exit code -6
-----------------

----------
components: Tests
messages: 286194
nosy: haypo
priority: normal
severity: normal
status: open
title: regrtest: don't fail immediately if a child does crash
type: enhancement
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29362>
_______________________________________

From report at bugs.python.org  Tue Jan 24 10:36:04 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 24 Jan 2017 15:36:04 +0000
Subject: [issue29362] regrtest: don't fail immediately if a child does crash
In-Reply-To: <1485272121.72.0.342410390691.issue29362@psf.upfronthosting.co.za>
Message-ID: <1485272164.36.0.407299377762.issue29362@psf.upfronthosting.co.za>


STINNER Victor added the comment:

At least, I would like to see which tests were executed, which tests failed, total duration, etc. Get a summary!

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29362>
_______________________________________

From report at bugs.python.org  Tue Jan 24 10:47:35 2017
From: report at bugs.python.org (Eric N. Vander Weele)
Date: Tue, 24 Jan 2017 15:47:35 +0000
Subject: [issue18235] _sysconfigdata.py wrong on AIX installations
In-Reply-To: <1371429201.94.0.372851990361.issue18235@psf.upfronthosting.co.za>
Message-ID: <1485272855.67.0.133168651101.issue18235@psf.upfronthosting.co.za>


Eric N. Vander Weele added the comment:

I have a (large) patch that completely eliminates the need for ld_so_aix and makeexp_aix.  I've applied and been using this with for Python 2.7 and 3.5+, but I still need to go back and validate the tests to ensure everything passes as expected.

It's still a work in progress, but the patch can be found at https://github.com/ericvw/cpython/commit/e889f5fd04c1b73ef06e9f6e60108b2a7718b7d6.

If this is out of context for this particular issue, I'm happy to raise a new issue to have a focus discussion about potentially landing this.  Ignore changes for README.AIX since I am addressing that in issue28845.

I believe if we can eliminate the wrapper scripts, it will simplify the build and linking for AIX for the interpreter and C-extension modules.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue18235>
_______________________________________

From report at bugs.python.org  Tue Jan 24 12:07:26 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Tue, 24 Jan 2017 17:07:26 +0000
Subject: [issue29328] struct module should support variable-length strings
In-Reply-To: <1484850272.25.0.472309940141.issue29328@psf.upfronthosting.co.za>
Message-ID: <1485277646.52.0.194164147461.issue29328@psf.upfronthosting.co.za>


Changes by Raymond Hettinger <raymond.hettinger at gmail.com>:


----------
nosy: +twouters

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29328>
_______________________________________

From report at bugs.python.org  Tue Jan 24 12:17:02 2017
From: report at bugs.python.org (Marc-Andre Lemburg)
Date: Tue, 24 Jan 2017 17:17:02 +0000
Subject: [issue18235] _sysconfigdata.py wrong on AIX installations
In-Reply-To: <1485272855.67.0.133168651101.issue18235@psf.upfronthosting.co.za>
Message-ID: <a3e2d872-67ee-bd3e-6472-ba20dfc6cbac@egenix.com>


Marc-Andre Lemburg added the comment:

On 24.01.2017 16:47, Eric N. Vander Weele wrote:
> 
> I have a (large) patch that completely eliminates the need for ld_so_aix and makeexp_aix.  I've applied and been using this with for Python 2.7 and 3.5+, but I still need to go back and validate the tests to ensure everything passes as expected.
> 
> It's still a work in progress, but the patch can be found at https://github.com/ericvw/cpython/commit/e889f5fd04c1b73ef06e9f6e60108b2a7718b7d6.

Hmm, the patch seems to be incomplete, as it just removes
all the AIX support scripts and mentions, without adding
anything new to accommodate for the removal.

Please note that building with both xlc_r and gcc needs to
be supported.

> If this is out of context for this particular issue, I'm happy to raise a new issue to have a focus discussion about potentially landing this.  Ignore changes for README.AIX since I am addressing that in issue28845.

Please open a new issue for this to discuss this change there.

> I believe if we can eliminate the wrapper scripts, it will simplify the build and linking for AIX for the interpreter and C-extension modules.

Sure, if that's possible. The scripts are rather old and AIX
has moved on since those days.

FWIW: They still do work without any problems, provided the
Makefile and distutils can find them.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue18235>
_______________________________________

From report at bugs.python.org  Tue Jan 24 12:37:35 2017
From: report at bugs.python.org (Eric N. Vander Weele)
Date: Tue, 24 Jan 2017 17:37:35 +0000
Subject: [issue18235] _sysconfigdata.py wrong on AIX installations
In-Reply-To: <1371429201.94.0.372851990361.issue18235@psf.upfronthosting.co.za>
Message-ID: <1485279455.75.0.35405394387.issue18235@psf.upfronthosting.co.za>


Eric N. Vander Weele added the comment:

> Hmm, the patch seems to be incomplete, as it just removes all the AIX support scripts and mentions, without adding anything new to accommodate for the removal.

The new changes to accommodate for the script removal are in https://github.com/ericvw/cpython/commit/e889f5fd04c1b73ef06e9f6e60108b2a7718b7d6#diff-e2d5a00791bce9a01f99bc6fd613a39dR9159 and https://github.com/ericvw/cpython/commit/e889f5fd04c1b73ef06e9f6e60108b2a7718b7d6#diff-67e997bcfdac55191033d57a16d1408aR2403.

> Please note that building with both xlc_r and gcc needs to be supported.

I'll need to double check gcc, I have it working with xlc_r already.

> Please open a new issue for this to discuss this change there.

Absolutely, I'll clean up the patch and provide more details for the changes in another issue.  Thanks.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue18235>
_______________________________________

From report at bugs.python.org  Tue Jan 24 13:06:33 2017
From: report at bugs.python.org (Ethan Furman)
Date: Tue, 24 Jan 2017 18:06:33 +0000
Subject: [issue29363] allow Python code to determine class help text
Message-ID: <1485281193.78.0.719790622563.issue29363@psf.upfronthosting.co.za>


New submission from Ethan Furman:

>From issue29338, msg286139:
--------------------------
It is easy to fix the [pydoc] test by adding missed lines. But I'm not sure that output the (correct) signature of enum classes makes the help better.

    Color(value, names=None, *, module=None, qualname=None, type=None, start=1)

Ethan, what are your thoughts?

----------
components: Argument Clinic
messages: 286199
nosy: ethan.furman, larry, ncoghlan, rhettinger, serhiy.storchaka, xiang.zhang, yselivanov
priority: normal
severity: normal
status: open
title: allow Python code to determine class help text
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29363>
_______________________________________

From report at bugs.python.org  Tue Jan 24 13:08:21 2017
From: report at bugs.python.org (Ethan Furman)
Date: Tue, 24 Jan 2017 18:08:21 +0000
Subject: [issue29363] allow Python code to determine class help text
In-Reply-To: <1485281193.78.0.719790622563.issue29363@psf.upfronthosting.co.za>
Message-ID: <1485281301.45.0.353250529049.issue29363@psf.upfronthosting.co.za>


Ethan Furman added the comment:

That that is very unhelpful help text.  :(

Better would be:

    Color(value)

So how do we allow Python code to determine the help text?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29363>
_______________________________________

From report at bugs.python.org  Tue Jan 24 13:14:11 2017
From: report at bugs.python.org (Mark Dickinson)
Date: Tue, 24 Jan 2017 18:14:11 +0000
Subject: [issue29282] Fused multiply-add: proposal to add math.fma()
In-Reply-To: <1484560897.19.0.0955404368118.issue29282@psf.upfronthosting.co.za>
Message-ID: <1485281651.68.0.364924900359.issue29282@psf.upfronthosting.co.za>


Mark Dickinson added the comment:

The patch needs tests for the case where a*b overflows and c is infinite (either of the same sign as a*b or not). This combination should never return NaN, but a poor emulation of fma might do so.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29282>
_______________________________________

From report at bugs.python.org  Tue Jan 24 13:17:04 2017
From: report at bugs.python.org (Eric Fahlgren)
Date: Tue, 24 Jan 2017 18:17:04 +0000
Subject: [issue29282] Fused multiply-add: proposal to add math.fma()
In-Reply-To: <1484560897.19.0.0955404368118.issue29282@psf.upfronthosting.co.za>
Message-ID: <1485281824.7.0.127064831721.issue29282@psf.upfronthosting.co.za>


Changes by Eric Fahlgren <ericfahlgren at gmail.com>:


----------
nosy: +eric.fahlgren

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29282>
_______________________________________

From report at bugs.python.org  Tue Jan 24 13:40:07 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Tue, 24 Jan 2017 18:40:07 +0000
Subject: [issue29363] allow Python code to determine class help text
In-Reply-To: <1485281193.78.0.719790622563.issue29363@psf.upfronthosting.co.za>
Message-ID: <1485283207.83.0.870729520965.issue29363@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

This line is came from the signature of the __call__ method.

>>> import enum, inspect
>>> class A(enum.Enum):
...    x = 1
... 
>>> inspect.signature(A)
<Signature (value, names=None, *, module=None, qualname=None, type=None, start=1)>
>>> inspect.signature(A.__call__)
<Signature (value, names=None, *, module=None, qualname=None, type=None, start=1)>
>>> inspect.signature(enum.EnumMeta.__call__)
<Signature (cls, value, names=None, *, module=None, qualname=None, type=None, start=1)>

But calling A with optional arguments doesn't work.

>>> A(1)
<A.x: 1>
>>> A('B', {'y': 2})
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/serhiy/py/cpython/Lib/enum.py", line 293, in __call__
    return cls._create_(value, names, module=module, qualname=qualname, type=type, start=start)
  File "/home/serhiy/py/cpython/Lib/enum.py", line 378, in _create_
    _, first_enum = cls._get_mixins_(bases)
  File "/home/serhiy/py/cpython/Lib/enum.py", line 436, in _get_mixins_
    raise TypeError("Cannot extend enumerations")
TypeError: Cannot extend enumerations

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29363>
_______________________________________

From report at bugs.python.org  Tue Jan 24 13:50:12 2017
From: report at bugs.python.org (Yury Selivanov)
Date: Tue, 24 Jan 2017 18:50:12 +0000
Subject: [issue29358] Add tp_fastnew and tp_fastinit to PyTypeObject,
 15-20% faster object instanciation
In-Reply-To: <1485246329.89.0.254567368952.issue29358@psf.upfronthosting.co.za>
Message-ID: <1485283812.08.0.372118889228.issue29358@psf.upfronthosting.co.za>


Yury Selivanov added the comment:

Great. Can you share the Richards benchmark results?  It specifically stresses class instantiation, so we should be able to see the improvement.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29358>
_______________________________________

From report at bugs.python.org  Tue Jan 24 13:54:25 2017
From: report at bugs.python.org (Jason Matthew)
Date: Tue, 24 Jan 2017 18:54:25 +0000
Subject: [issue29364] msilib Fetch raises MSIError rather than return None
Message-ID: <1485284065.77.0.143549147327.issue29364@psf.upfronthosting.co.za>


New submission from Jason Matthew:

Fairly new to MSIs here.  Working towards extracting the same information which is available via Orca graphical interfaces.  

I believe I've hit a bug within _msi.View class, namely the inability to safely determine the number of records in my result.  

Examples I've found depicting how to obtain query results lead me to believe _msi.View.Fetch() will return None if no more rows exist.  My development using python 2.7.11, Windows 7 enterprise and 2.7.13, Windows Server 2016, has shown Fetch() will raise a msilib.MSIError when no more rows are available.  MSIError is commonly used within msilib which means explicit error handling (or leniency in this case) may incorrectly suppress other error scenarios.

Entering a bug with the hope either a patch addresses this issue or documentation can be updated to reflect proper usage.  Example provided below.


    >>> import msilib
    >>> db = msilib.OpenDatabase("msi-1.0.0.8.msi", msilib.MSIDBOPEN_READONLY)
    >>> v = db.OpenView("select * from Property")
    >>> v.Execute(None)
    >>> while 1:
    ...     r = v.Fetch()
    ...     if not r:break
    ...     print r.GetString(1)
    ...
    UpgradeCode
    WixSharp_UI_INSTALLDIR
    WixSharp_InstallDialogs
    WIXUI_INSTALLDIR
    Manufacturer
    ProductCode
    ProductLanguage
    ProductName
    ProductVersion
    SecureCustomProperties
    Traceback (most recent call last):
      File "<stdin>", line 2, in <module>
    _msi.MSIError: unknown error 103

----------
components: Library (Lib)
messages: 286204
nosy: Jason Matthew
priority: normal
severity: normal
status: open
title: msilib Fetch raises MSIError rather than return None
type: behavior
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29364>
_______________________________________

From report at bugs.python.org  Tue Jan 24 13:54:49 2017
From: report at bugs.python.org (Roundup Robot)
Date: Tue, 24 Jan 2017 18:54:49 +0000
Subject: [issue29337] BytesWarning at compile time
In-Reply-To: <1484982586.78.0.422468246503.issue29337@psf.upfronthosting.co.za>
Message-ID: <20170124185446.103093.66347.2490245A@psf.io>


Roundup Robot added the comment:

New changeset 2bd2ef1696cf by Serhiy Storchaka in branch '3.5':
Issue #29337: Fixed possible BytesWarning when compare the code objects.
https://hg.python.org/cpython/rev/2bd2ef1696cf

New changeset f6c327f2daa6 by Serhiy Storchaka in branch '3.6':
Issue #29337: Fixed possible BytesWarning when compare the code objects.
https://hg.python.org/cpython/rev/f6c327f2daa6

New changeset 00fb30d4905d by Serhiy Storchaka in branch 'default':
Issue #29337: Fixed possible BytesWarning when compare the code objects.
https://hg.python.org/cpython/rev/00fb30d4905d

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29337>
_______________________________________

From report at bugs.python.org  Tue Jan 24 13:55:22 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Tue, 24 Jan 2017 18:55:22 +0000
Subject: [issue29337] BytesWarning at compile time
In-Reply-To: <1484982586.78.0.422468246503.issue29337@psf.upfronthosting.co.za>
Message-ID: <1485284122.3.0.45042315577.issue29337@psf.upfronthosting.co.za>


Changes by Serhiy Storchaka <storchaka+cpython at gmail.com>:


----------
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29337>
_______________________________________

From report at bugs.python.org  Tue Jan 24 14:08:30 2017
From: report at bugs.python.org (Josh Rosenberg)
Date: Tue, 24 Jan 2017 19:08:30 +0000
Subject: [issue29352] provide the authorative source for s[i:j] negative slice
 indices (<-len(s)) behavior for standard sequences
In-Reply-To: <1485190156.24.0.631194950251.issue29352@psf.upfronthosting.co.za>
Message-ID: <1485284910.29.0.612565958962.issue29352@psf.upfronthosting.co.za>


Josh Rosenberg added the comment:

I think the wording could be improved, but there is another option I wanted to put here. Right now, we're being overly detailed about the implementation, specifying the bounds substitutions performed. If we're just trying to describe logical behavior, we could simplify footnote 4 to, dropping explicit descriptions for "out of bounds" cases, getting:

The slice of *s* from *i* to *j* is defined as the sequence of items with index *k* such that ``i <= k < j`` and ``0 <= k < len(s)``. If *i* is omitted or ``None``, use ``0``. If *j* is omitted or ``None``, use ``len(s)``. If *i* is greater than or equal to *j*, the slice is empty.

That avoids needing to be explicit about substitutions in the < -len(s) case and > len(s) cases, since limiting the values of k to the intersection of range(i, j) and range(len(s)) covers both ends. I considered a single range, like ``max(0, i) <= k < min(j, len(s))``, but that wouldn't describe the upper bound on i or the lower bound on j properly, and ``min(max(0, i), len(s)) <= k < min(max(0, j), len(s))`` is ugly. Footnote 3 covers the adjustment for negative values already, which allows for the simpler description.

----------
nosy: +josh.r

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29352>
_______________________________________

From report at bugs.python.org  Tue Jan 24 14:30:34 2017
From: report at bugs.python.org (Josh Rosenberg)
Date: Tue, 24 Jan 2017 19:30:34 +0000
Subject: [issue29358] Add tp_fastnew and tp_fastinit to PyTypeObject,
 15-20% faster object instanciation
In-Reply-To: <1485246329.89.0.254567368952.issue29358@psf.upfronthosting.co.za>
Message-ID: <1485286234.03.0.249321340141.issue29358@psf.upfronthosting.co.za>


Josh Rosenberg added the comment:

So just to be clear, the issue with non-string or non-unique keywords is purely about performance, right, not correctness/non-crashiness? Non-string keywords, while technically accepted by CPython, are at best barely legal by the language standard. The language standard for calls specifies that even using the **expression syntax "expression must evaluate to a mapping, the contents of which are treated as additional keyword arguments" and keywords_arguments only allows identifier=expression, where identifier is a legal variable name (and therefore, a str).

If misuse of ** unpacking is slower, but still works, and correct usage is significantly faster, I wouldn't consider an inability to fix the performance for the misuse case a blocking issue; nice to have, but making common, good code faster is worth making rare, bad code slower.

----------
nosy: +josh.r

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29358>
_______________________________________

From report at bugs.python.org  Tue Jan 24 14:34:00 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 24 Jan 2017 19:34:00 +0000
Subject: [issue29337] BytesWarning at compile time
In-Reply-To: <1485284122.33.0.374810262539.issue29337@psf.upfronthosting.co.za>
Message-ID: <CAMpsgwahkO4EXzh1xbJiif-sPoWCAp3d=oRJdJvZqj7h43xPag@mail.gmail.com>


STINNER Victor added the comment:

I'm surprised that removing the type from the key works. Since tests pass,
it's ok for me :-)

I'm happy that I spent time to write these tests, I'm now more confident
when this code is modified. Serhiy asked me to write these tests if I
recall correctly ;-)

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29337>
_______________________________________

From report at bugs.python.org  Tue Jan 24 14:40:05 2017
From: report at bugs.python.org (Roundup Robot)
Date: Tue, 24 Jan 2017 19:40:05 +0000
Subject: [issue29083] Readd PyArg_VaParse to the stable API
In-Reply-To: <1482846112.52.0.144835029316.issue29083@psf.upfronthosting.co.za>
Message-ID: <20170124194000.103147.62551.C7F4342F@psf.io>


Roundup Robot added the comment:

New changeset 5cdf383ef35c by Serhiy Storchaka in branch '3.5':
Issue #29083: Fixed the declaration of some public API functions.
https://hg.python.org/cpython/rev/5cdf383ef35c

New changeset 8eae65136000 by Serhiy Storchaka in branch '3.6':
Issue #29083: Fixed the declaration of some public API functions.
https://hg.python.org/cpython/rev/8eae65136000

New changeset b63d096403f4 by Serhiy Storchaka in branch 'default':
Issue #29083: Fixed the declaration of some public API functions.
https://hg.python.org/cpython/rev/b63d096403f4

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29083>
_______________________________________

From report at bugs.python.org  Tue Jan 24 14:43:24 2017
From: report at bugs.python.org (Ethan Furman)
Date: Tue, 24 Jan 2017 19:43:24 +0000
Subject: [issue29363] allow Python code to determine class help text
In-Reply-To: <1485281193.78.0.719790622563.issue29363@psf.upfronthosting.co.za>
Message-ID: <1485287004.05.0.598152703086.issue29363@psf.upfronthosting.co.za>


Ethan Furman added the comment:

There are actually two signatures:

EnumCls(value) --> return member with value `value`

EnumCls(name, members, module, qualname, type, start) --> create new Enum

An example of the first:

class A(Enum):
    x = 1
A(1) --> <A.x: 1>

an example of the second:

class A(Enum):
    pass
B = A('B', {'y':2})
B(2) --> <B.y: 2>

The reason for the error you see is that Enums with members cannot be further subclassed.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29363>
_______________________________________

From report at bugs.python.org  Tue Jan 24 14:48:49 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Tue, 24 Jan 2017 19:48:49 +0000
Subject: [issue29363] allow Python code to determine class help text
In-Reply-To: <1485281193.78.0.719790622563.issue29363@psf.upfronthosting.co.za>
Message-ID: <1485287329.49.0.138978951157.issue29363@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

I'm wondering is it worth to set different __call__ methods for enum types with and without members?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29363>
_______________________________________

From report at bugs.python.org  Tue Jan 24 14:58:44 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Tue, 24 Jan 2017 19:58:44 +0000
Subject: [issue28810] Document bytecode changes in 3.6
In-Reply-To: <1480184644.53.0.660721117392.issue28810@psf.upfronthosting.co.za>
Message-ID: <1485287924.73.0.947764999468.issue28810@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

Several third-party bytecode manipulating projects still are not updated to 3.6. Correct documentation is needed for them. Could anyone please make a review of the patch (or maybe totally rewrite it)?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28810>
_______________________________________

From report at bugs.python.org  Tue Jan 24 15:14:24 2017
From: report at bugs.python.org (Roundup Robot)
Date: Tue, 24 Jan 2017 20:14:24 +0000
Subject: [issue29167] Race condition in enum.py:_decompose()
In-Reply-To: <1483613666.47.0.301459916652.issue29167@psf.upfronthosting.co.za>
Message-ID: <20170124201418.43868.83752.2D0CEBB1@psf.io>


Roundup Robot added the comment:

New changeset 95e184bd2d89 by Ethan Furman in branch '3.6':
closes issue29167: fix race condition in (Int)Flag
https://hg.python.org/cpython/rev/95e184bd2d89

New changeset e6b98c270718 by Ethan Furman in branch 'default':
issue29167: fix race condition in (Int)Flag
https://hg.python.org/cpython/rev/e6b98c270718

----------
nosy: +python-dev
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29167>
_______________________________________

From report at bugs.python.org  Tue Jan 24 15:27:11 2017
From: report at bugs.python.org (Ethan Furman)
Date: Tue, 24 Jan 2017 20:27:11 +0000
Subject: [issue29363] allow Python code to determine class help text
In-Reply-To: <1485281193.78.0.719790622563.issue29363@psf.upfronthosting.co.za>
Message-ID: <1485289631.65.0.180253826724.issue29363@psf.upfronthosting.co.za>


Ethan Furman added the comment:

Probably not because an enum class' __call__ comes from the type EnumMeta -- so having two different __call__ methods would mean two different metaclasses, and I'm pretty sure I don't want to go there.  ;)

There is a __doc__ defined for the __call__ method, though -- is there a reason inspect isn't using that?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29363>
_______________________________________

From report at bugs.python.org  Tue Jan 24 15:42:16 2017
From: report at bugs.python.org (Brett Cannon)
Date: Tue, 24 Jan 2017 20:42:16 +0000
Subject: [issue28810] Document bytecode changes in 3.6
In-Reply-To: <1480184644.53.0.660721117392.issue28810@psf.upfronthosting.co.za>
Message-ID: <1485290536.74.0.540132426994.issue28810@psf.upfronthosting.co.za>


Changes by Brett Cannon <brett at python.org>:


----------
assignee: docs at python -> brett.cannon

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28810>
_______________________________________

From report at bugs.python.org  Tue Jan 24 15:42:29 2017
From: report at bugs.python.org (Brett Cannon)
Date: Tue, 24 Jan 2017 20:42:29 +0000
Subject: [issue28810] Document bytecode changes in 3.6
In-Reply-To: <1480184644.53.0.660721117392.issue28810@psf.upfronthosting.co.za>
Message-ID: <1485290549.63.0.800067571715.issue28810@psf.upfronthosting.co.za>


Brett Cannon added the comment:

I'll review your patch sometime this week, Serhiy.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28810>
_______________________________________

From report at bugs.python.org  Tue Jan 24 16:03:35 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 24 Jan 2017 21:03:35 +0000
Subject: [issue29358] Add tp_fastnew and tp_fastinit to PyTypeObject,
 15-20% faster object instanciation
In-Reply-To: <1485246329.89.0.254567368952.issue29358@psf.upfronthosting.co.za>
Message-ID: <1485291815.89.0.61536042168.issue29358@psf.upfronthosting.co.za>


STINNER Victor added the comment:

First benchmark result on the fast_init compared to the default branch. Warning: the patch is still a WIP, there are maybe performance issues (in this case, this benchmark should help to identify them).
---
haypo at speed-python$ python3 -m perf compare_to 2017-01-23_14-02-default-cebc9c7ad195.json fast_init-0_ref_cebc9c7ad195.json -G --min-speed=2
Slower (6):
- telco: 13.0 ms +- 0.2 ms -> 13.8 ms +- 0.3 ms: 1.06x slower (+6%)
- scimark_sor: 391 ms +- 7 ms -> 413 ms +- 11 ms: 1.06x slower (+6%)
- fannkuch: 938 ms +- 7 ms -> 973 ms +- 13 ms: 1.04x slower (+4%)
- pickle_dict: 53.7 us +- 3.4 us -> 55.5 us +- 5.0 us: 1.03x slower (+3%)
- sqlite_synth: 8.00 us +- 0.12 us -> 8.25 us +- 0.14 us: 1.03x slower (+3%)
- scimark_monte_carlo: 202 ms +- 5 ms -> 207 ms +- 5 ms: 1.03x slower (+3%)

Faster (19):
- scimark_lu: 362 ms +- 13 ms -> 335 ms +- 17 ms: 1.08x faster (-8%)
- chameleon: 23.5 ms +- 0.3 ms -> 21.7 ms +- 0.3 ms: 1.08x faster (-7%)
- xml_etree_process: 179 ms +- 3 ms -> 168 ms +- 3 ms: 1.07x faster (-6%)
- pickle_pure_python: 1.03 ms +- 0.02 ms -> 982 us +- 11 us: 1.05x faster (-5%)
- regex_dna: 283 ms +- 1 ms -> 271 ms +- 1 ms: 1.05x faster (-4%)
- chaos: 239 ms +- 2 ms -> 230 ms +- 3 ms: 1.04x faster (-4%)
- django_template: 356 ms +- 5 ms -> 343 ms +- 5 ms: 1.04x faster (-4%)
- call_simple: 10.7 ms +- 0.4 ms -> 10.3 ms +- 0.3 ms: 1.04x faster (-3%)
- hexiom: 18.0 ms +- 0.1 ms -> 17.4 ms +- 0.1 ms: 1.04x faster (-3%)
- pathlib: 42.3 ms +- 0.5 ms -> 40.9 ms +- 0.4 ms: 1.03x faster (-3%)
- regex_compile: 379 ms +- 2 ms -> 367 ms +- 4 ms: 1.03x faster (-3%)
- go: 501 ms +- 7 ms -> 485 ms +- 4 ms: 1.03x faster (-3%)
- xml_etree_generate: 208 ms +- 4 ms -> 202 ms +- 3 ms: 1.03x faster (-3%)
- deltablue: 14.5 ms +- 0.3 ms -> 14.1 ms +- 0.2 ms: 1.03x faster (-3%)
- unpickle_pure_python: 652 us +- 8 us -> 634 us +- 9 us: 1.03x faster (-3%)
- richards: 147 ms +- 2 ms -> 143 ms +- 4 ms: 1.03x faster (-3%)
- nqueens: 213 ms +- 2 ms -> 208 ms +- 2 ms: 1.03x faster (-2%)
- raytrace: 1.08 sec +- 0.01 sec -> 1.05 sec +- 0.02 sec: 1.02x faster (-2%)
- unpickle: 31.0 us +- 0.4 us -> 30.3 us +- 1.0 us: 1.02x faster (-2%)

Benchmark hidden because not significant (34): (...)
Ignored benchmarks (5) of 2017-01-23_14-02-default-cebc9c7ad195.json: mako, sympy_expand, sympy_integrate, sympy_str, sympy_sum
---

Reminder: The fast_init branch is based on the issue #29259 (tp_fastcall), so the result combines tp_fastcall and tp_fastnew/fastinit.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29358>
_______________________________________

From report at bugs.python.org  Tue Jan 24 16:09:21 2017
From: report at bugs.python.org (Aviv Palivoda)
Date: Tue, 24 Jan 2017 21:09:21 +0000
Subject: [issue29121] sqlite3 Controlling Transactions documentation not
 updated
In-Reply-To: <1483200295.1.0.801871810477.issue29121@psf.upfronthosting.co.za>
Message-ID: <1485292161.61.0.752648142801.issue29121@psf.upfronthosting.co.za>


Aviv Palivoda added the comment:

I would just like to note that I think that the correct solution is to do an implicit commit before:
1. VACUUM
2. ATTACH
3. DETACH
4. BEGIN

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29121>
_______________________________________

From report at bugs.python.org  Tue Jan 24 16:15:09 2017
From: report at bugs.python.org (Eric N. Vander Weele)
Date: Tue, 24 Jan 2017 21:15:09 +0000
Subject: [issue28845] Clean up known issues for AIX
In-Reply-To: <1480542032.97.0.55863701863.issue28845@psf.upfronthosting.co.za>
Message-ID: <1485292509.41.0.6477115956.issue28845@psf.upfronthosting.co.za>


Eric N. Vander Weele added the comment:

> I request that you review issue27435 - in particular msg284557 - as I feel ctypes implementation for AIX is broken - at least as far as the supporting routines are concerned.

I believe this request is outside the scope of this particular issue.

> [...] AIX to test it.

It appears there is an AIX system in Python's buildbot farm to validate the curses example.  Is the next step have a core maintainer conduct the validation on an AIX system before this can move forward?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28845>
_______________________________________

From report at bugs.python.org  Tue Jan 24 16:46:42 2017
From: report at bugs.python.org (Alexandre Vassalotti)
Date: Tue, 24 Jan 2017 21:46:42 +0000
Subject: [issue17720] pickle.py's load_appends should call append() on objects
 other than lists
In-Reply-To: <1365876289.91.0.454785839611.issue17720@psf.upfronthosting.co.za>
Message-ID: <1485294402.79.0.216404100634.issue17720@psf.upfronthosting.co.za>


Alexandre Vassalotti added the comment:

I don't recall where I picked up the referred spec. Looking at PEP 307 and pickle docs [1], it does look like that objects which provides the listitems field of the reduce tuple should support extend() too.

It might best to try both in the implementation to avoid breaking existing code that relies on the previous behavior.

[1]: https://docs.python.org/3/library/pickle.html#object.__reduce__

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue17720>
_______________________________________

From report at bugs.python.org  Tue Jan 24 17:30:33 2017
From: report at bugs.python.org (Roundup Robot)
Date: Tue, 24 Jan 2017 22:30:33 +0000
Subject: [issue29289] Convert OrderedDict methods to Argument Clinic
In-Reply-To: <1484609084.15.0.406928886689.issue29289@psf.upfronthosting.co.za>
Message-ID: <20170124223029.21787.56564.A262AE81@psf.io>


Roundup Robot added the comment:

New changeset ffc0840762e4 by Serhiy Storchaka in branch 'default':
Issues #29311, #29289: Fixed and improved docstrings for dict and OrderedDict
https://hg.python.org/cpython/rev/ffc0840762e4

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29289>
_______________________________________

From report at bugs.python.org  Tue Jan 24 17:30:33 2017
From: report at bugs.python.org (Roundup Robot)
Date: Tue, 24 Jan 2017 22:30:33 +0000
Subject: [issue29311] Argument Clinic: convert dict methods
In-Reply-To: <1484758450.31.0.717694851278.issue29311@psf.upfronthosting.co.za>
Message-ID: <20170124223029.21787.69545.27634F7B@psf.io>


Roundup Robot added the comment:

New changeset ffc0840762e4 by Serhiy Storchaka in branch 'default':
Issues #29311, #29289: Fixed and improved docstrings for dict and OrderedDict
https://hg.python.org/cpython/rev/ffc0840762e4

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29311>
_______________________________________

From report at bugs.python.org  Tue Jan 24 17:31:46 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Tue, 24 Jan 2017 22:31:46 +0000
Subject: [issue29311] Argument Clinic: convert dict methods
In-Reply-To: <1484758450.31.0.717694851278.issue29311@psf.upfronthosting.co.za>
Message-ID: <1485297106.6.0.357026894956.issue29311@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

Thank you for your review Martin.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29311>
_______________________________________

From report at bugs.python.org  Tue Jan 24 18:33:07 2017
From: report at bugs.python.org (paul j3)
Date: Tue, 24 Jan 2017 23:33:07 +0000
Subject: [issue14074] argparse allows nargs>1 for positional arguments but
 doesn't allow metavar to be a tuple
In-Reply-To: <1329845918.75.0.31962840333.issue14074@psf.upfronthosting.co.za>
Message-ID: <1485300787.74.0.838880656085.issue14074@psf.upfronthosting.co.za>


Changes by paul j3 <ajipanca at gmail.com>:


----------
priority: normal -> high

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue14074>
_______________________________________

From report at bugs.python.org  Tue Jan 24 21:30:14 2017
From: report at bugs.python.org (Gregory P. Smith)
Date: Wed, 25 Jan 2017 02:30:14 +0000
Subject: [issue29365] parallel make test or make install can fail building
 pgen when configure --enable-optimizations is used
Message-ID: <1485311414.76.0.958809336193.issue29365@psf.upfronthosting.co.za>


New submission from Gregory P. Smith:

If you do a make -j12 test or make install before having run a separate 'make all' the build _can_ fail trying to build pgen.  (it seems to be parallel make related, it won't always fail)

Even if you have run a separate 'make all', the make test and make install steps are likely to rebuild a ton of stuff and rerun the profile generation stage.  Making the suggested "make, make test, make install" process effectively useless or at least take 3x longer than it should.

something along the lines of:

oss/cpython$ make -j12 install
gcc -pthread -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes  Parser/acceler.o Parser/grammar1.o Parser/listnode.o Parser/node.o Parser/parser.o Parser/parsetok.o Parser/bitset.o Parser/metagrammar.o Parser/firstsets.o Parser/grammar.o Parser/pgen.o Objects/obmalloc.o Python/mysnprintf.o Python/pyctype.o Parser/tokenizer_pgen.o Parser/printgrammar.o Parser/pgenmain.o -lpthread -ldl  -lutil -o Parser/pgen
Parser/grammar1.o: In function `PyGrammar_LabelRepr':
/home/gps/oss/cpython/build27/../2.7/Parser/grammar1.c:35: undefined reference to `__gcov_indirect_call_profiler'
Parser/grammar1.o: In function `PyGrammar_FindDFA':
/home/gps/oss/cpython/build27/../2.7/Parser/grammar1.c:13: undefined reference to `__gcov_indirect_call_profiler'
Parser/grammar1.o:(.data+0x80): undefined reference to `__gcov_merge_add'
Parser/grammar1.o: In function `_GLOBAL__sub_I_65535_0_PyGrammar_FindDFA':
/home/gps/oss/cpython/build27/../2.7/Parser/grammar1.c:57: undefined reference to `__gcov_init'
... a bunch more similar errors omitted ...
Parser/pgenmain.o: In function `_GLOBAL__sub_I_65535_0_Py_Exit':
/home/gps/oss/cpython/build27/../2.7/Parser/pgenmain.c:173: undefined reference to `__gcov_init'
collect2: error: ld returned 1 exit status
make: *** [Parser/pgen] Error 1
make: *** Waiting for unfinished jobs....
Parser/pgen ../2.7/Grammar/Grammar Include/graminit.h Python/graminit.c
make[2]: Parser/pgen: Command not found
make[2]: *** [Include/graminit.h] Error 127
make[2]: *** Waiting for unfinished jobs....
make[2]: Leaving directory `/home/gps/oss/cpython/build27'
make[1]: *** [build_all_generate_profile] Error 2
make[1]: Leaving directory `/home/gps/oss/cpython/build27'
make: *** [profile-opt] Error 2

----------
components: Build
messages: 286223
nosy: gregory.p.smith
priority: normal
severity: normal
status: open
title: parallel make test or make install can fail building pgen when configure --enable-optimizations is used
type: compile error
versions: Python 2.7, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29365>
_______________________________________

From report at bugs.python.org  Tue Jan 24 21:35:00 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Wed, 25 Jan 2017 02:35:00 +0000
Subject: [issue29365] parallel make test or make install can fail building
 pgen when configure --enable-optimizations is used
In-Reply-To: <1485311414.76.0.958809336193.issue29365@psf.upfronthosting.co.za>
Message-ID: <1485311700.59.0.754929162775.issue29365@psf.upfronthosting.co.za>


Xiang Zhang added the comment:

> Making the suggested "make, make test, make install" process effectively useless or at least take 3x longer than it should.

About this, I once opened #29243.

----------
nosy: +xiang.zhang

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29365>
_______________________________________

From report at bugs.python.org  Tue Jan 24 21:43:12 2017
From: report at bugs.python.org (raylu)
Date: Wed, 25 Jan 2017 02:43:12 +0000
Subject: [issue29366] os.listdir has inconsistent behavior when run on a
 non-directory
Message-ID: <1485312192.81.0.934042138535.issue29366@psf.upfronthosting.co.za>


New submission from raylu:

According to
https://github.com/python/cpython/blob/4ed71dae9d23a412f9f73d3a0b1f4be37543e49e/Lib/test/test_unicode_file_functions.py#L106
listdir can sometimes return FileNotFoundError on Windows.

Tangentially related: https://hg.python.org/cpython/rev/385c2ec78f16

----------
components: Library (Lib), Windows
messages: 286225
nosy: paul.moore, raylu, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: os.listdir has inconsistent behavior when run on a non-directory

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29366>
_______________________________________

From report at bugs.python.org  Tue Jan 24 22:39:38 2017
From: report at bugs.python.org (Jelle Zijlstra)
Date: Wed, 25 Jan 2017 03:39:38 +0000
Subject: [issue29310] Document typing.NamedTuple default argument syntax
In-Reply-To: <1484755757.54.0.970081096642.issue29310@psf.upfronthosting.co.za>
Message-ID: <1485315578.3.0.219570854499.issue29310@psf.upfronthosting.co.za>


Changes by Jelle Zijlstra <jelle.zijlstra at gmail.com>:


----------
keywords: +patch
Added file: http://bugs.python.org/file46410/issue29310.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29310>
_______________________________________

From report at bugs.python.org  Tue Jan 24 22:39:47 2017
From: report at bugs.python.org (Jelle Zijlstra)
Date: Wed, 25 Jan 2017 03:39:47 +0000
Subject: [issue29310] Document typing.NamedTuple default argument syntax
In-Reply-To: <1484755757.54.0.970081096642.issue29310@psf.upfronthosting.co.za>
Message-ID: <1485315587.6.0.536415032918.issue29310@psf.upfronthosting.co.za>


Changes by Jelle Zijlstra <jelle.zijlstra at gmail.com>:


----------
assignee: Jelle Zijlstra -> 

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29310>
_______________________________________

From report at bugs.python.org  Tue Jan 24 23:26:47 2017
From: report at bugs.python.org (INADA Naoki)
Date: Wed, 25 Jan 2017 04:26:47 +0000
Subject: [issue29243] --enable-optimizations makes common build commands
 always need to compile from scratch
In-Reply-To: <1484151314.82.0.308427338149.issue29243@psf.upfronthosting.co.za>
Message-ID: <1485318407.67.0.681436301578.issue29243@psf.upfronthosting.co.za>


INADA Naoki added the comment:

That's why I want to enable only --with-lto

----------
nosy: +inada.naoki

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29243>
_______________________________________

From report at bugs.python.org  Wed Jan 25 00:38:09 2017
From: report at bugs.python.org (paul j3)
Date: Wed, 25 Jan 2017 05:38:09 +0000
Subject: [issue14074] argparse allows nargs>1 for positional arguments but
 doesn't allow metavar to be a tuple
In-Reply-To: <1329845918.75.0.31962840333.issue14074@psf.upfronthosting.co.za>
Message-ID: <1485322689.78.0.710185023997.issue14074@psf.upfronthosting.co.za>


paul j3 added the comment:

An alternative fix is to disallow tuple metavars for positionals.  

A tuple metavar may look nice in the usage.  But a well selected dest seems better in the help line, and error messages.  Using dest in all 3 roles, as we do now, is the simplest compromise.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue14074>
_______________________________________

From report at bugs.python.org  Wed Jan 25 01:18:45 2017
From: report at bugs.python.org (Gregory P. Smith)
Date: Wed, 25 Jan 2017 06:18:45 +0000
Subject: [issue29365] parallel make test or make install can fail building
 pgen when configure --enable-optimizations is used
In-Reply-To: <1485311414.76.0.958809336193.issue29365@psf.upfronthosting.co.za>
Message-ID: <1485325125.06.0.245608442556.issue29365@psf.upfronthosting.co.za>


Gregory P. Smith added the comment:

great, the issue you filed covers my second paragraph comment which really should be its own issue. :)

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29365>
_______________________________________

From report at bugs.python.org  Wed Jan 25 01:19:02 2017
From: report at bugs.python.org (Gregory P. Smith)
Date: Wed, 25 Jan 2017 06:19:02 +0000
Subject: [issue29243] --enable-optimizations makes common build commands
 always need to compile from scratch
In-Reply-To: <1484151314.82.0.308427338149.issue29243@psf.upfronthosting.co.za>
Message-ID: <1485325142.74.0.526846822407.issue29243@psf.upfronthosting.co.za>


Changes by Gregory P. Smith <greg at krypto.org>:


----------
nosy: +gregory.p.smith

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29243>
_______________________________________

From report at bugs.python.org  Wed Jan 25 01:59:21 2017
From: report at bugs.python.org (Ma Lin)
Date: Wed, 25 Jan 2017 06:59:21 +0000
Subject: [issue29121] sqlite3 Controlling Transactions documentation not
 updated
In-Reply-To: <1483200295.1.0.801871810477.issue29121@psf.upfronthosting.co.za>
Message-ID: <1485327561.33.0.0711322596313.issue29121@psf.upfronthosting.co.za>


Changes by Ma Lin <malincns at 163.com>:


----------
nosy: +Ma Lin

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29121>
_______________________________________

From report at bugs.python.org  Wed Jan 25 02:00:47 2017
From: report at bugs.python.org (INADA Naoki)
Date: Wed, 25 Jan 2017 07:00:47 +0000
Subject: [issue29336] merge tuples in module
In-Reply-To: <1484969445.33.0.406694538272.issue29336@psf.upfronthosting.co.za>
Message-ID: <1485327647.31.0.68734663169.issue29336@psf.upfronthosting.co.za>


INADA Naoki added the comment:

> More generally, would it be possible to share co_consts (None,) tuples between code objects of two different modules? Or is it already the case with merge-constants.patch?

No. I didn't do it because I don't know it's worth enough.

> merge-constants.patch looks simple enought, but I'm not really impressed by such result. Is 2% worth it?

Maybe, our application is somewhat special.
I'll check memory usage again with only loading major OSS framework/libraries for this issue in next time.
(BTW, it's 3% if -OO is used)

But issue28813 or other optimizations may reduce tuple size and make it possible to share more tuples.
So I think we should evaluate this patch after them.

But I don't familiar with frontend (parser, AST).
I usually look bytecode and runtime.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29336>
_______________________________________

From report at bugs.python.org  Wed Jan 25 02:45:20 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Wed, 25 Jan 2017 07:45:20 +0000
Subject: [issue29336] merge tuples in module
In-Reply-To: <1484969445.33.0.406694538272.issue29336@psf.upfronthosting.co.za>
Message-ID: <1485330319.99.0.132630530539.issue29336@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

> merge-constants.patch looks simple enought, but I'm not really impressed by such result. Is 2% worth it?

I'm not impressed too. And merging constants can take time.

> Since code objects loaded by import are likely for stay for the whole lifetime of a process, I would be interested to experiment interning all constant objects (the tuple of objects, but also each object of these tuples, so support "nested interning") of all code objets (consts, names, varnames, freevars, cellvars) in the marshal module.

This could slow down marhalling. Some patches in issue20416 implements this, but current code is faster and much simpler and the benefit of deduplicating constants is too small.

> More generally, would it be possible to share co_consts (None,) tuples between code objects of two different modules? Or is it already the case with merge-constants.patch?

All patches in this issue shares constants only in one compilation unit (module). Otherwise the global intern dict would increase the memory usage too much.

See also the statistics in issue16475. The most memory is spent by strings, and the majority of strings already are interned.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29336>
_______________________________________

From report at bugs.python.org  Wed Jan 25 02:51:54 2017
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 25 Jan 2017 07:51:54 +0000
Subject: [issue29336] merge tuples in module
In-Reply-To: <1484969445.33.0.406694538272.issue29336@psf.upfronthosting.co.za>
Message-ID: <1485330714.82.0.0899269025843.issue29336@psf.upfronthosting.co.za>


STINNER Victor added the comment:

I vote +0 on merge-constants.patch: LGTM, may be useful, but not impressed :-)

Hopefully, .py => .pyc compilation "should" occur only once. Except of the main script, but more and more applications use a tiny entry point of 3 lines. I don't expect any significant slowdown. But even if there is a slowdown, IMHO it's worth it.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29336>
_______________________________________

From report at bugs.python.org  Wed Jan 25 04:07:10 2017
From: report at bugs.python.org (Ivan Levkivskyi)
Date: Wed, 25 Jan 2017 09:07:10 +0000
Subject: [issue29310] Document typing.NamedTuple default argument syntax
In-Reply-To: <1484755757.54.0.970081096642.issue29310@psf.upfronthosting.co.za>
Message-ID: <1485335230.1.0.469780470052.issue29310@psf.upfronthosting.co.za>


Ivan Levkivskyi added the comment:

Thank you for the patch! I added few comments.

----------
stage:  -> patch review

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29310>
_______________________________________

From report at bugs.python.org  Wed Jan 25 04:29:36 2017
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 25 Jan 2017 09:29:36 +0000
Subject: [issue29367] python-gdb: display wrapper_call()
Message-ID: <1485336576.45.0.391174330106.issue29367@psf.upfronthosting.co.za>


New submission from STINNER Victor:

python-gdb.py handles various Python calls, but not descriptors. Attached patch adds code to display calls to wrapperobject objects.

The patch adds the first "<method-wrapper ...>" line of the traceback:
---
(gdb) py-bt
Traceback (most recent call first):
  <method-wrapper '__init__' of CodecInfo object at 0x7ffff7e9d7a8>
  File "/home/haypo/prog/python/git_cpython/Lib/encodings/utf_8.py", line 41, in getregentry
    streamwriter=StreamWriter,
  File "/home/haypo/prog/python/git_cpython/Lib/encodings/__init__.py", line 122, in search_function
    entry = getregentry()
---

I wrote this patch while working on the issue #29358. Object instanciation is full of wrappers. A better debugger helps to me understand my bugs ;-)

Example to get such tracecback:
---
haypo at selma$ gdb -args ./python -m test -v test_sys
(gdb) b wrapper_call
(gdb) run
(gdb) py-bt
Traceback (most recent call first):
  <method-wrapper '__init__' of _EnumDict object at 0x7ffff023d388>
  File "/home/haypo/prog/python/default/Lib/enum.py", line 66, in __init__
    super().__init__()
  File "/home/haypo/prog/python/default/Lib/enum.py", line 117, in __prepare__
    enum_dict = _EnumDict()
  <built-in method __build_class__ of module object at remote 0x7ffff7f46c58>
  File "/home/haypo/prog/python/default/Lib/enum.py", line 509, in <module>
    class Enum(metaclass=EnumMeta):
  <built-in method exec of module object at remote 0x7ffff7f46c58>
  File "<frozen importlib._bootstrap>", line 205, in _call_with_frames_removed
  (...)
---

I guess that the next step is to write a test ;-)

----------
files: gdb_wrapper_call.patch
keywords: patch
messages: 286233
nosy: haypo, inada.naoki
priority: normal
severity: normal
status: open
title: python-gdb: display wrapper_call()
type: enhancement
versions: Python 3.7
Added file: http://bugs.python.org/file46411/gdb_wrapper_call.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29367>
_______________________________________

From report at bugs.python.org  Wed Jan 25 04:46:51 2017
From: report at bugs.python.org (Roundup Robot)
Date: Wed, 25 Jan 2017 09:46:51 +0000
Subject: [issue26149] Suggest PyCharm Community as an editor for Unix platforms
In-Reply-To: <1453153728.95.0.733172914649.issue26149@psf.upfronthosting.co.za>
Message-ID: <20170125094648.20918.73286.A917AEE8@psf.io>


Roundup Robot added the comment:

New changeset b43e270648a3 by Senthil Kumaran in branch '2.7':
issue26149 - Point to Wiki for Editors and Python IDEs on Unix.
https://hg.python.org/cpython/rev/b43e270648a3

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue26149>
_______________________________________

From report at bugs.python.org  Wed Jan 25 04:48:53 2017
From: report at bugs.python.org (Roundup Robot)
Date: Wed, 25 Jan 2017 09:48:53 +0000
Subject: [issue26149] Suggest PyCharm Community as an editor for Unix platforms
In-Reply-To: <1453153728.95.0.733172914649.issue26149@psf.upfronthosting.co.za>
Message-ID: <20170125094851.21040.44395.6D18BB85@psf.io>


Roundup Robot added the comment:

New changeset 25fc68e22eee by Senthil Kumaran in branch '3.6':
issue26149 - Point to Wiki for Editors and Python IDEs on Unix.
https://hg.python.org/cpython/rev/25fc68e22eee

New changeset 0fd4a3d8e32a by Senthil Kumaran in branch 'default':
[merge 3.6] - issue26149 - Point to Wiki for Editors and Python IDEs on Unix.
https://hg.python.org/cpython/rev/0fd4a3d8e32a

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue26149>
_______________________________________

From report at bugs.python.org  Wed Jan 25 04:49:26 2017
From: report at bugs.python.org (Senthil Kumaran)
Date: Wed, 25 Jan 2017 09:49:26 +0000
Subject: [issue26149] Suggest PyCharm Community as an editor for Unix platforms
In-Reply-To: <1453153728.95.0.733172914649.issue26149@psf.upfronthosting.co.za>
Message-ID: <1485337766.6.0.361950764566.issue26149@psf.upfronthosting.co.za>


Senthil Kumaran added the comment:

Thank you, Mariatta.

----------
nosy: +orsenthil
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue26149>
_______________________________________

From report at bugs.python.org  Wed Jan 25 04:55:50 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Wed, 25 Jan 2017 09:55:50 +0000
Subject: [issue29368] Optimize unpickling list-like objects
Message-ID: <1485338150.33.0.560817507263.issue29368@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka:

According to PEP 307 the extend method can be used for appending list items to the object.

    listitems    Optional, and new in this PEP.
                 If this is not None, it should be an iterator (not a
                 sequence!) yielding successive list items.  These list
                 items will be pickled, and appended to the object using
                 either obj.append(item) or obj.extend(list_of_items).
                 This is primarily used for list subclasses, but may
                 be used by other classes as long as they have append()
                 and extend() methods with the appropriate signature.
                 (Whether append() or extend() is used depends on which
                 pickle protocol version is used as well as the number
                 of items to append, so both must be supported.)

Proposed patch makes the extend method be used in the APPENDS opcode. To avoid breaking existing code the use of the extend method is optional.

Microbenchmark:

$ ./python -m timeit -s "import pickle, collections; p = pickle.dumps(collections.deque([None]*10000), 4)" -- "pickle.loads(p)"
Unpatched:  100 loops, best of 5: 2.02 msec per loop
Patched:    500 loops, best of 5: 833 usec per loop

----------
components: Library (Lib)
files: pickle-appends-extend.patch
keywords: patch
messages: 286237
nosy: alexandre.vassalotti, serhiy.storchaka
priority: normal
severity: normal
stage: patch review
status: open
title: Optimize unpickling list-like objects
type: performance
versions: Python 3.7
Added file: http://bugs.python.org/file46412/pickle-appends-extend.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29368>
_______________________________________

From report at bugs.python.org  Wed Jan 25 04:56:40 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Wed, 25 Jan 2017 09:56:40 +0000
Subject: [issue17720] pickle.py's load_appends should call append() on objects
 other than lists
In-Reply-To: <1365876289.91.0.454785839611.issue17720@psf.upfronthosting.co.za>
Message-ID: <1485338200.85.0.618321566918.issue17720@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

Thanks. Opened issue29368 for this.

----------
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue17720>
_______________________________________

From report at bugs.python.org  Wed Jan 25 05:12:03 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Wed, 25 Jan 2017 10:12:03 +0000
Subject: [issue29083] Readd PyArg_VaParse to the stable API
In-Reply-To: <1482846112.52.0.144835029316.issue29083@psf.upfronthosting.co.za>
Message-ID: <1485339123.34.0.642441641155.issue29083@psf.upfronthosting.co.za>


Changes by Serhiy Storchaka <storchaka+cpython at gmail.com>:


----------
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29083>
_______________________________________

From report at bugs.python.org  Wed Jan 25 05:42:03 2017
From: report at bugs.python.org (Cory Benfield)
Date: Wed, 25 Jan 2017 10:42:03 +0000
Subject: [issue29136] Add OP_NO_TLSv1_3
In-Reply-To: <1483391244.76.0.29431388133.issue29136@psf.upfronthosting.co.za>
Message-ID: <1485340923.96.0.0182141196207.issue29136@psf.upfronthosting.co.za>


Cory Benfield added the comment:

For those who want to keep track, the relevant OpenSSL ticket for configuring TLSv1.3 cipher suites is https://github.com/openssl/openssl/issues/2276.

----------
nosy: +Lukasa

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29136>
_______________________________________

From report at bugs.python.org  Wed Jan 25 05:42:30 2017
From: report at bugs.python.org (Julian Berman)
Date: Wed, 25 Jan 2017 10:42:30 +0000
Subject: [issue29182] Remove the warning in urllib docs that it doesn't do
 certificate validate by default.
In-Reply-To: <1483722117.81.0.319313681571.issue29182@psf.upfronthosting.co.za>
Message-ID: <1485340950.75.0.522074873053.issue29182@psf.upfronthosting.co.za>


Julian Berman added the comment:

The change note belongs outside the seealso.

I think also that wasn't exactly what Martin had in mind, I think he meant a `.. versionchanged` directive -- and given that this was originally a warning, personally I'd leave the warning but reword it, it's still relevant for pre-2.7.9 users.

E.g. something like:

.. seealso::

   <requests module info>

.. versionchanged:: 2.7.9

    HTTPS certificates now have hostname verification enabled by default.

.. warning::

    Versions prior to 2.7.9 do not safely verify hostnames. It is not recommended to use this module on these versions -- the aformentioned requests module can be used instead.

Or the like.

----------
nosy: +Julian

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29182>
_______________________________________

From report at bugs.python.org  Wed Jan 25 06:29:12 2017
From: report at bugs.python.org (Roundup Robot)
Date: Wed, 25 Jan 2017 11:29:12 +0000
Subject: [issue27867] various issues due to misuse of PySlice_GetIndicesEx
In-Reply-To: <1472226685.15.0.242668202848.issue27867@psf.upfronthosting.co.za>
Message-ID: <20170125112909.21084.8995.21EA06CE@psf.io>


Roundup Robot added the comment:

New changeset d5590f357d74 by Serhiy Storchaka in branch '2.7':
Issue #27867: Replaced function PySlice_GetIndicesEx() with a macro.
https://hg.python.org/cpython/rev/d5590f357d74

New changeset 96f5327f7253 by Serhiy Storchaka in branch '3.5':
Issue #27867: Function PySlice_GetIndicesEx() is replaced with a macro if
https://hg.python.org/cpython/rev/96f5327f7253

New changeset b4457fe7fdb8 by Serhiy Storchaka in branch '3.6':
Issue #27867: Function PySlice_GetIndicesEx() is replaced with a macro if
https://hg.python.org/cpython/rev/b4457fe7fdb8

New changeset 6093ce8eed6c by Serhiy Storchaka in branch 'default':
Issue #27867: Function PySlice_GetIndicesEx() is deprecated and replaced with
https://hg.python.org/cpython/rev/6093ce8eed6c

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue27867>
_______________________________________

From report at bugs.python.org  Wed Jan 25 07:16:39 2017
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 25 Jan 2017 12:16:39 +0000
Subject: [issue29368] Optimize unpickling list-like objects
In-Reply-To: <1485338150.33.0.560817507263.issue29368@psf.upfronthosting.co.za>
Message-ID: <1485346599.43.0.0120936097954.issue29368@psf.upfronthosting.co.za>


STINNER Victor added the comment:

What is the cost of adding an extra isinstance(d, collections.Sequence) check? It would be closer the current design ("white list" of types), safer and avoid bad surprises.

But Python has a long tradition of duck typing, so maybe isinstance() can be seen as overkill :-)

----------
nosy: +haypo

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29368>
_______________________________________

From report at bugs.python.org  Wed Jan 25 07:22:42 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Wed, 25 Jan 2017 12:22:42 +0000
Subject: [issue29368] Optimize unpickling list-like objects
In-Reply-To: <1485338150.33.0.560817507263.issue29368@psf.upfronthosting.co.za>
Message-ID: <1485346962.53.0.72105901897.issue29368@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

collections.Sequence has no relation to the pickle protocol.

Every object that return listitems from the __reduce__() method must support the extend() method according to the specification.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29368>
_______________________________________

From report at bugs.python.org  Wed Jan 25 07:33:14 2017
From: report at bugs.python.org (INADA Naoki)
Date: Wed, 25 Jan 2017 12:33:14 +0000
Subject: [issue29369] Use Py_IDENTIFIER in Python-ast.c
Message-ID: <1485347594.14.0.436037009882.issue29369@psf.upfronthosting.co.za>


New submission from INADA Naoki:

While I'm looking Python's memory usage, I found some dicts which key
is not interned "_fields", "_ast" and "__modules__".

----------
components: Interpreter Core
files: ast-identifier.patch
keywords: patch
messages: 286244
nosy: inada.naoki
priority: normal
severity: normal
status: open
title: Use Py_IDENTIFIER in Python-ast.c
versions: Python 3.7
Added file: http://bugs.python.org/file46413/ast-identifier.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29369>
_______________________________________

From report at bugs.python.org  Wed Jan 25 07:40:14 2017
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 25 Jan 2017 12:40:14 +0000
Subject: [issue29369] Use Py_IDENTIFIER in Python-ast.c
In-Reply-To: <1485347594.14.0.436037009882.issue29369@psf.upfronthosting.co.za>
Message-ID: <1485348014.8.0.226576915233.issue29369@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Python/Python-ast.c started with:

/* File automatically generated by Parser/asdl_c.py. */

Please fix the generator, not the generated file ;-)

----------
nosy: +haypo

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29369>
_______________________________________

From report at bugs.python.org  Wed Jan 25 07:40:25 2017
From: report at bugs.python.org (INADA Naoki)
Date: Wed, 25 Jan 2017 12:40:25 +0000
Subject: [issue29336] merge tuples in module
In-Reply-To: <1484969445.33.0.406694538272.issue29336@psf.upfronthosting.co.za>
Message-ID: <1485348025.46.0.783076646018.issue29336@psf.upfronthosting.co.za>


INADA Naoki added the comment:

As I said on ML, now I think embedding some co_* tuples into code object
is better way to reduce number of tuples and memory usage.
So I close this issue for now.

Thanks for review and comments.

----------
resolution:  -> postponed
status: open -> pending

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29336>
_______________________________________

From report at bugs.python.org  Wed Jan 25 08:01:21 2017
From: report at bugs.python.org (=?utf-8?b?0JTQuNC70Y/QvSDQn9Cw0LvQsNGD0LfQvtCy?=)
Date: Wed, 25 Jan 2017 13:01:21 +0000
Subject: [issue29370] "./configure --enable-optimizations && make install"
 does not install files in ${BINDIR} and ${LIBDIR}
Message-ID: <1485349281.44.0.908942370293.issue29370@psf.upfronthosting.co.za>


New submission from ????? ????????:

With Python 3.5.3 "./configure --enable-shared && make install DESTDIR=/tmp/ZZ1" does install files in /tmp/ZZ1/usr/local/bin/ and /tmp/ZZ1/usr/local/bin/ .

"./configure --enable-optimizations --enable-shared && make install DESTDIR=/tmp/ZZ2" does not install files in /tmp/ZZ2/usr/local/bin/ and /tmp/ZZ2/usr/local/bin/ .

----------
components: Installation
messages: 286247
nosy: dilyan.palauzov
priority: normal
severity: normal
status: open
title: "./configure --enable-optimizations && make install" does not install files in ${BINDIR} and ${LIBDIR}
type: behavior
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29370>
_______________________________________

From report at bugs.python.org  Wed Jan 25 08:03:56 2017
From: report at bugs.python.org (INADA Naoki)
Date: Wed, 25 Jan 2017 13:03:56 +0000
Subject: [issue29369] Use Py_IDENTIFIER in Python-ast.c
In-Reply-To: <1485347594.14.0.436037009882.issue29369@psf.upfronthosting.co.za>
Message-ID: <1485349436.68.0.400512445459.issue29369@psf.upfronthosting.co.za>


Changes by INADA Naoki <songofacandy at gmail.com>:


Added file: http://bugs.python.org/file46414/ast-identifier2.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29369>
_______________________________________

From report at bugs.python.org  Wed Jan 25 08:08:59 2017
From: report at bugs.python.org (INADA Naoki)
Date: Wed, 25 Jan 2017 13:08:59 +0000
Subject: [issue29369] Use Py_IDENTIFIER in Python-ast.c
In-Reply-To: <1485347594.14.0.436037009882.issue29369@psf.upfronthosting.co.za>
Message-ID: <1485349739.04.0.310081854038.issue29369@psf.upfronthosting.co.za>


Changes by INADA Naoki <songofacandy at gmail.com>:


Added file: http://bugs.python.org/file46415/ast-identifier3.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29369>
_______________________________________

From report at bugs.python.org  Wed Jan 25 08:13:00 2017
From: report at bugs.python.org (INADA Naoki)
Date: Wed, 25 Jan 2017 13:13:00 +0000
Subject: [issue29369] Use Py_IDENTIFIER in Python-ast.c
In-Reply-To: <1485347594.14.0.436037009882.issue29369@psf.upfronthosting.co.za>
Message-ID: <1485349980.73.0.0784591292949.issue29369@psf.upfronthosting.co.za>


Changes by INADA Naoki <songofacandy at gmail.com>:


Removed file: http://bugs.python.org/file46414/ast-identifier2.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29369>
_______________________________________

From report at bugs.python.org  Wed Jan 25 08:13:27 2017
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 25 Jan 2017 13:13:27 +0000
Subject: [issue29368] Optimize unpickling list-like objects
In-Reply-To: <1485338150.33.0.560817507263.issue29368@psf.upfronthosting.co.za>
Message-ID: <1485350007.19.0.833059122273.issue29368@psf.upfronthosting.co.za>


STINNER Victor added the comment:

> Every object that return listitems from the __reduce__() method must support the extend() method according to the specification.

Hum ok. I don't know well the pickle module, but according to your quote, yeah, the patch is valid.

pickle-appends-extend.patch LGTM, except minor comments.

It would be nice to get a feedback from Alexandre, but I'm not sure that he is still around :-/

What about Antoine Pitrou, are you around? :-)

----------
nosy: +pitrou

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29368>
_______________________________________

From report at bugs.python.org  Wed Jan 25 08:13:43 2017
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 25 Jan 2017 13:13:43 +0000
Subject: [issue29368] Optimize unpickling list-like objects: use extend()
 instead of append()
In-Reply-To: <1485338150.33.0.560817507263.issue29368@psf.upfronthosting.co.za>
Message-ID: <1485350023.37.0.231217420998.issue29368@psf.upfronthosting.co.za>


Changes by STINNER Victor <victor.stinner at gmail.com>:


----------
title: Optimize unpickling list-like objects -> Optimize unpickling list-like objects: use extend() instead of append()

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29368>
_______________________________________

From report at bugs.python.org  Wed Jan 25 08:15:59 2017
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 25 Jan 2017 13:15:59 +0000
Subject: [issue29369] Use Py_IDENTIFIER in Python-ast.c
In-Reply-To: <1485347594.14.0.436037009882.issue29369@psf.upfronthosting.co.za>
Message-ID: <1485350159.83.0.198078745501.issue29369@psf.upfronthosting.co.za>


STINNER Victor added the comment:

> While I'm looking Python's memory usage, I found some dicts which key is not interned "_fields", "_ast" and "__modules__".

AST objects are supposed to be temporary. Interning strings used in the AST processor would make these strings immortal and so increase the memory usage, no?

What is the purpose of the patch? Speedup or reduce the memory usage?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29369>
_______________________________________

From report at bugs.python.org  Wed Jan 25 08:18:28 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Wed, 25 Jan 2017 13:18:28 +0000
Subject: [issue29368] Optimize unpickling list-like objects: use extend()
 instead of append()
In-Reply-To: <1485338150.33.0.560817507263.issue29368@psf.upfronthosting.co.za>
Message-ID: <1485350308.88.0.822375067833.issue29368@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

See issue17720 for a feedback from Alexandre.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29368>
_______________________________________

From report at bugs.python.org  Wed Jan 25 08:26:21 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Wed, 25 Jan 2017 13:26:21 +0000
Subject: [issue29369] Use Py_IDENTIFIER in Python-ast.c
In-Reply-To: <1485347594.14.0.436037009882.issue29369@psf.upfronthosting.co.za>
Message-ID: <1485350781.83.0.158975535857.issue29369@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

LGTM. _Py_IDENTIFIER is private API, but is not new in Python-ast.c.

I expect insignificant speedup and reduce the memory usage from the patch.

----------
nosy: +serhiy.storchaka

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29369>
_______________________________________

From report at bugs.python.org  Wed Jan 25 08:27:15 2017
From: report at bugs.python.org (INADA Naoki)
Date: Wed, 25 Jan 2017 13:27:15 +0000
Subject: [issue29369] Use Py_IDENTIFIER in Python-ast.c
In-Reply-To: <1485347594.14.0.436037009882.issue29369@psf.upfronthosting.co.za>
Message-ID: <1485350835.44.0.936587103568.issue29369@psf.upfronthosting.co.za>


INADA Naoki added the comment:

> AST objects are supposed to be temporary. Interning strings used in the AST processor would make these strings immortal and so increase the memory usage, no?
> What is the purpose of the patch? Speedup or reduce the memory usage?

The three identifiers are used in type.  Dozen (100+) types using them.

>>> import _ast
>>> _ast.Dict.__dict__
mappingproxy({'_fields': ('keys', 'values'), '__module__': '_ast', '__doc__': None})
>>> _ast.DictComp.__dict__
mappingproxy({'_fields': ('key', 'value', 'generators'), '__module__': '_ast', '__doc__': None})


Each string uses about 50 bytes. So 50 * 3 * 100 = 15KB may be reduced.

But main purpose is less noise when looking memory usage of Python.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29369>
_______________________________________

From report at bugs.python.org  Wed Jan 25 08:32:16 2017
From: report at bugs.python.org (Gereon Kaiping)
Date: Wed, 25 Jan 2017 13:32:16 +0000
Subject: [issue29333] ConfigParser calls Interpolation.before_read after
 reading
In-Reply-To: <1484913036.08.0.781257093151.issue29333@psf.upfronthosting.co.za>
Message-ID: <1485351136.11.0.616829954001.issue29333@psf.upfronthosting.co.za>


Changes by Gereon Kaiping <gereon.kaiping+python at gmail.com>:


----------
components: +Library (Lib)
type:  -> behavior
versions: +Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29333>
_______________________________________

From report at bugs.python.org  Wed Jan 25 08:34:01 2017
From: report at bugs.python.org (Roundup Robot)
Date: Wed, 25 Jan 2017 13:34:01 +0000
Subject: [issue29369] Use Py_IDENTIFIER in Python-ast.c
In-Reply-To: <1485347594.14.0.436037009882.issue29369@psf.upfronthosting.co.za>
Message-ID: <20170125133354.18743.51124.EE6EF7B5@psf.io>


Roundup Robot added the comment:

New changeset a507882736b2 by INADA Naoki in branch 'default':
Issue #29369: Use Py_IDENTIFIER in Python-ast.c
https://hg.python.org/cpython/rev/a507882736b2

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29369>
_______________________________________

From report at bugs.python.org  Wed Jan 25 08:34:23 2017
From: report at bugs.python.org (INADA Naoki)
Date: Wed, 25 Jan 2017 13:34:23 +0000
Subject: [issue29369] Use Py_IDENTIFIER in Python-ast.c
In-Reply-To: <1485347594.14.0.436037009882.issue29369@psf.upfronthosting.co.za>
Message-ID: <1485351263.05.0.923549646532.issue29369@psf.upfronthosting.co.za>


Changes by INADA Naoki <songofacandy at gmail.com>:


----------
resolution:  -> fixed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29369>
_______________________________________

From report at bugs.python.org  Wed Jan 25 08:36:09 2017
From: report at bugs.python.org (Marco Buttu)
Date: Wed, 25 Jan 2017 13:36:09 +0000
Subject: [issue29371] Typo in doctest documentation
Message-ID: <1485351369.68.0.185486175403.issue29371@psf.upfronthosting.co.za>


New submission from Marco Buttu:

>From the doctest documentation [1]:

"Symbolic names for the flags are supplied as module constants, which can be or'ed together and passed to various functions."

Is there a typo in "...which can be or'ed together..."? Maybe "collected together"?


[1] https://docs.python.org/3/library/doctest.html#option-flags

----------
assignee: docs at python
components: Documentation
messages: 286254
nosy: docs at python, marco.buttu
priority: normal
severity: normal
status: open
title: Typo in doctest documentation
versions: Python 2.7, Python 3.3, Python 3.4, Python 3.5, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29371>
_______________________________________

From report at bugs.python.org  Wed Jan 25 08:45:21 2017
From: report at bugs.python.org (Jim Fasarakis-Hilliard)
Date: Wed, 25 Jan 2017 13:45:21 +0000
Subject: [issue29371] Typo in doctest documentation
In-Reply-To: <1485351369.68.0.185486175403.issue29371@psf.upfronthosting.co.za>
Message-ID: <1485351921.82.0.963105949839.issue29371@psf.upfronthosting.co.za>


Jim Fasarakis-Hilliard added the comment:

>From what I understand, "or'ed" here stands for combining the options using `|` (https://docs.python.org/3/reference/expressions.html#binary-bitwise-operations).

You can see an example of that in the source for `doctest.py` https://hg.python.org/cpython/file/3.6/Lib/doctest.py#l144

----------
nosy: +Jim Fasarakis-Hilliard

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29371>
_______________________________________

From report at bugs.python.org  Wed Jan 25 09:13:49 2017
From: report at bugs.python.org (Marco Buttu)
Date: Wed, 25 Jan 2017 14:13:49 +0000
Subject: [issue29371] Typo in doctest documentation
In-Reply-To: <1485351369.68.0.185486175403.issue29371@psf.upfronthosting.co.za>
Message-ID: <1485353629.91.0.075194407263.issue29371@psf.upfronthosting.co.za>


Marco Buttu added the comment:

I think you got the meaning. I have never read it before :/ If you think the meaning is clear enough for everyone, I close the issue.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29371>
_______________________________________

From report at bugs.python.org  Wed Jan 25 09:17:36 2017
From: report at bugs.python.org (Marco Buttu)
Date: Wed, 25 Jan 2017 14:17:36 +0000
Subject: [issue29371] Typo in doctest documentation
In-Reply-To: <1485351369.68.0.185486175403.issue29371@psf.upfronthosting.co.za>
Message-ID: <1485353856.17.0.680473411134.issue29371@psf.upfronthosting.co.za>


Marco Buttu added the comment:

What about this?

"Symbolic names for the flags are supplied as module constants, which can be OR'ed together (flagA | flagB | ...) and passed to various functions."

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29371>
_______________________________________

From report at bugs.python.org  Wed Jan 25 09:43:25 2017
From: report at bugs.python.org (R. David Murray)
Date: Wed, 25 Jan 2017 14:43:25 +0000
Subject: [issue29366] os.listdir has inconsistent behavior when run on a
 non-directory
In-Reply-To: <1485312192.81.0.934042138535.issue29366@psf.upfronthosting.co.za>
Message-ID: <1485355405.34.0.758669237297.issue29366@psf.upfronthosting.co.za>


R. David Murray added the comment:

Could you please clarify what bug you are reporting?

----------
nosy: +r.david.murray

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29366>
_______________________________________

From report at bugs.python.org  Wed Jan 25 09:58:45 2017
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 25 Jan 2017 14:58:45 +0000
Subject: [issue29369] Use Py_IDENTIFIER in Python-ast.c
In-Reply-To: <1485347594.14.0.436037009882.issue29369@psf.upfronthosting.co.za>
Message-ID: <1485356325.44.0.217047793908.issue29369@psf.upfronthosting.co.za>


STINNER Victor added the comment:

+    result = PyObject_CallFunction((PyObject*)&PyType_Type, "s(O){OOOO}",
+                    type, base,
+                    _PyUnicode_FromId(&PyId__fields), fnames,
+                    _PyUnicode_FromId(&PyId___module__),
+                    _PyUnicode_FromId(&PyId__ast));

You should check if _PyUnicode_FromId() returns NULL if it was the first call and the UTF-8 decode failed to allocate memory.

You might initialize all these identifiers (and check for errors) in init_types() to avoid having to check for errors each time they are used.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29369>
_______________________________________

From report at bugs.python.org  Wed Jan 25 10:04:20 2017
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 25 Jan 2017 15:04:20 +0000
Subject: [issue29369] Use Py_IDENTIFIER in Python-ast.c
In-Reply-To: <1485347594.14.0.436037009882.issue29369@psf.upfronthosting.co.za>
Message-ID: <1485356660.7.0.938422140333.issue29369@psf.upfronthosting.co.za>


STINNER Victor added the comment:

> The three identifiers are used in type.  Dozen (100+) types using them.

I tried once to put *all* (most common) _Py_IDENTIFIER() in a single file and share them, but I abandoned my attempt:

http://bugs.python.org/issue19515#msg202385
Antoine Pitrou: "Well, then IMHO it's not worth it."

See issues #19512 and #19515.


"""
>>> import _ast
>>> _ast.Dict.__dict__
mappingproxy({'_fields': ('keys', 'values'), '__module__': '_ast', '__doc__': None})
"""

But the _ast module is not imported by default, only in programs importing "ast" explicitly.

Well, I'm not opposed to the change, I'm just trying to understand how the code is used ;-)

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29369>
_______________________________________

From report at bugs.python.org  Wed Jan 25 10:16:32 2017
From: report at bugs.python.org (A. Skrobov)
Date: Wed, 25 Jan 2017 15:16:32 +0000
Subject: [issue26526] In parsermodule.c,
 replace over 2KLOC of hand-crafted validation code, with a DFA
In-Reply-To: <1457600551.84.0.318395917649.issue26526@psf.upfronthosting.co.za>
Message-ID: <1485357392.86.0.670789650358.issue26526@psf.upfronthosting.co.za>


A. Skrobov added the comment:

Oh btw, the comment in the beginning of Grammar/Grammar

> # Note:  Changing the grammar specified in this file will most likely
> #        require corresponding changes in the parser module
> #        (../Modules/parsermodule.c).

is no longer true: after this patch went in, changing the grammar no longer requires translating the changes into Modules/parsermodule.c

Can somebody please remove the obsolete comment from there?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue26526>
_______________________________________

From report at bugs.python.org  Wed Jan 25 10:51:03 2017
From: report at bugs.python.org (INADA Naoki)
Date: Wed, 25 Jan 2017 15:51:03 +0000
Subject: [issue29369] Use Py_IDENTIFIER in Python-ast.c
In-Reply-To: <1485347594.14.0.436037009882.issue29369@psf.upfronthosting.co.za>
Message-ID: <1485359463.16.0.748071654124.issue29369@psf.upfronthosting.co.za>


INADA Naoki added the comment:

> You should check if _PyUnicode_FromId() returns NULL if it was the first call and the UTF-8 decode failed to allocate memory.

thanks. new patch will fix it.

> You might initialize all these identifiers (and check for errors) in init_types() to avoid having to check for errors each time they are used.

Here is not so performance critical part.

> But the _ast module is not imported by default, only in programs importing "ast" explicitly.
> Well, I'm not opposed to the change, I'm just trying to understand how the code is used ;-)

We use Flask. Flask is based on Werkzeug. Werkzeug imports inspect.
inspect imports ast. ast imports _ast.

----------
Added file: http://bugs.python.org/file46416/ast-identifier4.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29369>
_______________________________________

From report at bugs.python.org  Wed Jan 25 10:54:35 2017
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 25 Jan 2017 15:54:35 +0000
Subject: [issue29369] Use Py_IDENTIFIER in Python-ast.c
In-Reply-To: <1485347594.14.0.436037009882.issue29369@psf.upfronthosting.co.za>
Message-ID: <1485359675.03.0.0688165444407.issue29369@psf.upfronthosting.co.za>


STINNER Victor added the comment:

"We use Flask. Flask is based on Werkzeug. Werkzeug imports inspect. inspect imports ast. ast imports _ast."

Oh, this was unexpected to me :-)

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29369>
_______________________________________

From report at bugs.python.org  Wed Jan 25 10:56:43 2017
From: report at bugs.python.org (Jeroen Demeyer)
Date: Wed, 25 Jan 2017 15:56:43 +0000
Subject: [issue13285] signal module ignores external signal changes
In-Reply-To: <1319801278.56.0.946220269602.issue13285@psf.upfronthosting.co.za>
Message-ID: <1485359803.23.0.386791794707.issue13285@psf.upfronthosting.co.za>


Jeroen Demeyer added the comment:

Let me add that this "low-level opaque object" would be rather easy to implement on POSIX systems (I have no clue about other systems such as Windows). I could implement it, but it would be good to have some pre-approval from Python devs that it's a good idea to do this.

----------
nosy: +jdemeyer

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue13285>
_______________________________________

From report at bugs.python.org  Wed Jan 25 11:03:44 2017
From: report at bugs.python.org (Lee Griffiths)
Date: Wed, 25 Jan 2017 16:03:44 +0000
Subject: [issue29372] RotatingFileHandler rotates empty logfile if it emits a
 large string
Message-ID: <1485360224.38.0.947637323441.issue29372@psf.upfronthosting.co.za>


New submission from Lee Griffiths:

If you write a really big string to an empty file, and that string is > maxBytes, then `RotatingFileHandler` (or even `BaseRotatingHandler`) will roll that empty file into the backup queue.

I think it should instead use that empty file for the mega-string. By not doing so it "wastes" a slot in the backup queue.

It's a very minor issue: I doubt it really happens IRL, but I noticed it when extending RotatingFileHandler in py2.7 to add gzip stuff and my test cases used a really small size (16 bytes!)


Here's a test file:

    #!/usr/bin/env python3
    # coding=utf-8
    
    import logging
    import os
    import tempfile
    
    from logging.handlers import RotatingFileHandler
    
    
    class MockRecord(object):
        def __init__(self, msg):
            self.msg = msg
            self.stack_info = None
            self.exc_info = None
            self.exc_text = None
    
        def getMessage(self):
            return self.msg
    
    
    def test_file_rollover_from_mega_string(temp_dir_path):
        # This is a pretty weird test.
        # It tests that writing a huge string to a blank file causes the blank
        # file to be archived and the huge string written to the next log file.
        #
        # Normally the log files would have a large max bytes so we'd have to
        # be writing a giant string to an empty file for this to happen.
        # But, even if it does, it's what BaseRotatingHandler does, so...
    
        log_path = os.path.join(temp_dir_path, "mylog.log")
    
        handler = RotatingFileHandler(log_path, maxBytes=16, backupCount=5)
        handler.setFormatter(logging.Formatter())
        with open(log_path) as log:
            assert log.read() == ""
    
        # ----------------------------------------------
        handler.emit(MockRecord("There once was a test from bitbucket"))
        with open(log_path) as log:
            log_read = log.read()
            assert log_read == "There once was a test from bitbucket\n"
    
        with open(log_path + ".1") as log:
            log_read = log.read()
            assert log_read == ""
    
        # ----------------------------------------------
        handler.emit(MockRecord("11 chars"))
        with open(log_path) as log:
            log_read = log.read()
            assert log_read == "11 chars\n"
    
        with open(log_path + ".1") as log:
            log_read = log.read()
            assert log_read == "There once was a test from bitbucket\n"
    
        with open(log_path + ".2") as log:
            log_read = log.read()
            assert log_read == ""
    
        handler.close()
    
    
    test_file_rollover_from_mega_string(tempfile.mkdtemp())


and here's a patch that I think will fix it:

~/src/others/cpython (master *%=)$ cat empty_rollover.patch 
diff --git a/Lib/logging/handlers.py b/Lib/logging/handlers.py
index 7d77973..0dabfd7 100644
--- a/Lib/logging/handlers.py
+++ b/Lib/logging/handlers.py
@@ -186,7 +186,11 @@ class RotatingFileHandler(BaseRotatingHandler):
         if self.maxBytes > 0:                   # are we rolling over?
             msg = "%s\n" % self.format(record)
             self.stream.seek(0, 2)  #due to non-posix-compliant Windows feature
-            if self.stream.tell() + len(msg) >= self.maxBytes:
+            size = self.stream.tell()
+            if size == 0:
+                # No point rolling-over an empty file
+                return 0
+            elif size + len(msg) >= self.maxBytes:
                 return 1
         return 0

----------
components: Library (Lib)
messages: 286265
nosy: Poddster, vinay.sajip
priority: normal
severity: normal
status: open
title: RotatingFileHandler rotates empty logfile if it emits a large string
type: resource usage
versions: Python 2.7, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29372>
_______________________________________

From report at bugs.python.org  Wed Jan 25 11:15:42 2017
From: report at bugs.python.org (Michael Haubenwallner)
Date: Wed, 25 Jan 2017 16:15:42 +0000
Subject: [issue26439] ctypes.util.find_library fails when ldconfig/glibc not
 available (e.g., AIX)
In-Reply-To: <1456413023.18.0.874865601403.issue26439@psf.upfronthosting.co.za>
Message-ID: <1485360942.29.0.510665294705.issue26439@psf.upfronthosting.co.za>


Changes by Michael Haubenwallner <michael.haubenwallner at ssi-schaefer.com>:


----------
nosy: +haubi

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue26439>
_______________________________________

From report at bugs.python.org  Wed Jan 25 11:34:00 2017
From: report at bugs.python.org (Thomas Kluyver)
Date: Wed, 25 Jan 2017 16:34:00 +0000
Subject: [issue13285] signal module ignores external signal changes
In-Reply-To: <1319801278.56.0.946220269602.issue13285@psf.upfronthosting.co.za>
Message-ID: <1485362040.52.0.658833629893.issue13285@psf.upfronthosting.co.za>


Thomas Kluyver added the comment:

I pitched both the opaque handle and the context manager to python-ideas, but didn't get any responses, positive or negative.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue13285>
_______________________________________

From report at bugs.python.org  Wed Jan 25 12:20:31 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Wed, 25 Jan 2017 17:20:31 +0000
Subject: [issue29369] Use Py_IDENTIFIER in Python-ast.c
In-Reply-To: <1485347594.14.0.436037009882.issue29369@psf.upfronthosting.co.za>
Message-ID: <1485364831.28.0.173792203506.issue29369@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

I think additional checks are not needed. PyObject_CallFunction() allows passing NULLs.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29369>
_______________________________________

From report at bugs.python.org  Wed Jan 25 12:21:16 2017
From: report at bugs.python.org (Zac Medico)
Date: Wed, 25 Jan 2017 17:21:16 +0000
Subject: [issue29373] subprocess.Popen os.close(p2cread) in _execute_child can
 cause garbage collection of self.stdout to close a reused file
 descriptor
Message-ID: <1485364876.35.0.214175144503.issue29373@psf.upfronthosting.co.za>


New submission from Zac Medico:

In Popen _execute_child method, os.close(p2cread) closes a file descriptor which is referenced by self.stdout with closefd=True, created by this code:

   self.stdout = io.open(c2pread, 'rb', bufsize)

When self.stdout is finally garbage collected, it can close a file descriptor which has been reused since the os.close(p2cread) call. In the rare cases when this happens, it results in a traceback like the following:

Traceback (most recent call last):
  File "/usr/lib64/python3.4/subprocess.py", line 1418, in _execute_child
    part = _eintr_retry_call(os.read, errpipe_read, 50000)
  File "/usr/lib64/python3.4/subprocess.py", line 491, in _eintr_retry_call
    return func(*args)
OSError: [Errno 9] Bad file descriptor

I have observed this issue with python 3.4.5, and the code flaw also appears to be present in 3.5, 3.6, and 3.7. This sort of garbage collection issue has been highlighted in the first comment of http://bugs.python.org/issue16140, but there is still a remaining problem with this os.close(p2cread) call.

----------
messages: 286268
nosy: zmedico
priority: normal
severity: normal
status: open
title: subprocess.Popen os.close(p2cread) in _execute_child can cause garbage collection of self.stdout to close a reused file descriptor
versions: Python 3.4, Python 3.5, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29373>
_______________________________________

From report at bugs.python.org  Wed Jan 25 12:25:31 2017
From: report at bugs.python.org (Petr MOTEJLEK)
Date: Wed, 25 Jan 2017 17:25:31 +0000
Subject: [issue29374] Doc bug: signal.sigwait and signal.sigtimedwait do not
 work outside the Main thread
Message-ID: <1485365131.36.0.15146613889.issue29374@psf.upfronthosting.co.za>


New submission from Petr MOTEJLEK:

Hi,

The documentation for signal.signal() clearly states that it is only supposed to be called on MainThread

However, it does not say so for the signal.sigwait() and neither signal.sigtimedwait()

I think this is an error on the documentation side of things (unless I misread it). When either signal.sigwait or signal.sigtimedwait are called outside MainThread, they simply never catch any signals (signal.sigwait blocks indefinitely)

I did not test this on Windows, but on both Linux and OS X the behavior is the same

Consider the below simple code

  import signal
  import os
  def sigwait():
    print("Send me a signal, my PID is {p}".format(p=os.getpid()))  
    print("Got the signal: {i}".format(i=signal.sigwait((signal.SIGUSR1,))))

If sigwait() is called on MainThread and the process receives SIGUSR1, "Got the signal: ..." gets printed. However, calling sigwait in a different thread blocks the thread indefinitely. The behavior is the same with signal.sigtimedwait() as well

----------
assignee: docs at python
components: Documentation
messages: 286269
nosy: docs at python, petr at motejlek.net
priority: normal
severity: normal
status: open
title: Doc bug: signal.sigwait and signal.sigtimedwait do not work outside the Main thread
type: behavior
versions: Python 3.4, Python 3.5, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29374>
_______________________________________

From report at bugs.python.org  Wed Jan 25 12:36:29 2017
From: report at bugs.python.org (Zac Medico)
Date: Wed, 25 Jan 2017 17:36:29 +0000
Subject: [issue29373] subprocess.Popen os.close(p2cread) in _execute_child can
 cause garbage collection of self.stdout to close a reused file
 descriptor
In-Reply-To: <1485364876.35.0.214175144503.issue29373@psf.upfronthosting.co.za>
Message-ID: <1485365789.66.0.684357409902.issue29373@psf.upfronthosting.co.za>


Zac Medico added the comment:

Actually, os.close(p2cread) closes a different file descriptor than c2pread, so my analysis is not correct. I don't know the precise cause of the EBADF error that I have observed. I suppose it could be some other code closing a file descriptor that belongs to a file object. Closing for now. Sorry for the noise.

I will follow up on the issue here:

https://bugs.gentoo.org/show_bug.cgi?id=582098

----------
resolution:  -> third party
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29373>
_______________________________________

From report at bugs.python.org  Wed Jan 25 12:42:16 2017
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 25 Jan 2017 17:42:16 +0000
Subject: [issue29358] Add tp_fastnew and tp_fastinit to PyTypeObject,
 15-20% faster object instanciation
In-Reply-To: <1485246329.89.0.254567368952.issue29358@psf.upfronthosting.co.za>
Message-ID: <1485366136.93.0.356107766724.issue29358@psf.upfronthosting.co.za>


STINNER Victor added the comment:

I implemented more optimizations:

* type_call() uses FASTCALL
* object_new() and object_init() use FASTCALL
* _asyncio.Future new & init use FASTCALL

As a result, create an _asyncio.Future is now 1.45x faster:

haypo at smithers$ ./python -m perf timeit -s 'import _asyncio, asyncio; Future=_asyncio.Future; loop=asyncio.get_event_loop()' 'Future(loop=loop)' --compare-to=../default-ref/python 
Median +- std dev: [ref] 411 ns +- 8 ns -> [fast_init] 283 ns +- 10 ns: 1.45x faster (-31%)


Yury> Great. Can you share the Richards benchmark results?  It specifically stresses class instantiation, so we should be able to see the improvement.

While bm_richards.py doesn't call _PyStack_AsTuple() anyone in the benchmark (it seems like it full uses FASTCALL), it seems like there is simply zero impact on performance :-(

$ ./python -m perf compare_to richards_ref.json richards_fastinit.json 
Benchmark hidden because not significant (1): richards

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29358>
_______________________________________

From report at bugs.python.org  Wed Jan 25 14:00:51 2017
From: report at bugs.python.org (Vinay Sajip)
Date: Wed, 25 Jan 2017 19:00:51 +0000
Subject: [issue29372] RotatingFileHandler rotates empty logfile if it emits a
 large string
In-Reply-To: <1485360224.38.0.947637323441.issue29372@psf.upfronthosting.co.za>
Message-ID: <1485370851.02.0.631378523751.issue29372@psf.upfronthosting.co.za>


Vinay Sajip added the comment:

I don't see the point - your proposed solution only works if the log file is completely empty at rollover time, but if the file contains a few bytes from an earlier "short" log message, it might seem just as wasteful of a backup slot as if it had zero bytes. I think the understanding is that maxBytes should be large compared to the size of an average log message, and that if that isn't the case, the user will live with the consequences because it's a contrived scenario (for tests, etc.) rather than a real-life one. There's always the option of subclassing if a user really does need bespoke rollover behaviour.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29372>
_______________________________________

From report at bugs.python.org  Wed Jan 25 14:29:11 2017
From: report at bugs.python.org (Glyph Lefkowitz)
Date: Wed, 25 Jan 2017 19:29:11 +0000
Subject: [issue10544] yield expression inside generator expression does nothing
In-Reply-To: <1290799279.15.0.89740732651.issue10544@psf.upfronthosting.co.za>
Message-ID: <1485372551.85.0.998655423928.issue10544@psf.upfronthosting.co.za>


Glyph Lefkowitz added the comment:

Is the fact that 'await' produces a syntax error in this context the same bug or a new one?

----------
nosy: +glyph

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue10544>
_______________________________________

From report at bugs.python.org  Wed Jan 25 14:36:50 2017
From: report at bugs.python.org (Ivan Levkivskyi)
Date: Wed, 25 Jan 2017 19:36:50 +0000
Subject: [issue10544] yield expression inside generator expression does nothing
In-Reply-To: <1290799279.15.0.89740732651.issue10544@psf.upfronthosting.co.za>
Message-ID: <1485373010.24.0.804526000541.issue10544@psf.upfronthosting.co.za>


Ivan Levkivskyi added the comment:

> Is the fact that 'await' produces a syntax error in this context the same bug or a new one?

What kind of SyntaxError? await outside an async function is prohibited, bare await is also prohibited.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue10544>
_______________________________________

From report at bugs.python.org  Wed Jan 25 15:06:22 2017
From: report at bugs.python.org (raylu)
Date: Wed, 25 Jan 2017 20:06:22 +0000
Subject: [issue29366] os.listdir has inconsistent behavior when run on a
 non-directory
In-Reply-To: <1485312192.81.0.934042138535.issue29366@psf.upfronthosting.co.za>
Message-ID: <1485374782.03.0.583082667647.issue29366@psf.upfronthosting.co.za>


raylu added the comment:

os.listdir should always return NotADirectoryError when run on a regular file. Users shouldn't have to special-case win32/FileNotFoundError.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29366>
_______________________________________

From report at bugs.python.org  Wed Jan 25 15:14:44 2017
From: report at bugs.python.org (Jonathan Guthrie)
Date: Wed, 25 Jan 2017 20:14:44 +0000
Subject: [issue29375] httplib: wrong Host header when connecting to IPv6
 link-local address
Message-ID: <1485375284.1.0.310369100275.issue29375@psf.upfronthosting.co.za>


New submission from Jonathan Guthrie:

This is related to issue 5111.

An IPv6 link-local address must include a scope specifier as part of the address passed to the HTTPConnection or HTTPSConnection constructor, that scope specifier is not being stripped from the address passed in the HTTP 1.1 Host: header line.

So, suppose I was attempting to connect to an HTTP server on fe80::8aae:1dff:fea4:29c8.  That's a link-local address, so I must give the scope specifier, which is the interface I can use to connect to that address.  So, the address I would pass to the HTTPConnection constructor might look like "fe80::8aae:1dff:fea4:29c8%eth0".  The appropriate Host line in the HTTP request would be "Host: [fe80::8aae:1dff:fea4:29c8]" but it is actually "Host: [fe80::8aae:1dff:fea4:29c8%eth0]"

Compliant HTTP servers reject requests including this line as malformed.

----------
messages: 286276
nosy: JonathanGuthrie
priority: normal
severity: normal
status: open
title: httplib: wrong Host header when connecting to IPv6 link-local address
type: behavior
versions: Python 2.7, Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29375>
_______________________________________

From report at bugs.python.org  Wed Jan 25 15:17:39 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Wed, 25 Jan 2017 20:17:39 +0000
Subject: [issue29028] Use-After-Free in PyString_FromStringAndSize() of
 stringobject.c
In-Reply-To: <1482247531.75.0.221402980867.issue29028@psf.upfronthosting.co.za>
Message-ID: <1485375459.0.0.902632456021.issue29028@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

Proposed patch fixes the issue. But it is hard to write a reliable patch.

----------
Added file: http://bugs.python.org/file46417/buffer-use-after-free-3.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29028>
_______________________________________

From report at bugs.python.org  Wed Jan 25 15:21:18 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Wed, 25 Jan 2017 20:21:18 +0000
Subject: [issue29366] os.listdir has inconsistent behavior when run on a
 non-directory
In-Reply-To: <1485312192.81.0.934042138535.issue29366@psf.upfronthosting.co.za>
Message-ID: <1485375678.62.0.939360021242.issue29366@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

Blame Windows for this behavior. os.listdir() is just a wrapper around few system calls.

It is expected that functions in the os module can raise different exceptions on different platforms.

----------
nosy: +serhiy.storchaka

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29366>
_______________________________________

From report at bugs.python.org  Wed Jan 25 15:38:22 2017
From: report at bugs.python.org (Eryk Sun)
Date: Wed, 25 Jan 2017 20:38:22 +0000
Subject: [issue29366] os.listdir has inconsistent behavior when run on a
 non-directory
In-Reply-To: <1485312192.81.0.934042138535.issue29366@psf.upfronthosting.co.za>
Message-ID: <1485376702.54.0.00210002602068.issue29366@psf.upfronthosting.co.za>


Eryk Sun added the comment:

Python appends "\*.*" to the path, so in the case in which the leaf element is a file, FindFirstFile naturally fails with ERROR_PATH_NOT_FOUND.

Python 3.5+ could maybe switch to calling CreateFile to open a handle and GetFileInformationByHandleEx to get the FILE_ID_BOTH_DIR_INFO (added in Vista, so this couldn't be backported to 2.7). It would first have to get the FILE_BASIC_INFO for the file attributes to ensure that the target is a directory. This could also support listing directories by file descriptor on Windows, since the CRT has O_OBTAIN_DIR (0x2000) for opening a directory.

----------
nosy: +eryksun

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29366>
_______________________________________

From report at bugs.python.org  Wed Jan 25 16:01:56 2017
From: report at bugs.python.org (Michael Kleehammer)
Date: Wed, 25 Jan 2017 21:01:56 +0000
Subject: [issue29236] 'an ASCII string of one or more PEM-encoded
 certificates' needs to be unicode
In-Reply-To: <1484097460.25.0.906949156541.issue29236@psf.upfronthosting.co.za>
Message-ID: <1485378116.9.0.309898706776.issue29236@psf.upfronthosting.co.za>


Changes by Michael Kleehammer <michael at kleehammer.com>:


----------
nosy: +mkleehammer

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29236>
_______________________________________

From report at bugs.python.org  Wed Jan 25 16:22:30 2017
From: report at bugs.python.org (Martin Panter)
Date: Wed, 25 Jan 2017 21:22:30 +0000
Subject: [issue29375] httplib: wrong Host header when connecting to IPv6
 link-local address
In-Reply-To: <1485375284.1.0.310369100275.issue29375@psf.upfronthosting.co.za>
Message-ID: <1485379350.21.0.509767147939.issue29375@psf.upfronthosting.co.za>


Changes by Martin Panter <vadmium+py at gmail.com>:


----------
components: +Library (Lib)
stage:  -> needs patch
versions: +Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29375>
_______________________________________

From report at bugs.python.org  Wed Jan 25 17:15:12 2017
From: report at bugs.python.org (Glyph Lefkowitz)
Date: Wed, 25 Jan 2017 22:15:12 +0000
Subject: [issue10544] yield expression inside generator expression does nothing
In-Reply-To: <1290799279.15.0.89740732651.issue10544@psf.upfronthosting.co.za>
Message-ID: <1485382512.62.0.298219008896.issue10544@psf.upfronthosting.co.za>


Glyph Lefkowitz added the comment:

>>> async def foo():
...     bar = [await x for x in range(10)]
  File "<input>", line 2
SyntaxError: 'await' expressions in comprehensions are not supported

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue10544>
_______________________________________

From report at bugs.python.org  Wed Jan 25 17:17:39 2017
From: report at bugs.python.org (Gregory P. Smith)
Date: Wed, 25 Jan 2017 22:17:39 +0000
Subject: [issue29370] "./configure --enable-optimizations && make install"
 does not install files in ${BINDIR} and ${LIBDIR}
In-Reply-To: <1485349281.44.0.908942370293.issue29370@psf.upfronthosting.co.za>
Message-ID: <1485382659.54.0.933987413417.issue29370@psf.upfronthosting.co.za>


Changes by Gregory P. Smith <greg at krypto.org>:


----------
nosy: +gregory.p.smith

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29370>
_______________________________________

From report at bugs.python.org  Wed Jan 25 17:19:32 2017
From: report at bugs.python.org (Gregory P. Smith)
Date: Wed, 25 Jan 2017 22:19:32 +0000
Subject: [issue29375] httplib: wrong Host header when connecting to IPv6
 link-local address
In-Reply-To: <1485375284.1.0.310369100275.issue29375@psf.upfronthosting.co.za>
Message-ID: <1485382772.06.0.939439000631.issue29375@psf.upfronthosting.co.za>


Changes by Gregory P. Smith <greg at krypto.org>:


----------
nosy: +gregory.p.smith

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29375>
_______________________________________

From report at bugs.python.org  Wed Jan 25 17:21:17 2017
From: report at bugs.python.org (Gregory P. Smith)
Date: Wed, 25 Jan 2017 22:21:17 +0000
Subject: [issue25667] Supply dual-stack (IPv4/IPv6) socket bind routine
In-Reply-To: <1447939973.83.0.965937708444.issue25667@psf.upfronthosting.co.za>
Message-ID: <1485382877.9.0.797028372607.issue25667@psf.upfronthosting.co.za>


Changes by Gregory P. Smith <greg at krypto.org>:


----------
nosy: +gregory.p.smith

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue25667>
_______________________________________

From report at bugs.python.org  Wed Jan 25 17:21:26 2017
From: report at bugs.python.org (Gregory P. Smith)
Date: Wed, 25 Jan 2017 22:21:26 +0000
Subject: [issue24209] Allow IPv6 bind in http.server
In-Reply-To: <1431784485.93.0.0140060801451.issue24209@psf.upfronthosting.co.za>
Message-ID: <1485382886.1.0.655157253452.issue24209@psf.upfronthosting.co.za>


Changes by Gregory P. Smith <greg at krypto.org>:


----------
nosy: +gregory.p.smith

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24209>
_______________________________________

From report at bugs.python.org  Wed Jan 25 17:23:59 2017
From: report at bugs.python.org (Gregory P. Smith)
Date: Wed, 25 Jan 2017 22:23:59 +0000
Subject: [issue23448] urllib2 needs to remove scope from IPv6 address when
 creating Host header
In-Reply-To: <1423680023.83.0.120732364895.issue23448@psf.upfronthosting.co.za>
Message-ID: <1485383039.78.0.172618087395.issue23448@psf.upfronthosting.co.za>


Changes by Gregory P. Smith <greg at krypto.org>:


----------
nosy: +gregory.p.smith

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23448>
_______________________________________

From report at bugs.python.org  Wed Jan 25 17:24:21 2017
From: report at bugs.python.org (Gregory P. Smith)
Date: Wed, 25 Jan 2017 22:24:21 +0000
Subject: [issue27500] ProactorEventLoop cannot open connection to ::1
In-Reply-To: <1468346135.06.0.410333112523.issue27500@psf.upfronthosting.co.za>
Message-ID: <1485383061.27.0.504280172787.issue27500@psf.upfronthosting.co.za>


Changes by Gregory P. Smith <greg at krypto.org>:


----------
nosy: +gregory.p.smith

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue27500>
_______________________________________

From report at bugs.python.org  Wed Jan 25 17:24:57 2017
From: report at bugs.python.org (Ivan Levkivskyi)
Date: Wed, 25 Jan 2017 22:24:57 +0000
Subject: [issue10544] yield expression inside generator expression does nothing
In-Reply-To: <1290799279.15.0.89740732651.issue10544@psf.upfronthosting.co.za>
Message-ID: <1485383097.52.0.478787579047.issue10544@psf.upfronthosting.co.za>


Ivan Levkivskyi added the comment:

Python 3.5 does not support this, you should use Python 3.6 (plus await x will fail when you will run the coroutine, since you cannot await on int).

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue10544>
_______________________________________

From report at bugs.python.org  Wed Jan 25 17:32:06 2017
From: report at bugs.python.org (Glyph Lefkowitz)
Date: Wed, 25 Jan 2017 22:32:06 +0000
Subject: [issue10544] yield expression inside generator expression does nothing
In-Reply-To: <1290799279.15.0.89740732651.issue10544@psf.upfronthosting.co.za>
Message-ID: <1485383526.47.0.0804608609526.issue10544@psf.upfronthosting.co.za>


Glyph Lefkowitz added the comment:

OK, cool. So, long term, there will be a way to do this (suspend within a generator expression). Thanks for the pointer.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue10544>
_______________________________________

From report at bugs.python.org  Wed Jan 25 17:32:26 2017
From: report at bugs.python.org (Glyph Lefkowitz)
Date: Wed, 25 Jan 2017 22:32:26 +0000
Subject: [issue10544] yield expression inside generator expression does nothing
In-Reply-To: <1290799279.15.0.89740732651.issue10544@psf.upfronthosting.co.za>
Message-ID: <1485383546.82.0.676287817133.issue10544@psf.upfronthosting.co.za>


Glyph Lefkowitz added the comment:

(As far as awaiting on int, yes, I know how await works, I was focusing on the syntax.)

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue10544>
_______________________________________

From report at bugs.python.org  Wed Jan 25 17:48:25 2017
From: report at bugs.python.org (Martin Panter)
Date: Wed, 25 Jan 2017 22:48:25 +0000
Subject: [issue29374] Doc bug: signal.sigwait and signal.sigtimedwait do not
 work outside the Main thread
In-Reply-To: <1485365131.36.0.15146613889.issue29374@psf.upfronthosting.co.za>
Message-ID: <1485384505.0.0.443124014535.issue29374@psf.upfronthosting.co.za>


Martin Panter added the comment:

This works for me on Linux:

>>> signal.pthread_sigmask(signal.SIG_BLOCK, {signal.SIGUSR1})
set()
>>> import threading
>>> t = threading.Thread(target=sigwait)
>>> t.start()
Send me a signal, my PID is 24197
>>> os.kill(os.getpid(), signal.SIGUSR1)
Got the signal: 10
>>> t.join()

Posix <http://pubs.opengroup.org/onlinepubs/9699919799/functions/sigwait.html> only defines sigwait() if the signals are already blocked. Two reasons behind this come to mind:

1. There would be a race where the signal handler may be called first (or the signal may be ignored) at the OS level, and the sigwait() function will miss the signal and block.

2. If the signal is handled in the context of another thread that isn?t using sigwait(), it may be consumed (handled or ignored) in the context of the other thread, with no effect on your sigwait() call.

This detail of blocking the signal seems to be a common error, so maybe the Python documentation could help point it out. (Issue 25868 comes to mind; there is a test case that IMO should block a signal before waiting for it.)

----------
nosy: +martin.panter
versions:  -Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29374>
_______________________________________

From report at bugs.python.org  Wed Jan 25 18:10:08 2017
From: report at bugs.python.org (Tom Myers)
Date: Wed, 25 Jan 2017 23:10:08 +0000
Subject: [issue29376] threading._DummyThread.__repr__ raises AssertionError
Message-ID: <1485385808.95.0.00162143215773.issue29376@psf.upfronthosting.co.za>


New submission from Tom Myers:

_DummyThread objects have _is_stopped = False, but _tstate_lock = None.  This causes an AssertionError in is_alive and __repr__ .

  File ".../cp34/lib/python3.4/threading.py", line 814, in __repr__
    self.is_alive() # easy way to get ._is_stopped set when appropriate
  File ".../cp34/lib/python3.4/threading.py", line 1110, in is_alive
    self._wait_for_tstate_lock(False)
  File ".../cp34/lib/python3.4/threading.py", line 1066, in _wait_for_tstate_lock
    assert self._is_stopped
AssertionError

----------
components: Library (Lib)
messages: 286285
nosy: Tom Myers
priority: normal
severity: normal
status: open
title: threading._DummyThread.__repr__ raises AssertionError
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29376>
_______________________________________

From report at bugs.python.org  Wed Jan 25 18:18:40 2017
From: report at bugs.python.org (Martin Panter)
Date: Wed, 25 Jan 2017 23:18:40 +0000
Subject: [issue29375] httplib: wrong Host header when connecting to IPv6
 link-local address
In-Reply-To: <1485375284.1.0.310369100275.issue29375@psf.upfronthosting.co.za>
Message-ID: <1485386320.91.0.695111000675.issue29375@psf.upfronthosting.co.za>


Martin Panter added the comment:

More closely related: Issue 23448, about the same thing with urllib, which adds the Host value itself. Any solution should be shared between both modules.

----------
nosy: +martin.panter

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29375>
_______________________________________

From report at bugs.python.org  Wed Jan 25 18:19:32 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Wed, 25 Jan 2017 23:19:32 +0000
Subject: [issue29270] super call in ctypes sub-class fails in 3.6
In-Reply-To: <1484345422.14.0.967631121608.issue29270@psf.upfronthosting.co.za>
Message-ID: <1485386372.68.0.440239001263.issue29270@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

See also similar issue with scrapy: https://github.com/scrapy-plugins/scrapy-djangoitem/issues/18.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29270>
_______________________________________

From report at bugs.python.org  Wed Jan 25 19:20:13 2017
From: report at bugs.python.org (INADA Naoki)
Date: Thu, 26 Jan 2017 00:20:13 +0000
Subject: [issue29369] Use Py_IDENTIFIER in Python-ast.c
In-Reply-To: <1485347594.14.0.436037009882.issue29369@psf.upfronthosting.co.za>
Message-ID: <1485390013.15.0.99636843593.issue29369@psf.upfronthosting.co.za>


INADA Naoki added the comment:

Thanks, Serhiy.  You're right.

----------
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29369>
_______________________________________

From report at bugs.python.org  Wed Jan 25 20:53:46 2017
From: report at bugs.python.org (STINNER Victor)
Date: Thu, 26 Jan 2017 01:53:46 +0000
Subject: [issue29259] Add tp_fastcall to PyTypeObject: support FASTCALL
 calling convention for all callable objects
In-Reply-To: <1484310274.99.0.832491900546.issue29259@psf.upfronthosting.co.za>
Message-ID: <1485395626.42.0.577836037408.issue29259@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Patch version 6: type inheritance and all wrappers should now be handled correctly. Changes:

* Add fastwrapper to wrapperbase structure
* Add d_use_fastwrapper to PyWrapperDescrObject structure
* wrapper_call() uses fast call
* Protect access to tp_fastcall with Py_TPFLAGS_HAVE_FASTCALL in all cases
* _Py_RawFastCallDict() doesn't check the result anymore, it's now checked in the caller: self can be NULL for static methods (try test_bytes unit test!)
* Fix type inheritance: update_one_slot() now makes sure that tp_call and tp_fastcall are synchronize. Before, the update was done after but not in all cases.
* Fix fastcall_wrapper() to support "StaticType.tp_call(obj, ...);" calls: Py_TYPE(obj)->tp_fastcall can be NULL, we have to search non-NULL tp_fastcall in base classes in this case.

IMHO the most tricky parts of the change are in update_one_slot() and add_operators().

Many fixes were "backported" from my work on the issue #29358 (tp_fastnew and tp_fastinit). Technically, I cherry-picked commits and fixes all the merge conflicts.

It may be worth to use FASTCALL in more wrappers (wrap_xxx functions in typeobject.c) of slotdefs.

----------
Added file: http://bugs.python.org/file46418/tp_fastcall-6.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29259>
_______________________________________

From report at bugs.python.org  Wed Jan 25 21:18:06 2017
From: report at bugs.python.org (Roundup Robot)
Date: Thu, 26 Jan 2017 02:18:06 +0000
Subject: [issue29358] Add tp_fastnew and tp_fastinit to PyTypeObject,
 15-20% faster object instanciation
In-Reply-To: <1485246329.89.0.254567368952.issue29358@psf.upfronthosting.co.za>
Message-ID: <20170126021801.96417.27437.9DA1B0A4@psf.io>


Roundup Robot added the comment:

New changeset 8feb6a5ce4c6 by Victor Stinner in branch 'default':
Issue #29358: Add postcondition checks on types
https://hg.python.org/cpython/rev/8feb6a5ce4c6

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29358>
_______________________________________

From report at bugs.python.org  Wed Jan 25 21:18:12 2017
From: report at bugs.python.org (STINNER Victor)
Date: Thu, 26 Jan 2017 02:18:12 +0000
Subject: [issue29259] Add tp_fastcall to PyTypeObject: support FASTCALL
 calling convention for all callable objects
In-Reply-To: <1484310274.99.0.832491900546.issue29259@psf.upfronthosting.co.za>
Message-ID: <1485397092.25.0.831368051011.issue29259@psf.upfronthosting.co.za>


Changes by STINNER Victor <victor.stinner at gmail.com>:


Removed file: http://bugs.python.org/file46418/tp_fastcall-6.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29259>
_______________________________________

From report at bugs.python.org  Wed Jan 25 21:19:38 2017
From: report at bugs.python.org (STINNER Victor)
Date: Thu, 26 Jan 2017 02:19:38 +0000
Subject: [issue29259] Add tp_fastcall to PyTypeObject: support FASTCALL
 calling convention for all callable objects
In-Reply-To: <1484310274.99.0.832491900546.issue29259@psf.upfronthosting.co.za>
Message-ID: <1485397178.86.0.214607742432.issue29259@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Oh, I had a local change not commited yet, so tp_fastcall-6.patch didn't get the [review] button. I just pushed the commit and then posted again the exact same patch.

----------
Added file: http://bugs.python.org/file46419/tp_fastcall-6.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29259>
_______________________________________

From report at bugs.python.org  Wed Jan 25 22:08:34 2017
From: report at bugs.python.org (=?utf-8?q?=C3=81d=C3=A1m_Szieberth?=)
Date: Thu, 26 Jan 2017 03:08:34 +0000
Subject: [issue29356] TypeError by typing for Ellipsis
In-Reply-To: <1485233572.17.0.570161775222.issue29356@psf.upfronthosting.co.za>
Message-ID: <1485400114.29.0.734950932653.issue29356@psf.upfronthosting.co.za>


?d?m Szieberth added the comment:

Well, now I believe the way I suggested would conflict with other uses of Ellipsis in type hints.

----------
resolution:  -> not a bug
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29356>
_______________________________________

From report at bugs.python.org  Wed Jan 25 22:58:07 2017
From: report at bugs.python.org (Ma Lin)
Date: Thu, 26 Jan 2017 03:58:07 +0000
Subject: [issue29121] sqlite3 Controlling Transactions documentation not
 updated
In-Reply-To: <1483200295.1.0.801871810477.issue29121@psf.upfronthosting.co.za>
Message-ID: <1485403087.75.0.666545510846.issue29121@psf.upfronthosting.co.za>


Ma Lin added the comment:

After read some issues, there are two major concerns:
1, backward compatibility.
2, implicit commit is bad, tolerates bug-prone codes.

However they are not contradictory if we print a warning when doing an implicit commit. Some messages like this:
    Warning: sqlite3 module issued an implicit commit before executing VACUUM, you should commit this transaction by yourself, that's a good habit, etc..

Then we can keep backward compatibility, and no longer commit implicitly.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29121>
_______________________________________

From report at bugs.python.org  Thu Jan 26 01:39:29 2017
From: report at bugs.python.org (Caitlin)
Date: Thu, 26 Jan 2017 06:39:29 +0000
Subject: [issue22651] Open file in a+ mode reads from end of file in Python 3.4
In-Reply-To: <1413446231.99.0.734842825619.issue22651@psf.upfronthosting.co.za>
Message-ID: <1485412769.78.0.395282198427.issue22651@psf.upfronthosting.co.za>


Caitlin added the comment:

Yeah, the workaround seemed to work for me when trying to open file on http://buywebsitetrafficreviews.org/, but this should be fixed somehow anyway.

----------
nosy: +caitlinwalker37

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue22651>
_______________________________________

From report at bugs.python.org  Thu Jan 26 02:35:57 2017
From: report at bugs.python.org (Roundup Robot)
Date: Thu, 26 Jan 2017 07:35:57 +0000
Subject: [issue26526] In parsermodule.c,
 replace over 2KLOC of hand-crafted validation code, with a DFA
In-Reply-To: <1457600551.84.0.318395917649.issue26526@psf.upfronthosting.co.za>
Message-ID: <20170126073554.105433.87337.79D2918F@psf.io>


Roundup Robot added the comment:

New changeset 3275d4b584a2 by Benjamin Peterson in branch '3.6':
remove comment about updating the parser module; we do not need to do that anymore (#26526)
https://hg.python.org/cpython/rev/3275d4b584a2

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue26526>
_______________________________________

From report at bugs.python.org  Thu Jan 26 02:56:08 2017
From: report at bugs.python.org (STINNER Victor)
Date: Thu, 26 Jan 2017 07:56:08 +0000
Subject: [issue22651] Open file in a+ mode reads from end of file in Python 3.4
In-Reply-To: <1413446231.99.0.734842825619.issue22651@psf.upfronthosting.co.za>
Message-ID: <1485417368.88.0.300174123341.issue22651@psf.upfronthosting.co.za>


Changes by STINNER Victor <victor.stinner at gmail.com>:


----------
Removed message: http://bugs.python.org/msg286294

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue22651>
_______________________________________

From report at bugs.python.org  Thu Jan 26 03:08:35 2017
From: report at bugs.python.org (STINNER Victor)
Date: Thu, 26 Jan 2017 08:08:35 +0000
Subject: [issue22651] Open file in a+ mode reads from end of file in Python 3.4
In-Reply-To: <1413446231.99.0.734842825619.issue22651@psf.upfronthosting.co.za>
Message-ID: <1485418115.67.0.801853687146.issue22651@psf.upfronthosting.co.za>


STINNER Victor added the comment:

On Python 2, it's easy to workaround the issue, just always go the end explicitly:

   f = open(filename, "a+")
   f.seek(0, os.SEEK_END)

Or use the io module which doesn't use the stdio of the C library and has a portable and reliable behaviour:

   f = io.open(filename, "a+")


Georg Brandl: "This is a somewhat unfortunate difference between the major C libs and Python's new IO, but probably too late to change."

Since it's easy to workaround the issue and only one user complained lat 2 years (3 years?), sorry, I close now the issue.

I add this bug to my long list of bugs in the C stdio which won't be fixed in Python 2...
http://haypo-notes.readthedocs.io/python.html#bugs-in-the-c-stdio-used-by-the-python-i-o

----------
resolution:  -> wont fix
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue22651>
_______________________________________

From report at bugs.python.org  Thu Jan 26 03:18:59 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Thu, 26 Jan 2017 08:18:59 +0000
Subject: [issue29368] Optimize unpickling list-like objects: use extend()
 instead of append()
In-Reply-To: <1485338150.33.0.560817507263.issue29368@psf.upfronthosting.co.za>
Message-ID: <1485418739.18.0.429375506456.issue29368@psf.upfronthosting.co.za>


Raymond Hettinger added the comment:

This code looks correct and reasonable.  The tests all pass for me.  I think you can go forward and apply the patch.

----------
assignee:  -> serhiy.storchaka
nosy: +rhettinger

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29368>
_______________________________________

From report at bugs.python.org  Thu Jan 26 03:44:09 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Thu, 26 Jan 2017 08:44:09 +0000
Subject: [issue29328] struct module should support variable-length strings
In-Reply-To: <1484850272.25.0.472309940141.issue29328@psf.upfronthosting.co.za>
Message-ID: <1485420249.85.0.590616168161.issue29328@psf.upfronthosting.co.za>


Raymond Hettinger added the comment:

s/Where/Whether/

----------
priority: normal -> low

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29328>
_______________________________________

From report at bugs.python.org  Thu Jan 26 03:46:56 2017
From: report at bugs.python.org (STINNER Victor)
Date: Thu, 26 Jan 2017 08:46:56 +0000
Subject: [issue29259] Add tp_fastcall to PyTypeObject: support FASTCALL
 calling convention for all callable objects
In-Reply-To: <1484310274.99.0.832491900546.issue29259@psf.upfronthosting.co.za>
Message-ID: <1485420416.94.0.0869215775916.issue29259@psf.upfronthosting.co.za>


STINNER Victor added the comment:

performance results on tp_fastcall-6.patch (diff >= 2%):
---
haypo at speed-python$ python3 -m perf compare_to /home/haypo/benchmarks/2017-01-23_14-02-default-cebc9c7ad195.json tp_fastcall-6_ref_cebc9c7ad195.json -G --min-speed=2
Slower (6):
- scimark_monte_carlo: 202 ms +- 5 ms -> 222 ms +- 8 ms: 1.10x slower (+10%)
- telco: 13.0 ms +- 0.2 ms -> 14.2 ms +- 0.4 ms: 1.09x slower (+9%)
- call_method: 11.0 ms +- 0.3 ms -> 11.5 ms +- 0.5 ms: 1.05x slower (+5%)
- scimark_sparse_mat_mult: 8.57 ms +- 0.14 ms -> 8.92 ms +- 0.15 ms: 1.04x slower (+4%)
- fannkuch: 938 ms +- 7 ms -> 974 ms +- 11 ms: 1.04x slower (+4%)
- scimark_fft: 667 ms +- 9 ms -> 690 ms +- 10 ms: 1.03x slower (+3%)

Faster (17):
- scimark_lu: 362 ms +- 13 ms -> 322 ms +- 8 ms: 1.12x faster (-11%)
- chameleon: 23.5 ms +- 0.3 ms -> 21.3 ms +- 0.3 ms: 1.10x faster (-9%)
- xml_etree_process: 179 ms +- 3 ms -> 169 ms +- 3 ms: 1.06x faster (-5%)
- xml_etree_generate: 208 ms +- 4 ms -> 199 ms +- 3 ms: 1.05x faster (-4%)
- logging_silent: 586 ns +- 8 ns -> 562 ns +- 7 ns: 1.04x faster (-4%)
- richards: 147 ms +- 2 ms -> 141 ms +- 3 ms: 1.04x faster (-4%)
- pickle_pure_python: 1.03 ms +- 0.02 ms -> 994 us +- 15 us: 1.04x faster (-4%)
- regex_compile: 379 ms +- 2 ms -> 367 ms +- 4 ms: 1.03x faster (-3%)
- raytrace: 1.08 sec +- 0.01 sec -> 1.04 sec +- 0.02 sec: 1.03x faster (-3%)
- unpickle: 31.0 us +- 0.4 us -> 30.0 us +- 0.4 us: 1.03x faster (-3%)
- pathlib: 42.3 ms +- 0.5 ms -> 41.0 ms +- 0.5 ms: 1.03x faster (-3%)
- scimark_sor: 391 ms +- 7 ms -> 379 ms +- 12 ms: 1.03x faster (-3%)
- genshi_xml: 167 ms +- 2 ms -> 162 ms +- 2 ms: 1.03x faster (-3%)
- django_template: 356 ms +- 5 ms -> 347 ms +- 4 ms: 1.03x faster (-3%)
- regex_dna: 283 ms +- 1 ms -> 276 ms +- 4 ms: 1.03x faster (-3%)
- hexiom: 18.0 ms +- 0.1 ms -> 17.6 ms +- 0.2 ms: 1.02x faster (-2%)
- call_simple: 10.7 ms +- 0.4 ms -> 10.5 ms +- 0.2 ms: 1.02x faster (-2%)

Benchmark hidden because not significant (41): (...)
---

Or less verbose output, only show difference >= 5%:
---
haypo at speed-python$ python3 -m perf compare_to /home/haypo/benchmarks/2017-01-23_14-02-default-cebc9c7ad195.json tp_fastcall-6_ref_cebc9c7ad195.json -G --min-speed=5
Slower (2):
- scimark_monte_carlo: 202 ms +- 5 ms -> 222 ms +- 8 ms: 1.10x slower (+10%)
- telco: 13.0 ms +- 0.2 ms -> 14.2 ms +- 0.4 ms: 1.09x slower (+9%)

Faster (3):
- scimark_lu: 362 ms +- 13 ms -> 322 ms +- 8 ms: 1.12x faster (-11%)
- chameleon: 23.5 ms +- 0.3 ms -> 21.3 ms +- 0.3 ms: 1.10x faster (-9%)
- xml_etree_process: 179 ms +- 3 ms -> 169 ms +- 3 ms: 1.06x faster (-5%)

Benchmark hidden because not significant (59): (...)
---

To be honest, I'm disappointed, I expected more faster benchmarks. Maybe the cost of the temporary tuple/dict to pass arguments to function is not as high as I expected.

telco is also my favorite benchmark, I'm sad that it's 1.09x slower. I should investigate that.

scimark_monte_carlo and scimark_lu benchmarks are still unstable for an unknown reasons, so I don't care too much of these two benchmarks.

* https://speed.python.org/timeline/#/?exe=5&ben=scimark_monte_carlo&env=1&revs=50&equid=off&quarts=on&extr=on
* https://speed.python.org/timeline/#/?exe=5&ben=scimark_lu&env=1&revs=50&equid=off&quarts=on&extr=on

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29259>
_______________________________________

From report at bugs.python.org  Thu Jan 26 04:32:34 2017
From: report at bugs.python.org (=?utf-8?q?C=C3=A9dric_Bellegarde?=)
Date: Thu, 26 Jan 2017 09:32:34 +0000
Subject: [issue28518] execute("begin immediate") throwing OperationalError
In-Reply-To: <1477307118.31.0.70785963389.issue28518@psf.upfronthosting.co.za>
Message-ID: <1485423154.33.0.599976592974.issue28518@psf.upfronthosting.co.za>


C?dric Bellegarde added the comment:

Adding a commit doesn't fix the issue here:
sql.commit()
sql.execute('VACUUM')

And on run:
cannot VACUUM from within a transaction

----------
nosy: +C?dric Bellegarde

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28518>
_______________________________________

From report at bugs.python.org  Thu Jan 26 04:52:43 2017
From: report at bugs.python.org (Andreas Bergmeier)
Date: Thu, 26 Jan 2017 09:52:43 +0000
Subject: [issue23462] All os.exec*e variants crash on Windows
In-Reply-To: <1423903526.7.0.282245151833.issue23462@psf.upfronthosting.co.za>
Message-ID: <1485424363.46.0.396095192454.issue23462@psf.upfronthosting.co.za>


Andreas Bergmeier added the comment:

Just hit that bug. Is this really not yet fixed in any newer version?

----------
nosy: +Andreas Bergmeier

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23462>
_______________________________________

From report at bugs.python.org  Thu Jan 26 05:00:11 2017
From: report at bugs.python.org (STINNER Victor)
Date: Thu, 26 Jan 2017 10:00:11 +0000
Subject: [issue23462] All os.exec*e variants crash on Windows
In-Reply-To: <1423903526.7.0.282245151833.issue23462@psf.upfronthosting.co.za>
Message-ID: <1485424811.45.0.123454562215.issue23462@psf.upfronthosting.co.za>


STINNER Victor added the comment:

> Just hit that bug. Is this really not yet fixed in any newer version?

The bug was fixed in Python 3.6.0. It was fixed by Steve Dower in its implementation of the PEP 529, in the change e20c7d8a8187:

-    execv(path_char, argvlist);
+#ifdef HAVE_WEXECV
+    _wexecv(path->wide, argvlist);
+#else
+    execv(path->narrow, argvlist);
+#endif

I'm not sure that it's easy to backport the fix in Python 3.5.

Python 3.6 no walways use Unicode internally to call the Windows API, whereas Python 3.5 used bytes or Unicode depending on the arguments.

So I suggest to upgrade to Python 3.6, or use bytes in the meanwhile.

Would it be ok to close the bug?

----------
nosy: +haypo

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23462>
_______________________________________

From report at bugs.python.org  Thu Jan 26 05:13:55 2017
From: report at bugs.python.org (STINNER Victor)
Date: Thu, 26 Jan 2017 10:13:55 +0000
Subject: [issue29259] Add tp_fastcall to PyTypeObject: support FASTCALL
 calling convention for all callable objects
In-Reply-To: <1484310274.99.0.832491900546.issue29259@psf.upfronthosting.co.za>
Message-ID: <1485425635.8.0.599918987638.issue29259@psf.upfronthosting.co.za>


STINNER Victor added the comment:

tp_fastcall-6.patch: "call_method: 11.0 ms +- 0.3 ms -> 11.5 ms +- 0.5 ms: 1.05x slower (+5%)"

I looked at the C code: the patch doesn't change any C function used by this benchmark. IHMO it's only a better of code locality. Maybe the PGO compilation produced less efficient code with the patch.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29259>
_______________________________________

From report at bugs.python.org  Thu Jan 26 05:16:18 2017
From: report at bugs.python.org (Manuel Krebber)
Date: Thu, 26 Jan 2017 10:16:18 +0000
Subject: [issue29377] Add the 'wrapper_descriptor' type to the types module
Message-ID: <1485425778.09.0.153066086326.issue29377@psf.upfronthosting.co.za>


New submission from Manuel Krebber:

There currently is no type in the types module for the slot wrappers/wrapper_descriptor types.

I would like to have something like

    WrapperDescriptor = type(object.__init__)

added to it (or maybe even add it to MethodType). This would be helpful to check more easily if some object is a method.

I can create a pull request for this if desired.

----------
components: Library (Lib)
messages: 286304
nosy: Wheerd
priority: normal
severity: normal
status: open
title: Add the 'wrapper_descriptor' type to the types module
type: enhancement

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29377>
_______________________________________

From report at bugs.python.org  Thu Jan 26 05:52:58 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Thu, 26 Jan 2017 10:52:58 +0000
Subject: [issue17720] pickle.py's load_appends should call append() on objects
 other than lists
In-Reply-To: <1365876289.91.0.454785839611.issue17720@psf.upfronthosting.co.za>
Message-ID: <1485427978.34.0.607439261237.issue17720@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

> From PEP 7:

Sorry, I meant PEP 307 of course.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue17720>
_______________________________________

From report at bugs.python.org  Thu Jan 26 06:18:04 2017
From: report at bugs.python.org (Ivan Levkivskyi)
Date: Thu, 26 Jan 2017 11:18:04 +0000
Subject: [issue29377] Add the 'wrapper_descriptor' type to the types module
In-Reply-To: <1485425778.09.0.153066086326.issue29377@psf.upfronthosting.co.za>
Message-ID: <1485429484.38.0.823737334299.issue29377@psf.upfronthosting.co.za>


Ivan Levkivskyi added the comment:

Manuel, it would be helpful if you submit a patch.

Guido, this is necessary for ``get_type_hints`` to work nicer with built-in methods, see https://github.com/python/typing/pull/368

----------
nosy: +gvanrossum, levkivskyi
versions: +Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29377>
_______________________________________

From report at bugs.python.org  Thu Jan 26 06:45:42 2017
From: report at bugs.python.org (INADA Naoki)
Date: Thu, 26 Jan 2017 11:45:42 +0000
Subject: [issue29259] Add tp_fastcall to PyTypeObject: support FASTCALL
 calling convention for all callable objects
In-Reply-To: <1484310274.99.0.832491900546.issue29259@psf.upfronthosting.co.za>
Message-ID: <1485431142.47.0.753379150913.issue29259@psf.upfronthosting.co.za>


INADA Naoki added the comment:

Another run at my machine. (tp_fastcall-6.patch)

+ ../python.default -m perf compare_to default.json patched.json -G --min-speed=2
Slower (8):
- logging_silent: 736 ns +- 7 ns -> 783 ns +- 12 ns: 1.06x slower (+6%)
- unpickle_pure_python: 829 us +- 13 us -> 875 us +- 12 us: 1.06x slower (+6%)
- scimark_sor: 482 ms +- 8 ms -> 507 ms +- 14 ms: 1.05x slower (+5%)
- xml_etree_iterparse: 223 ms +- 6 ms -> 228 ms +- 7 ms: 1.02x slower (+2%)
- scimark_fft: 728 ms +- 16 ms -> 745 ms +- 19 ms: 1.02x slower (+2%)
- scimark_sparse_mat_mult: 8.71 ms +- 0.32 ms -> 8.90 ms +- 0.28 ms: 1.02x slower (+2%)
- json_dumps: 29.3 ms +- 0.3 ms -> 29.9 ms +- 0.3 ms: 1.02x slower (+2%)
- pathlib: 51.2 ms +- 0.6 ms -> 52.3 ms +- 0.8 ms: 1.02x slower (+2%)

Faster (8):
- unpack_sequence: 132 ns +- 2 ns -> 123 ns +- 2 ns: 1.08x faster (-7%)
- xml_etree_parse: 319 ms +- 11 ms -> 303 ms +- 11 ms: 1.05x faster (-5%)
- call_simple: 14.3 ms +- 0.5 ms -> 13.7 ms +- 0.5 ms: 1.04x faster (-4%)
- call_method_unknown: 16.6 ms +- 1.1 ms -> 16.0 ms +- 0.2 ms: 1.04x faster (-4%)
- call_method: 15.1 ms +- 0.1 ms -> 14.5 ms +- 0.5 ms: 1.04x faster (-4%)
- xml_etree_process: 241 ms +- 4 ms -> 235 ms +- 4 ms: 1.03x faster (-3%)
- call_method_slots: 14.7 ms +- 0.2 ms -> 14.3 ms +- 0.2 ms: 1.02x faster (-2%)
- genshi_xml: 197 ms +- 3 ms -> 193 ms +- 2 ms: 1.02x faster (-2%)

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29259>
_______________________________________

From report at bugs.python.org  Thu Jan 26 07:24:41 2017
From: report at bugs.python.org (Ma Lin)
Date: Thu, 26 Jan 2017 12:24:41 +0000
Subject: [issue28518] execute("begin immediate") throwing OperationalError
In-Reply-To: <1477307118.31.0.70785963389.issue28518@psf.upfronthosting.co.za>
Message-ID: <1485433481.79.0.0219140835909.issue28518@psf.upfronthosting.co.za>


Ma Lin added the comment:

You can set isolation_level = None in sqlite3.connect() parameters, then sqlite3 module will not begin a transaction implicitly.

Another way is disabling auto-begin-transaction temporarily:
sql.isolation_level = None
sql.execute('VACUUM')
sql.isolation_level = ''  # <- note that this is the default value of isolation_level

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28518>
_______________________________________

From report at bugs.python.org  Thu Jan 26 07:36:14 2017
From: report at bugs.python.org (Eryk Sun)
Date: Thu, 26 Jan 2017 12:36:14 +0000
Subject: [issue29366] os.listdir has inconsistent behavior when run on a
 non-directory
In-Reply-To: <1485312192.81.0.934042138535.issue29366@psf.upfronthosting.co.za>
Message-ID: <1485434174.2.0.958765930846.issue29366@psf.upfronthosting.co.za>


Eryk Sun added the comment:

> FindFirstFile naturally fails with ERROR_PATH_NOT_FOUND

Getting this error actually depends on the file system. I don't see it with NTFS, which returns STATUS_NOT_A_DIRECTORY, which gets translated to ERROR_DIRECTORY. On the other hand, VboxSF (VirtualBox shared folder) returns STATUS_OBJECT_PATH_NOT_FOUND, which gets translated to ERROR_PATH_NOT_FOUND.

Also, there are many possible errors when trying to list a path on a device that isn't managed by a file system (e.g. \\.\PhysicalDrive0). The device can return a status value that FindFirstFile doesn't special case, such as STATUS_UNSUCCESSFUL, STATUS_NOT_FOUND, or STATUS_OBJECT_NAME_INVALID. The NtOpenFile call may even succeed, but then the NtQueryDirectoryFile will probably fail with STATUS_INVALID_DEVICE_REQUEST.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29366>
_______________________________________

From report at bugs.python.org  Thu Jan 26 07:40:55 2017
From: report at bugs.python.org (Nick Coghlan)
Date: Thu, 26 Jan 2017 12:40:55 +0000
Subject: [issue29270] super call in ctypes sub-class fails in 3.6
In-Reply-To: <1484345422.14.0.967631121608.issue29270@psf.upfronthosting.co.za>
Message-ID: <1485434455.06.0.427373900383.issue29270@psf.upfronthosting.co.za>


Nick Coghlan added the comment:

The scrapy case looks to just be the new metaclass constraint that's already covered in the "Porting to Python 3.6" guide: https://github.com/scrapy/scrapy/pull/2509/files

The ctypes case is more complicated, as its actually *reusing* the same class namespace to define two different classes, which is fundamentally incompatible with the way zero-argument super works.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29270>
_______________________________________

From report at bugs.python.org  Thu Jan 26 08:26:29 2017
From: report at bugs.python.org (Manuel Krebber)
Date: Thu, 26 Jan 2017 13:26:29 +0000
Subject: [issue29377] Add the 'wrapper_descriptor' type to the types module
In-Reply-To: <1485425778.09.0.153066086326.issue29377@psf.upfronthosting.co.za>
Message-ID: <1485437189.05.0.28078345272.issue29377@psf.upfronthosting.co.za>


Changes by Manuel Krebber <admin at wheerd.de>:


----------
keywords: +patch
Added file: http://bugs.python.org/file46420/0001-Added-SlotWrapperType-and-MethodWrapperType-to-the-t.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29377>
_______________________________________

From report at bugs.python.org  Thu Jan 26 08:34:24 2017
From: report at bugs.python.org (Manuel Krebber)
Date: Thu, 26 Jan 2017 13:34:24 +0000
Subject: [issue29377] Add the 'wrapper_descriptor' type to the types module
In-Reply-To: <1485425778.09.0.153066086326.issue29377@psf.upfronthosting.co.za>
Message-ID: <1485437664.21.0.532251633923.issue29377@psf.upfronthosting.co.za>


Manuel Krebber added the comment:

I would suggest the names SlotWrapperType and MethodWrapperType because I think they match their string representations the closest.

For checking whether something is a method/function one could also use inspect.isroutine (or inspect.ismethoddescriptor), but that is inconsistent with regards to builtin and python methods:

>>> import inspect
>>> inspect.isroutine(object.__init__)
True
>>> inspect.isroutine(object().__init__)
False
>>> class A:
...   def f(self):
...     pass
...
>>> inspect.isroutine(A.f)
True
>>> inspect.isroutine(A().f)
True

Maybe a function to detect the second case is needed.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29377>
_______________________________________

From report at bugs.python.org  Thu Jan 26 08:47:50 2017
From: report at bugs.python.org (Dima Tisnek)
Date: Thu, 26 Jan 2017 13:47:50 +0000
Subject: [issue28689] OpenSSL 1.1.0c test failures
In-Reply-To: <1479132647.76.0.450425015981.issue28689@psf.upfronthosting.co.za>
Message-ID: <1485438470.47.0.744197729344.issue28689@psf.upfronthosting.co.za>


Dima Tisnek added the comment:

1.1.0d is due today. Who wants to test it? :)

----------
nosy: +Dima.Tisnek

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28689>
_______________________________________

From report at bugs.python.org  Thu Jan 26 08:55:30 2017
From: report at bugs.python.org (INADA Naoki)
Date: Thu, 26 Jan 2017 13:55:30 +0000
Subject: [issue29259] Add tp_fastcall to PyTypeObject: support FASTCALL
 calling convention for all callable objects
In-Reply-To: <1484310274.99.0.832491900546.issue29259@psf.upfronthosting.co.za>
Message-ID: <1485438930.6.0.972711390281.issue29259@psf.upfronthosting.co.za>


INADA Naoki added the comment:

While I feel your work is great, performance benefit seems very small,
compared complexity of this patch.

I think AST level optimize is more important. co_const and co_stacksize
can be more small if constant folding is done in AST.

And I think global cache (#10401 or #28158) is important too.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29259>
_______________________________________

From report at bugs.python.org  Thu Jan 26 09:08:49 2017
From: report at bugs.python.org (STINNER Victor)
Date: Thu, 26 Jan 2017 14:08:49 +0000
Subject: [issue29259] Add tp_fastcall to PyTypeObject: support FASTCALL
 calling convention for all callable objects
In-Reply-To: <1484310274.99.0.832491900546.issue29259@psf.upfronthosting.co.za>
Message-ID: <1485439729.14.0.66968452831.issue29259@psf.upfronthosting.co.za>


STINNER Victor added the comment:

"While I feel your work is great, performance benefit seems very small,
compared complexity of this patch."

I have to agree. I spent a lot of times on benhchmarking these tp_fast* changes. While one or two benchmarks are faster, it's not really the case for the others.

I also agree with the complexity. In Python 3.6, most FASTCALL changes were internals. For example, using PyObject_CallFunctionObjArgs() now uses FASTCALL internally, without having to modify callers of the API. I tried to only use _PyObject_FastCallDict/Keywords() in a few places where the speedup was significant.

The main visible change of Python 3.6 FASTCALL is the new METH_CALL calling convention for C function. Your change modifying print() to use METH_CALL has a significant impact on the telco benchmark, without no drawback. I tested further changes to use METH_FASTCALL in struct and decimal modules, and they optimize telco even more.

To continue the optimization work, I guess that using METH_CALL in more cases, using Argument Clinic whenever possible, would have a more concrete and measurable impact on performances, than this big tp_fastcall patch.

But I'm not ready to abandon the whole approach yet, so I change the status to Pending. I may come back in one or two months, to check if I didn't miss anything obvious to unlock even more optimizations ;-)

----------
status: open -> pending

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29259>
_______________________________________

From report at bugs.python.org  Thu Jan 26 09:14:22 2017
From: report at bugs.python.org (STINNER Victor)
Date: Thu, 26 Jan 2017 14:14:22 +0000
Subject: [issue29358] Add tp_fastnew and tp_fastinit to PyTypeObject,
 15-20% faster object instanciation
In-Reply-To: <1485246329.89.0.254567368952.issue29358@psf.upfronthosting.co.za>
Message-ID: <1485440062.7.0.21095046098.issue29358@psf.upfronthosting.co.za>


STINNER Victor added the comment:

While they are obvious speedup on microbenchmarks, it doesn't speedup "macro" benchmarks from performance as much as I expected.

The changes required in typeobject.c to support "tp_new or tp_fastnew" and "tp_init or tp_fastinit" are large and very complex. I'm not sure that my code is correct. update_one_slot(), add_operators() and PyType_Ready() functions are complex beast, sadly almost not documented, except such comment...

            /* The __new__ wrapper is not a wrapper descriptor,
               so must be special-cased differently.
               If we don't do this, creating an instance will
               always use slot_tp_new which will look up
               __new__ in the MRO which will call tp_new_wrapper
               which will look through the base classes looking
               for a static base and call its tp_new (usually
               PyType_GenericNew), after performing various
               sanity checks and constructing a new argument
               list.  Cut all that nonsense short -- this speeds
               up instance creation tremendously. */
            specific = (void *)type->tp_new;
            /* XXX I'm not 100% sure that there isn't a hole
               in this reasoning that requires additional
               sanity checks.  I'll buy the first person to
               point out a bug in this reasoning a beer. */

How am I supposed to be confident in front of such coment :-D

I expect that calling a functions (tp_call) is a more common operation than instanciating a new object (tp_new + tp_init). So I don't think that the overall change is worth it.

For all these reaons, I close the issue as REJECTED.

----------
resolution:  -> rejected
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29358>
_______________________________________

From report at bugs.python.org  Thu Jan 26 09:52:42 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Thu, 26 Jan 2017 14:52:42 +0000
Subject: [issue29358] Add tp_fastnew and tp_fastinit to PyTypeObject,
 15-20% faster object instanciation
In-Reply-To: <1485246329.89.0.254567368952.issue29358@psf.upfronthosting.co.za>
Message-ID: <1485442362.22.0.270752826242.issue29358@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

As for update_one_slot() see also issue5322 and issue25731.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29358>
_______________________________________

From report at bugs.python.org  Thu Jan 26 09:58:30 2017
From: report at bugs.python.org (STINNER Victor)
Date: Thu, 26 Jan 2017 14:58:30 +0000
Subject: [issue29358] Add tp_fastnew and tp_fastinit to PyTypeObject,
 15-20% faster object instanciation
In-Reply-To: <1485246329.89.0.254567368952.issue29358@psf.upfronthosting.co.za>
Message-ID: <1485442710.91.0.578550436139.issue29358@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Serhiy Storchaka:
> As for update_one_slot() see also issue5322 and issue25731.

Oh, thanks for the pointers! Now I understand much better these bugs.

I'm quite sure that they are still flaws in this code when a type is modified after PyType_Ready(), like sysmodule.c::

    /* prevent user from creating new instances */
    FlagsType.tp_init = NULL;
    FlagsType.tp_new = NULL;


But each time I had to dig into typeobject.c, my head is going to explode :-D I may try to understand one more time, but not today ;-)

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29358>
_______________________________________

From report at bugs.python.org  Thu Jan 26 11:02:41 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Thu, 26 Jan 2017 16:02:41 +0000
Subject: [issue29368] Optimize unpickling list-like objects: use extend()
 instead of append()
In-Reply-To: <1485338150.33.0.560817507263.issue29368@psf.upfronthosting.co.za>
Message-ID: <1485446561.46.0.654367399716.issue29368@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

Updated patch addresses Antoine's comment. It adds the comment explaining a fallback.

----------
Added file: http://bugs.python.org/file46421/pickle-appends-extend-2.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29368>
_______________________________________

From report at bugs.python.org  Thu Jan 26 11:21:37 2017
From: report at bugs.python.org (Nick Coghlan)
Date: Thu, 26 Jan 2017 16:21:37 +0000
Subject: [issue26355] Emit major version based canonical URLs for docs
In-Reply-To: <1455353539.4.0.541474968967.issue26355@psf.upfronthosting.co.za>
Message-ID: <1485447697.52.0.789228428075.issue26355@psf.upfronthosting.co.za>


Nick Coghlan added the comment:

Belatedly following up on this, yeah, the RTFD page indicates that the header link should look like:

    <link rel="canonical" href="http://docs.python.org/3/<path>">

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue26355>
_______________________________________

From report at bugs.python.org  Thu Jan 26 11:24:26 2017
From: report at bugs.python.org (INADA Naoki)
Date: Thu, 26 Jan 2017 16:24:26 +0000
Subject: [issue29304] dict: simplify lookup function
In-Reply-To: <1484706973.14.0.775713839146.issue29304@psf.upfronthosting.co.za>
Message-ID: <1485447866.97.0.916579743079.issue29304@psf.upfronthosting.co.za>


INADA Naoki added the comment:

Digging history, duplicated code is introduced here. (1997-01-17)

https://github.com/python/cpython/commit/99304174680d4c724476dad300ae7fc638842bf0#diff-2131209d0deb0e50c93a88ec6c7b0d52

	/* Optimizations based on observations by Jyrki Alakuijala
	   (paraphrased):
	   - This routine is very heavily used, so should be AFAP
	   (As Fast As Possible).
	   - Most of the time, the first try is a hit or a definite
	   miss; so postpone the calculation of incr until we know the
	   first try was a miss.
	   - Write the loop twice, so we can move the test for
	   freeslot==NULL out of the loop.
	   - Write the loop using pointer increments and comparisons
	   rather than using an integer loop index.
	   Note that it behooves the compiler to calculate the values
	   of incr*sizeof(*ep) outside the loops and use this in the
	   increment of ep.  I've reduced the number of register
	   variables to the two most obvious candidates.

At the time, there was a single lookdict function, and three tries.

* The comment said the first try was for skipping `inc` calculation.
  While `inc` had been removed already, I think this implies skipping
  freeslot==NULL test.

* After first try, there are two loop for skipping freeslot==NULL test
  until first dummy found.  This optimization looks gone.


As I said above, lookdict_unicode_nodummy and lookdict_split only search
from table without dummies.
And lookdict_unicode() and lookdict() are not so important lookmapping() was in 1997,
duplicated code only for skipping one freeslot==NULL doesn't make sense.

One possible optimization is removing freeslot completely. because:

* freeslot is used only when inserting. finding is more important.
* insertdict() can find dummy quickly, only looking dk_indices.

But before trying optimization, I suggest to remove duplicated code first.
Macro bench doesn't show significant difference at least.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29304>
_______________________________________

From report at bugs.python.org  Thu Jan 26 11:25:04 2017
From: report at bugs.python.org (Kaeptm Blaubaer)
Date: Thu, 26 Jan 2017 16:25:04 +0000
Subject: [issue29378] Invalid example in documentation for PyErr_Fetch
Message-ID: <1485447904.37.0.693014852804.issue29378@psf.upfronthosting.co.za>


New submission from Kaeptm Blaubaer:

In the example are the pointers to pointers to PyObject too many,
because then PyErr_Fetch and PyErr_Restore would get too many pointers to pointers.

----------
assignee: docs at python
components: Documentation
messages: 286321
nosy: Kaeptm Blaubaer, docs at python
priority: normal
severity: normal
status: open
title: Invalid example in documentation for PyErr_Fetch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29378>
_______________________________________

From report at bugs.python.org  Thu Jan 26 11:28:41 2017
From: report at bugs.python.org (Kaeptm Blaubaer)
Date: Thu, 26 Jan 2017 16:28:41 +0000
Subject: [issue27727] Update Tools/freeze to use .vcxproj files
In-Reply-To: <1470839867.79.0.0217135468923.issue27727@psf.upfronthosting.co.za>
Message-ID: <1485448121.06.0.728141550403.issue27727@psf.upfronthosting.co.za>


Changes by Kaeptm Blaubaer <kaeptmblaubaer1000 at gmail.com>:


----------
type: behavior -> 

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue27727>
_______________________________________

From report at bugs.python.org  Thu Jan 26 11:28:50 2017
From: report at bugs.python.org (Kaeptm Blaubaer)
Date: Thu, 26 Jan 2017 16:28:50 +0000
Subject: [issue27727] Update Tools/freeze to use .vcxproj files
In-Reply-To: <1470839867.79.0.0217135468923.issue27727@psf.upfronthosting.co.za>
Message-ID: <1485448130.98.0.435132451778.issue27727@psf.upfronthosting.co.za>


Changes by Kaeptm Blaubaer <kaeptmblaubaer1000 at gmail.com>:


----------
type:  -> behavior

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue27727>
_______________________________________

From report at bugs.python.org  Thu Jan 26 11:32:05 2017
From: report at bugs.python.org (INADA Naoki)
Date: Thu, 26 Jan 2017 16:32:05 +0000
Subject: [issue29304] dict: simplify lookup function
In-Reply-To: <1484706973.14.0.775713839146.issue29304@psf.upfronthosting.co.za>
Message-ID: <1485448325.11.0.000218670602493.issue29304@psf.upfronthosting.co.za>


INADA Naoki added the comment:

BTW, perturb shift uses (i << 2) + i, instead of i*5.
I feel it doesn't make sense in 21th century.  I'll change it too.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29304>
_______________________________________

From report at bugs.python.org  Thu Jan 26 12:02:38 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Thu, 26 Jan 2017 17:02:38 +0000
Subject: [issue29304] dict: simplify lookup function
In-Reply-To: <1484706973.14.0.775713839146.issue29304@psf.upfronthosting.co.za>
Message-ID: <1485450158.63.0.140607456579.issue29304@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

I agreed that the duplication was made for reasons, but these reasons can be not valid or less weighty now, after a number of changes made last years like shared keys or compact dicts. An overhead of additional levels of indirection and complex code of dk_get_index() can be much larger than the benefit of code duplicating.

I think neither macrobenchmarks nor microbenchmarks would not show the true effect of this change. The code should be analyzed with more precise tools, similarly to how the set implementation was tuned. I think the patch should not be pushed without such analysis. Perhaps Raymond will found a time to do this.

----------
nosy: +serhiy.storchaka

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29304>
_______________________________________

From report at bugs.python.org  Thu Jan 26 12:05:56 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Thu, 26 Jan 2017 17:05:56 +0000
Subject: [issue29378] Invalid example in documentation for PyErr_Fetch
In-Reply-To: <1485447904.37.0.693014852804.issue29378@psf.upfronthosting.co.za>
Message-ID: <1485450356.96.0.55388065097.issue29378@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

The example looks correct to me. Could you please add more information? What example looks correct to you?

----------
nosy: +serhiy.storchaka

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29378>
_______________________________________

From report at bugs.python.org  Thu Jan 26 12:09:47 2017
From: report at bugs.python.org (Piotr Dobrogost)
Date: Thu, 26 Jan 2017 17:09:47 +0000
Subject: [issue29379] Custom handlers not used when passing "context" argument
 to urllib2.urlopen()
Message-ID: <1485450587.01.0.396469985739.issue29379@psf.upfronthosting.co.za>


New submission from Piotr Dobrogost:

When urllib2.urlopen() is passed "context" argument the new opener is being built (https://hg.python.org/cpython/file/a06454b1afa1/Lib/urllib2.py#l147) and used instead custom opener which might had been already installed with urllib2.install_opener(). This might lead to problems with proxies if custom ProxyHandler was used in custom opener ? see http://stackoverflow.com/q/36089694/95735 as an example.

----------
components: Library (Lib)
messages: 286325
nosy: piotr.dobrogost
priority: normal
severity: normal
status: open
title: Custom handlers not used when passing "context" argument to urllib2.urlopen()
type: behavior
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29379>
_______________________________________

From report at bugs.python.org  Thu Jan 26 12:27:24 2017
From: report at bugs.python.org (STINNER Victor)
Date: Thu, 26 Jan 2017 17:27:24 +0000
Subject: [issue29300] Modify the _struct module to use FASTCALL and Argument
 Clinic
In-Reply-To: <1484670705.59.0.0193443230555.issue29300@psf.upfronthosting.co.za>
Message-ID: <1485451644.48.0.172023468436.issue29300@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Patch version 2 converts most functions and methods to Argument Clinic, except of unpack() and unpack_into() which require "*args" support in Argument Clinic: see the issue #20291.

----------
Added file: http://bugs.python.org/file46422/struct_fastcall-2.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29300>
_______________________________________

From report at bugs.python.org  Thu Jan 26 12:27:58 2017
From: report at bugs.python.org (STINNER Victor)
Date: Thu, 26 Jan 2017 17:27:58 +0000
Subject: [issue29300] Modify the _struct module to use FASTCALL and Argument
 Clinic
In-Reply-To: <1484670705.59.0.0193443230555.issue29300@psf.upfronthosting.co.za>
Message-ID: <1485451678.09.0.1479559811.issue29300@psf.upfronthosting.co.za>


STINNER Victor added the comment:

I reformatted docstrings, and wrote a summary line.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29300>
_______________________________________

From report at bugs.python.org  Thu Jan 26 12:53:09 2017
From: report at bugs.python.org (INADA Naoki)
Date: Thu, 26 Jan 2017 17:53:09 +0000
Subject: [issue29304] dict: simplify lookup function
In-Reply-To: <1484706973.14.0.775713839146.issue29304@psf.upfronthosting.co.za>
Message-ID: <1485453189.73.0.344249685194.issue29304@psf.upfronthosting.co.za>


INADA Naoki added the comment:

>  I think the patch should not be pushed without such analysis. Perhaps Raymond will found a time to do this.

Ok, I won't push until expert's LGTM.

> One possible optimization is removing freeslot completely. because:
> 
> * freeslot is used only when inserting. finding is more important.
> * insertdict() can find dummy quickly, only looking dk_indices.
>
> But before trying optimization,

I found this is not only optimization, but also refactoring.
There is function for finding insertion point already, and insertdict()
and PyDict_SetDefault() use it in some cases.
So this can be done without adding any code.

+ ../python.default -m perf compare_to default.json patched.json -G --min-speed=2
Slower (5):
- logging_silent: 736 ns +- 7 ns -> 758 ns +- 15 ns: 1.03x slower (+3%)
- unpickle: 32.2 us +- 0.4 us -> 33.0 us +- 0.3 us: 1.03x slower (+3%)
- unpickle_pure_python: 829 us +- 13 us -> 849 us +- 13 us: 1.02x slower (+2%)
- meteor_contest: 200 ms +- 2 ms -> 204 ms +- 2 ms: 1.02x slower (+2%)
- scimark_sor: 482 ms +- 8 ms -> 493 ms +- 9 ms: 1.02x slower (+2%)

Faster (6):
- unpack_sequence: 132 ns +- 2 ns -> 123 ns +- 0 ns: 1.07x faster (-7%)
- call_simple: 14.3 ms +- 0.5 ms -> 13.7 ms +- 0.3 ms: 1.05x faster (-5%)
- sqlite_synth: 10.3 us +- 0.3 us -> 9.91 us +- 0.21 us: 1.03x faster (-3%)
- mako: 40.7 ms +- 0.5 ms -> 39.8 ms +- 0.3 ms: 1.02x faster (-2%)
- xml_etree_parse: 319 ms +- 11 ms -> 311 ms +- 14 ms: 1.02x faster (-2%)
- chameleon: 30.4 ms +- 0.4 ms -> 29.8 ms +- 0.3 ms: 1.02x faster (-2%)

Benchmark hidden because not significant (53): ...

----------
Added file: http://bugs.python.org/file46423/dict-refactoring-2.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29304>
_______________________________________

From report at bugs.python.org  Thu Jan 26 13:04:18 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Thu, 26 Jan 2017 18:04:18 +0000
Subject: [issue29300] Modify the _struct module to use FASTCALL and Argument
 Clinic
In-Reply-To: <1484670705.59.0.0193443230555.issue29300@psf.upfronthosting.co.za>
Message-ID: <1485453858.74.0.145969304847.issue29300@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

It looks to me that the Py_buffer converter can be used for all the buffer parameter and special purposed converter can be used for the fmt parameter.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29300>
_______________________________________

From report at bugs.python.org  Thu Jan 26 13:22:58 2017
From: report at bugs.python.org (Ivan Levkivskyi)
Date: Thu, 26 Jan 2017 18:22:58 +0000
Subject: [issue29377] Add the 'wrapper_descriptor' type to the types module
In-Reply-To: <1485425778.09.0.153066086326.issue29377@psf.upfronthosting.co.za>
Message-ID: <1485454978.23.0.355450761009.issue29377@psf.upfronthosting.co.za>


Ivan Levkivskyi added the comment:

Manuel, thank you for a patch!

Two comments:
1. Please produce your patch using Mercurial ``hg diff`` command, so that it could be recognized by review tool and merged easily.
2. Your patch should also include few tests (Lib/test/test_types.py) and documentation (Doc/library/types.rst)

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29377>
_______________________________________

From report at bugs.python.org  Thu Jan 26 13:54:32 2017
From: report at bugs.python.org (INADA Naoki)
Date: Thu, 26 Jan 2017 18:54:32 +0000
Subject: [issue29304] dict: simplify lookup function
In-Reply-To: <1484706973.14.0.775713839146.issue29304@psf.upfronthosting.co.za>
Message-ID: <1485456872.04.0.883683988827.issue29304@psf.upfronthosting.co.za>


INADA Naoki added the comment:

dict-refactoring-3.patch:

$ ../python.default -m perf compare_to default.json patched2.json -G --min-speed=2
Slower (7):
- scimark_lu: 422 ms +- 35 ms -> 442 ms +- 11 ms: 1.05x slower (+5%)
- logging_silent: 736 ns +- 7 ns -> 761 ns +- 21 ns: 1.03x slower (+3%)
- scimark_sor: 482 ms +- 8 ms -> 494 ms +- 7 ms: 1.03x slower (+3%)
- meteor_contest: 200 ms +- 2 ms -> 205 ms +- 2 ms: 1.02x slower (+2%)
- unpickle: 32.2 us +- 0.4 us -> 32.9 us +- 0.5 us: 1.02x slower (+2%)
- unpickle_pure_python: 829 us +- 13 us -> 848 us +- 14 us: 1.02x slower (+2%)
- scimark_sparse_mat_mult: 8.71 ms +- 0.32 ms -> 8.89 ms +- 0.13 ms: 1.02x slower (+2%)

Faster (8):
- unpack_sequence: 132 ns +- 2 ns -> 123 ns +- 2 ns: 1.07x faster (-7%)
- call_simple: 14.3 ms +- 0.5 ms -> 13.4 ms +- 0.3 ms: 1.07x faster (-6%)
- call_method: 15.1 ms +- 0.1 ms -> 14.5 ms +- 0.2 ms: 1.04x faster (-4%)
- mako: 40.7 ms +- 0.5 ms -> 39.6 ms +- 0.5 ms: 1.03x faster (-3%)
- scimark_monte_carlo: 266 ms +- 7 ms -> 258 ms +- 6 ms: 1.03x faster (-3%)
- chameleon: 30.4 ms +- 0.4 ms -> 29.6 ms +- 0.4 ms: 1.03x faster (-3%)
- xml_etree_parse: 319 ms +- 11 ms -> 312 ms +- 15 ms: 1.02x faster (-2%)
- pickle_pure_python: 1.28 ms +- 0.03 ms -> 1.26 ms +- 0.02 ms: 1.02x faster (-2%)

# microbench

$ ./python -m perf timeit --compare-to=`pwd`/python.default -s 'r=range(1000)' -- '{k:k for k in r}'                                                                               
python.default: ..................... 60.0 us +- 0.3 us
python: ..................... 61.7 us +- 0.4 us

Median +- std dev: [python.default] 60.0 us +- 0.3 us -> [python] 61.7 us +- 0.4 us: 1.03x slower (+3%)

$ ./python -m perf timeit --compare-to=`pwd`/python.default -s 'ks=[str(k) for k in range(1000)]; d={k:k for k in ks}' -- 'for k in ks: d[k]'                                      
python.default: ..................... 37.1 us +- 0.9 us
python: ..................... 37.7 us +- 0.9 us

Median +- std dev: [python.default] 37.1 us +- 0.9 us -> [python] 37.7 us +- 0.9 us: 1.02x slower (+2%)

Hmm, 3% slower?
I'll rerun benchmarks with PGO+LTO build.

----------
Added file: http://bugs.python.org/file46424/dict-refactoring-3.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29304>
_______________________________________

From report at bugs.python.org  Thu Jan 26 15:32:55 2017
From: report at bugs.python.org (Jeroen Demeyer)
Date: Thu, 26 Jan 2017 20:32:55 +0000
Subject: [issue13285] signal module ignores external signal changes
In-Reply-To: <1319801278.56.0.946220269602.issue13285@psf.upfronthosting.co.za>
Message-ID: <1485462775.99.0.803111868992.issue13285@psf.upfronthosting.co.za>


Jeroen Demeyer added the comment:

Here is a proposal for an API:

* getsignal: return the Python-level signal handler (this is an existing function)

* setsignal: set the Python-level signal handler (but not the OS-level signal handler)

* getossignal: get the OS-level signal handler as opaque object

* setossignal: set the OS-level signal handler with an opaque object

Using these primitives, you could implement anything that you want on a higher level.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue13285>
_______________________________________

From report at bugs.python.org  Thu Jan 26 16:13:35 2017
From: report at bugs.python.org (Jonathan Guthrie)
Date: Thu, 26 Jan 2017 21:13:35 +0000
Subject: [issue29375] httplib: wrong Host header when connecting to IPv6
 link-local address
In-Reply-To: <1485375284.1.0.310369100275.issue29375@psf.upfronthosting.co.za>
Message-ID: <1485465215.16.0.639363157458.issue29375@psf.upfronthosting.co.za>


Jonathan Guthrie added the comment:

Yes, it is more closely related to Issue 23448.  My search for related issues apparently wasn't exhaustive enough.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29375>
_______________________________________

From report at bugs.python.org  Thu Jan 26 16:22:28 2017
From: report at bugs.python.org (Jonathan Guthrie)
Date: Thu, 26 Jan 2017 21:22:28 +0000
Subject: [issue23448] urllib2 needs to remove scope from IPv6 address when
 creating Host header
In-Reply-To: <1423680023.83.0.120732364895.issue23448@psf.upfronthosting.co.za>
Message-ID: <1485465748.58.0.960719074359.issue23448@psf.upfronthosting.co.za>


Jonathan Guthrie added the comment:

Michael Sweet's draft RFC requiring that the scope should be included in the Host line expired in May 2014 and I can't find where it ever went anywhere.  Does anyone have any updated information?

----------
nosy: +JonathanGuthrie

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23448>
_______________________________________

From report at bugs.python.org  Thu Jan 26 17:15:59 2017
From: report at bugs.python.org (Manuel Krebber)
Date: Thu, 26 Jan 2017 22:15:59 +0000
Subject: [issue29377] Add the 'wrapper_descriptor' type to the types module
In-Reply-To: <1485425778.09.0.153066086326.issue29377@psf.upfronthosting.co.za>
Message-ID: <1485468959.79.0.667870367324.issue29377@psf.upfronthosting.co.za>


Manuel Krebber added the comment:

I added some docs, but I am not sure what I would want to test here. There are no tests for types.BuiltinMethodType either. Maybe the string representation of it could be tested, but an isinstance test seems pretty redundant. I hope this patch file works better, I created the last one with git diff.

----------
Added file: http://bugs.python.org/file46425/slot-wrapper-types.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29377>
_______________________________________

From report at bugs.python.org  Thu Jan 26 17:26:46 2017
From: report at bugs.python.org (Matthias Bussonnier)
Date: Thu, 26 Jan 2017 22:26:46 +0000
Subject: [issue26355] Emit major version based canonical URLs for docs
In-Reply-To: <1455353539.4.0.541474968967.issue26355@psf.upfronthosting.co.za>
Message-ID: <1485469606.05.0.0489279550073.issue26355@psf.upfronthosting.co.za>


Matthias Bussonnier added the comment:

Does this have to be implemented on the doc build of EOL pythons versions (like 2.6), or can it be a script which is ran once on these old docs ? 


One of the issues I had trying to implement that on other projects was that you don't know in advance what the future pages will be or if it will be gone. So you can go this route[1], in which case it's a 10 line fix that I'm happy to contribute also here. 

Typically `https://docs.python.org/2/c-api/string.html` has no cannonical on 3 whouch you can't know while building with sphinx. 

Or I though about injecting an html comment with a html comment that you search and replace once you publish the "new" version and only if the target page exists on the new version. 


So : 

- is that ok to have non existing cannonical ? 
- is that ok if adding <rel>s on docs is (or requires) a post-sphinx script ? 

1: https://github.com/xonsh/xonsh/pull/1914/files

Thanks

----------
nosy: +mbussonn

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue26355>
_______________________________________

From report at bugs.python.org  Thu Jan 26 18:06:13 2017
From: report at bugs.python.org (Ivan Levkivskyi)
Date: Thu, 26 Jan 2017 23:06:13 +0000
Subject: [issue29377] Add the 'wrapper_descriptor' type to the types module
In-Reply-To: <1485425778.09.0.153066086326.issue29377@psf.upfronthosting.co.za>
Message-ID: <1485471973.77.0.280282766673.issue29377@psf.upfronthosting.co.za>


Ivan Levkivskyi added the comment:

> but an isinstance test seems pretty redundant.

Tests are never redundant :-) Just add one-two asserts that you think should be true about issubclass and isinstance with these types (like self.assertIsInstance(''.__add__, types.MethodWrapperType)). String representation on the contrary is less important.

For some reason your patch is still not recognized by review tool. But don't worry about this, if it will not work, I will try to fix it.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29377>
_______________________________________

From report at bugs.python.org  Thu Jan 26 18:45:31 2017
From: report at bugs.python.org (Antoine Pitrou)
Date: Thu, 26 Jan 2017 23:45:31 +0000
Subject: [issue25667] Supply dual-stack (IPv4/IPv6) socket bind routine
In-Reply-To: <1447939973.83.0.965937708444.issue25667@psf.upfronthosting.co.za>
Message-ID: <1485474331.42.0.86532232202.issue25667@psf.upfronthosting.co.za>


Changes by Antoine Pitrou <pitrou at free.fr>:


----------
versions: +Python 3.7 -Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue25667>
_______________________________________

From report at bugs.python.org  Thu Jan 26 19:14:31 2017
From: report at bugs.python.org (Martin Panter)
Date: Fri, 27 Jan 2017 00:14:31 +0000
Subject: [issue15769] urllib.request.urlopen with cafile or capath set
 overrides any previous Handlers
In-Reply-To: <1345685709.12.0.30740709281.issue15769@psf.upfronthosting.co.za>
Message-ID: <1485476071.33.0.275105718222.issue15769@psf.upfronthosting.co.za>


Martin Panter added the comment:

Nothing has been fixed; I don?t see any evidence that this is ?out of date?. Here is a more complete test:

import urllib.request
opener = urllib.request.build_opener(urllib.request.HTTPCookieProcessor())
urllib.request.install_opener(opener)
request = 'https://httpbin.org/cookies/set?name=value'
# Download certifiate chain for https://httpbin.org/ into cacert.pem (I used Firefox)
sock = urllib.request.urlopen(request, cafile = 'cacert.pem')
sock.read()  # b'{\n  "cookies": {}\n}\n' (No cookies!)
sock.close()
sock = urllib.request.urlopen(request)  # Default SSL settings
sock.read()  # b'{\n  "cookies": {\n    "name": "value"\n  }\n}\n'

I?m not comfortable with applying a patch that ?doesn?t mangle [the global state] too much?. Anyway, this is the same as Issue 18543, which has slightly more recent discussion.

----------
nosy: +martin.panter
resolution: out of date -> duplicate
status: pending -> closed
superseder:  -> urllib.parse.urlopen shouldn't ignore installed opener when called with any SSL argument

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue15769>
_______________________________________

From report at bugs.python.org  Thu Jan 26 19:14:35 2017
From: report at bugs.python.org (Matthias Bussonnier)
Date: Fri, 27 Jan 2017 00:14:35 +0000
Subject: [issue26355] Emit major version based canonical URLs for docs
In-Reply-To: <1455353539.4.0.541474968967.issue26355@psf.upfronthosting.co.za>
Message-ID: <1485476075.56.0.64546187859.issue26355@psf.upfronthosting.co.za>


Matthias Bussonnier added the comment:

Here is a patch that add a canonical link to the current documentation that should apply cleanly to 3.4 and above. I can do similar for older versions.

----------
keywords: +patch
Added file: http://bugs.python.org/file46426/cannonical-doc-for-3.4plus.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue26355>
_______________________________________

From report at bugs.python.org  Thu Jan 26 19:21:38 2017
From: report at bugs.python.org (Martin Panter)
Date: Fri, 27 Jan 2017 00:21:38 +0000
Subject: [issue29379] Custom handlers not used when passing "context" argument
 to urllib2.urlopen()
In-Reply-To: <1485450587.01.0.396469985739.issue29379@psf.upfronthosting.co.za>
Message-ID: <1485476498.98.0.648777621046.issue29379@psf.upfronthosting.co.za>


Martin Panter added the comment:

I presume this is the same as in Issue 18543 (and a few other duplicates). Let me know if I got it wrong.

IMO there is no easy fix. The best solution may be to just document the behaviour as a limitation of the API, and design a new/improved API for the future (Py 3.7+) that handles these cases.

----------
nosy: +martin.panter
resolution:  -> duplicate
status: open -> closed
superseder:  -> urllib.parse.urlopen shouldn't ignore installed opener when called with any SSL argument

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29379>
_______________________________________

From report at bugs.python.org  Fri Jan 27 02:55:18 2017
From: report at bugs.python.org (Manuel Krebber)
Date: Fri, 27 Jan 2017 07:55:18 +0000
Subject: [issue29377] Add the 'wrapper_descriptor' type to the types module
In-Reply-To: <1485425778.09.0.153066086326.issue29377@psf.upfronthosting.co.za>
Message-ID: <1485503718.94.0.511301167786.issue29377@psf.upfronthosting.co.za>


Manuel Krebber added the comment:

Alright, I added some tests and tried it again with the patch.

----------
Added file: http://bugs.python.org/file46427/slot-wrapper-types.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29377>
_______________________________________

From report at bugs.python.org  Fri Jan 27 03:09:41 2017
From: report at bugs.python.org (Manuel Krebber)
Date: Fri, 27 Jan 2017 08:09:41 +0000
Subject: [issue26355] Emit major version based canonical URLs for docs
In-Reply-To: <1455353539.4.0.541474968967.issue26355@psf.upfronthosting.co.za>
Message-ID: <1485504581.56.0.472971565241.issue26355@psf.upfronthosting.co.za>


Manuel Krebber added the comment:

I create the last diff without creating a commit, so maybe this one works better.

----------
nosy: +Wheerd
Added file: http://bugs.python.org/file46428/slot-wrapper-types.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue26355>
_______________________________________

From report at bugs.python.org  Fri Jan 27 03:10:11 2017
From: report at bugs.python.org (Manuel Krebber)
Date: Fri, 27 Jan 2017 08:10:11 +0000
Subject: [issue26355] Emit major version based canonical URLs for docs
In-Reply-To: <1455353539.4.0.541474968967.issue26355@psf.upfronthosting.co.za>
Message-ID: <1485504611.17.0.759309630481.issue26355@psf.upfronthosting.co.za>


Changes by Manuel Krebber <admin at wheerd.de>:


Removed file: http://bugs.python.org/file46428/slot-wrapper-types.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue26355>
_______________________________________

From report at bugs.python.org  Fri Jan 27 03:11:11 2017
From: report at bugs.python.org (Manuel Krebber)
Date: Fri, 27 Jan 2017 08:11:11 +0000
Subject: [issue29377] Add the 'wrapper_descriptor' type to the types module
In-Reply-To: <1485425778.09.0.153066086326.issue29377@psf.upfronthosting.co.za>
Message-ID: <1485504671.61.0.0486749352161.issue29377@psf.upfronthosting.co.za>


Manuel Krebber added the comment:

I created the last patch without commiting, so maybe this one will work properly with the revision tool.

----------
Added file: http://bugs.python.org/file46429/slot-wrapper-types.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29377>
_______________________________________

From report at bugs.python.org  Fri Jan 27 03:12:23 2017
From: report at bugs.python.org (Manuel Krebber)
Date: Fri, 27 Jan 2017 08:12:23 +0000
Subject: [issue26355] Emit major version based canonical URLs for docs
In-Reply-To: <1455353539.4.0.541474968967.issue26355@psf.upfronthosting.co.za>
Message-ID: <1485504743.13.0.254913015679.issue26355@psf.upfronthosting.co.za>


Manuel Krebber added the comment:

Sorry, I accidentally replied to the worng issue -.-

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue26355>
_______________________________________

From report at bugs.python.org  Fri Jan 27 03:22:56 2017
From: report at bugs.python.org (Senthil Kumaran)
Date: Fri, 27 Jan 2017 08:22:56 +0000
Subject: [issue29182] Remove the warning in urllib docs that it doesn't do
 certificate validate by default.
In-Reply-To: <1483722117.81.0.319313681571.issue29182@psf.upfronthosting.co.za>
Message-ID: <1485505376.5.0.463285406847.issue29182@psf.upfronthosting.co.za>


Senthil Kumaran added the comment:

Hi Julian, here is the patch that addresses you comments. Thanks

----------
Added file: http://bugs.python.org/file46430/issue29182-patch3.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29182>
_______________________________________

From report at bugs.python.org  Fri Jan 27 03:53:57 2017
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 27 Jan 2017 08:53:57 +0000
Subject: [issue29300] Modify the _struct module to use FASTCALL and Argument
 Clinic
In-Reply-To: <1484670705.59.0.0193443230555.issue29300@psf.upfronthosting.co.za>
Message-ID: <1485507237.81.0.764966280061.issue29300@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Patch version 3:

* Use also Argument Clinic for Struct.__init__()
* Rename fmt to format: in the code and docstring. By the way, Struct docstring was wrong: Struct() accepts a 'format' keyword argument, not 'fmt'
* Use Py_buffer converter for the buffer parameter, except for iter_unpack()
* Document that unpack_from() now accepts keywords and write an unit test
* Fix Struct.__init__() error message: the method accepts also Unicode
* Document that Struct accepts str encodable to ASCII and bytes

I would be nice to use Py_buffer format for the "buffer" argument of iter_unpack(), but Argument Clinic calls PyBuffer_Release(&buffer), whereas the buffer should stay alive after the function exit. Is there a way to clone/copy a Py_buffer?

For the fmt/format argument: s_init() uses custom code to accept Unicode encodable to ASCII and bytes objects. Using Argument Clinic to check fmt type would require to write a converter. I would prefer to not make too many changes in a single patch, and I don't know how to write such converter. I suggest to do that later and keep the existing code. It seems like all functions getting a format ends in s_init() to check the format argument.

----------
Added file: http://bugs.python.org/file46431/struct_fastcall-3.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29300>
_______________________________________

From report at bugs.python.org  Fri Jan 27 04:30:02 2017
From: report at bugs.python.org (Julian Berman)
Date: Fri, 27 Jan 2017 09:30:02 +0000
Subject: [issue29182] Remove the warning in urllib docs that it doesn't do
 certificate validate by default.
In-Reply-To: <1483722117.81.0.319313681571.issue29182@psf.upfronthosting.co.za>
Message-ID: <1485509402.3.0.280435363377.issue29182@psf.upfronthosting.co.za>


Julian Berman added the comment:

Cool! If I can nitpick one last time, in the versionchanged block, you have `HTTPS URIs` but in the warning you use `HTTPS Urls` -- probably best to use consistent casing and URL vs URI.

Other than that, lgtm to merge.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29182>
_______________________________________

From report at bugs.python.org  Fri Jan 27 04:49:13 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Fri, 27 Jan 2017 09:49:13 +0000
Subject: [issue29300] Modify the _struct module to use FASTCALL and Argument
 Clinic
In-Reply-To: <1484670705.59.0.0193443230555.issue29300@psf.upfronthosting.co.za>
Message-ID: <1485510553.24.0.397613645704.issue29300@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

The converter for the format parameter already exists. It is cache_struct(). You just need to change its interface.

> * Fix Struct.__init__() error message: the method accepts also Unicode
> * Document that Struct accepts str encodable to ASCII and bytes

I think this should be backported to older versions. Maybe there is an open issue for this.

> I would be nice to use Py_buffer format for the "buffer" argument of iter_unpack(), but Argument Clinic calls PyBuffer_Release(&buffer), whereas the buffer should stay alive after the function exit. Is there a way to clone/copy a Py_buffer?

I agreed that the Py_buffer converter is not applicable here. Just object is good.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29300>
_______________________________________

From report at bugs.python.org  Fri Jan 27 05:16:46 2017
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 27 Jan 2017 10:16:46 +0000
Subject: [issue29300] Modify the _struct module to use FASTCALL and Argument
 Clinic
In-Reply-To: <1484670705.59.0.0193443230555.issue29300@psf.upfronthosting.co.za>
Message-ID: <1485512206.43.0.0257577541372.issue29300@psf.upfronthosting.co.za>


STINNER Victor added the comment:

> The converter for the format parameter already exists. It is cache_struct(). You just need to change its interface.

The code to convert the format string is in s_init(). The code increases the reference counter, I don't see how to produce the Py_DECREF() code.

Again, I would prefer to do that later. The change is already big enough :-)


> I think this should be backported to older versions.

Ok, will do once this change is accepted.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29300>
_______________________________________

From report at bugs.python.org  Fri Jan 27 05:40:47 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Fri, 27 Jan 2017 10:40:47 +0000
Subject: [issue29300] Modify the _struct module to use FASTCALL and Argument
 Clinic
In-Reply-To: <1484670705.59.0.0193443230555.issue29300@psf.upfronthosting.co.za>
Message-ID: <1485513647.11.0.304890002758.issue29300@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

static int
cache_struct_converter(PyObject *arg, PyObject **s_object)
{
    if (arg == NULL) {
        Py_DECREF();
        return 1;
    }
    *s_object = cache_struct(arg); // actually inline this
    if (*s_object == NULL)
        return 0;
    return Py_CLEANUP_SUPPORTED;
}

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29300>
_______________________________________

From report at bugs.python.org  Fri Jan 27 05:42:23 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Fri, 27 Jan 2017 10:42:23 +0000
Subject: [issue29300] Modify the _struct module to use FASTCALL and Argument
 Clinic
In-Reply-To: <1484670705.59.0.0193443230555.issue29300@psf.upfronthosting.co.za>
Message-ID: <1485513743.86.0.065628856436.issue29300@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

See also ascii_buffer_converter in Modules/binascii.c for example.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29300>
_______________________________________

From report at bugs.python.org  Fri Jan 27 07:44:05 2017
From: report at bugs.python.org (Roundup Robot)
Date: Fri, 27 Jan 2017 12:44:05 +0000
Subject: [issue29308] venv should match virtualenv VIRTUAL_ENV_DISABLE_PROMPT
 config
In-Reply-To: <1484750377.25.0.809506276992.issue29308@psf.upfronthosting.co.za>
Message-ID: <20170127124400.42540.34768.A6D6B0F1@psf.io>


Roundup Robot added the comment:

New changeset 91b949dc1984 by Vinay Sajip in branch '3.5':
Fixes #29308: Respect VIRTUAL_ENV_DISABLE_PROMPT in Activate.ps1.
https://hg.python.org/cpython/rev/91b949dc1984

New changeset d3890b44159c by Vinay Sajip in branch '3.6':
Fixes #29308: Merged fix from 3.5.
https://hg.python.org/cpython/rev/d3890b44159c

New changeset aef895fef120 by Vinay Sajip in branch 'default':
Closes #29308: Merged fix from 3.6.
https://hg.python.org/cpython/rev/aef895fef120

----------
nosy: +python-dev
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29308>
_______________________________________

From report at bugs.python.org  Fri Jan 27 08:05:18 2017
From: report at bugs.python.org (Roundup Robot)
Date: Fri, 27 Jan 2017 13:05:18 +0000
Subject: [issue28784] shlex.shlex punctuation_chars documentation should use
 posix=True
In-Reply-To: <1479973450.22.0.44978748963.issue28784@psf.upfronthosting.co.za>
Message-ID: <20170127130515.104192.29003.70A07087@psf.io>


Roundup Robot added the comment:

New changeset ff3312ce1d14 by Vinay Sajip in branch '3.6':
Fixes #28784: Clarified use of shlex.shlex with punctuation_chars.
https://hg.python.org/cpython/rev/ff3312ce1d14

New changeset 46f8188f8646 by Vinay Sajip in branch 'default':
Closes #28784: Merged update from 3.6.
https://hg.python.org/cpython/rev/46f8188f8646

----------
nosy: +python-dev
resolution:  -> fixed
stage:  -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28784>
_______________________________________

From report at bugs.python.org  Fri Jan 27 08:06:04 2017
From: report at bugs.python.org (INADA Naoki)
Date: Fri, 27 Jan 2017 13:06:04 +0000
Subject: [issue1346238] A constant folding optimization pass for the AST
Message-ID: <1485522364.26.0.31362269259.issue1346238@psf.upfronthosting.co.za>


Changes by INADA Naoki <songofacandy at gmail.com>:


----------
nosy: +inada.naoki

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue1346238>
_______________________________________

From report at bugs.python.org  Fri Jan 27 08:06:11 2017
From: report at bugs.python.org (INADA Naoki)
Date: Fri, 27 Jan 2017 13:06:11 +0000
Subject: [issue11549] Build-out an AST optimizer,
 moving some functionality out of the peephole optimizer
In-Reply-To: <1300164402.79.0.766591131661.issue11549@psf.upfronthosting.co.za>
Message-ID: <1485522371.06.0.148507466973.issue11549@psf.upfronthosting.co.za>


Changes by INADA Naoki <songofacandy at gmail.com>:


----------
nosy: +inada.naoki

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue11549>
_______________________________________

From report at bugs.python.org  Fri Jan 27 08:10:32 2017
From: report at bugs.python.org (Antoine Pitrou)
Date: Fri, 27 Jan 2017 13:10:32 +0000
Subject: [issue25667] Supply dual-stack (IPv4/IPv6) socket bind routine
In-Reply-To: <1447939973.83.0.965937708444.issue25667@psf.upfronthosting.co.za>
Message-ID: <1485522632.76.0.879390633049.issue25667@psf.upfronthosting.co.za>


Changes by Antoine Pitrou <pitrou at free.fr>:


----------
nosy: +pitrou

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue25667>
_______________________________________

From report at bugs.python.org  Fri Jan 27 08:12:17 2017
From: report at bugs.python.org (Antoine Pitrou)
Date: Fri, 27 Jan 2017 13:12:17 +0000
Subject: [issue25667] Supply dual-stack (IPv4/IPv6) socket bind routine
In-Reply-To: <1447939973.83.0.965937708444.issue25667@psf.upfronthosting.co.za>
Message-ID: <1485522737.55.0.266341708378.issue25667@psf.upfronthosting.co.za>


Antoine Pitrou added the comment:

Related though with a different strategy: Tornado's bind_sockets() routine:
https://github.com/tornadoweb/tornado/blob/master/tornado/netutil.py#L114-L138

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue25667>
_______________________________________

From report at bugs.python.org  Fri Jan 27 08:47:35 2017
From: report at bugs.python.org (Mirko Tebaldi (Realtebo))
Date: Fri, 27 Jan 2017 13:47:35 +0000
Subject: [issue29380] A lot of warning while executing make install
Message-ID: <1485524855.82.0.489793661635.issue29380@psf.upfronthosting.co.za>


New submission from Mirko Tebaldi (Realtebo):

/usr/src/Python-3.6.0/Modules/_ctypes/libffi/src/x86/ffi64.c: In function 'classify_argument':
/usr/src/Python-3.6.0/Modules/_ctypes/libffi/src/x86/ffi64.c:195:18: warning: suggest braces around empty body in an 'else' statement [-Wempty-body]
    FFI_ASSERT (0);
                  ^
/usr/src/Python-3.6.0/Modules/_ctypes/libffi/src/x86/ffi64.c:224:16: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
  for (i = 0; i < words; i++)
                ^
/usr/src/Python-3.6.0/Modules/_ctypes/libffi/src/x86/ffi64.c:245:20: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
      for (i = 0; i < num; i++)
                    ^
/usr/src/Python-3.6.0/Modules/_ctypes/libffi/src/x86/ffi64.c:264:20: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
      for (i = 1; i < words; i++)
                    ^
/usr/src/Python-3.6.0/Modules/_ctypes/libffi/src/x86/ffi64.c:270:16: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
  for (i = 0; i < words; i++)
                ^
/usr/src/Python-3.6.0/Modules/_ctypes/libffi/src/x86/ffi64.c: In function 'examine_argument':
/usr/src/Python-3.6.0/Modules/_ctypes/libffi/src/x86/ffi64.c:323:17: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
   for (i = 0; i < n; ++i)
                 ^
/usr/src/Python-3.6.0/Modules/_ctypes/libffi/src/x86/ffi64.c: In function 'ffi_call':
/usr/src/Python-3.6.0/Modules/_ctypes/libffi/src/x86/ffi64.c:484:18: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
    for (j = 0; j < n; j++, a += 8, size -= 8)
                  ^
/usr/src/Python-3.6.0/Modules/_ctypes/libffi/src/x86/ffi64.c: In function 'ffi_closure_unix64_inner':
/usr/src/Python-3.6.0/Modules/_ctypes/libffi/src/x86/ffi64.c:659:18: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
    for (j = 0; j < n; j++, a += 8)
                  ^

----------
messages: 286355
nosy: Mirko Tebaldi (Realtebo)
priority: normal
severity: normal
status: open
title: A lot of warning while executing make install
type: compile error
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29380>
_______________________________________

From report at bugs.python.org  Fri Jan 27 09:04:37 2017
From: report at bugs.python.org (Eric V. Smith)
Date: Fri, 27 Jan 2017 14:04:37 +0000
Subject: [issue29380] A lot of warning while executing make install
In-Reply-To: <1485524855.82.0.489793661635.issue29380@psf.upfronthosting.co.za>
Message-ID: <1485525877.26.0.896114835926.issue29380@psf.upfronthosting.co.za>


Eric V. Smith added the comment:

Can you give us your platform, platform version, compiler, and compiler version? Also, the entire output of "make install" (or whatever make command you're running).

Thanks.

----------
nosy: +eric.smith

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29380>
_______________________________________

From report at bugs.python.org  Fri Jan 27 09:40:05 2017
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 27 Jan 2017 14:40:05 +0000
Subject: [issue29300] Modify the _struct module to use FASTCALL and Argument
 Clinic
In-Reply-To: <1484670705.59.0.0193443230555.issue29300@psf.upfronthosting.co.za>
Message-ID: <1485528005.77.0.696284466403.issue29300@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Thank you for your reviews Serhiy :-) Here is a new patch to take in account your latest comments.

I didn't change the API anymore: Struct.unpack_from() accepts keywords, unpack_from() doesn't.

----------
Added file: http://bugs.python.org/file46432/struct_fastcall-4.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29300>
_______________________________________

From report at bugs.python.org  Fri Jan 27 09:45:51 2017
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 27 Jan 2017 14:45:51 +0000
Subject: [issue29300] Modify the _struct module to use FASTCALL and Argument
 Clinic
In-Reply-To: <1484670705.59.0.0193443230555.issue29300@psf.upfronthosting.co.za>
Message-ID: <1485528351.1.0.11972480454.issue29300@psf.upfronthosting.co.za>


STINNER Victor added the comment:

> See also ascii_buffer_converter in Modules/binascii.c for example.

Nice. I didn't know the "def cleanup" feature. I used it in my patch version 4.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29300>
_______________________________________

From report at bugs.python.org  Fri Jan 27 09:46:48 2017
From: report at bugs.python.org (Michael Haubenwallner)
Date: Fri, 27 Jan 2017 14:46:48 +0000
Subject: [issue27435] ctypes library loading and AIX - also for 2.7.X (and
 later)
In-Reply-To: <1467383699.63.0.495142233463.issue27435@psf.upfronthosting.co.za>
Message-ID: <1485528408.91.0.539433400317.issue27435@psf.upfronthosting.co.za>


Changes by Michael Haubenwallner <michael.haubenwallner at ssi-schaefer.com>:


----------
nosy: +haubi

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue27435>
_______________________________________

From report at bugs.python.org  Fri Jan 27 09:56:35 2017
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 27 Jan 2017 14:56:35 +0000
Subject: [issue29301] decimal: Use FASTCALL and/or Argument Clinic
In-Reply-To: <1484671124.21.0.493200110844.issue29301@psf.upfronthosting.co.za>
Message-ID: <1485528995.27.0.978406224904.issue29301@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Hello, I'm back! :-)

I almost abandonned my the tp_fastcall change (#29259).

print() now uses FASTCALL (#29296), it already made bm_telco faster:
https://speed.python.org/timeline/#/?exe=5&ben=telco&env=1&revs=50&equid=off&quarts=on&extr=on
(see the speedup around January 16 at the right)

For the unpack module, I'm still working on my patch in the issue #29300, but it doesn't seem to have a significant impact on bm_telco.

So the remaining question is the usage of FASTCALL or Argument Clinic in the _decimal module.


> Since the performance impact is unclear, I prefer to close this issue. I will come back if argument parsing is more clearly identified as a performance bottleneck.

I ran a new benchmark and FASTCALL makes bm_telco benchmark 1.11x faster:

Median +- std dev: [ref] 19.4 ms +- 0.7 ms -> [decimal-2.patch] 17.5 ms +- 0.6 ms: 1.11x faster (-10%)

So I decided to reopen the issue.

Stefan: what do you think of using Argument Clinic and/or FASTCALL in _decimal? Is "bm_telco 1.11x faster" significant enough for you to justify such change?

"The best would be to use Argument Clinic. AC exists since Python 3.5, so it should be possible to keep the same code base on Python 3.5-3.7, only generated code would be different."

----------
resolution: rejected -> 
status: closed -> open

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29301>
_______________________________________

From report at bugs.python.org  Fri Jan 27 09:57:37 2017
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 27 Jan 2017 14:57:37 +0000
Subject: [issue29346] datetime doesn't check for min/max dates anymore in
 Python 3.6.0 => crash
In-Reply-To: <1485107254.51.0.0185006280427.issue29346@psf.upfronthosting.co.za>
Message-ID: <1485529057.47.0.302688186382.issue29346@psf.upfronthosting.co.za>


Changes by STINNER Victor <victor.stinner at gmail.com>:


----------
title: datetime.utcfromtimestamp() returns strange result for very large values -> datetime doesn't check for min/max dates anymore in Python 3.6.0 => crash

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29346>
_______________________________________

From report at bugs.python.org  Fri Jan 27 09:58:10 2017
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 27 Jan 2017 14:58:10 +0000
Subject: [issue29346] datetime doesn't check for min/max dates anymore in
 Python 3.6.0 => crash
In-Reply-To: <1485107254.51.0.0185006280427.issue29346@psf.upfronthosting.co.za>
Message-ID: <1485529090.23.0.127350678234.issue29346@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Since it's a duplicate, I close this issue.

----------
status: open -> closed
superseder:  -> datetime.fromtimestamp() doesn't check min/max year anymore: regression of Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29346>
_______________________________________

From report at bugs.python.org  Fri Jan 27 09:58:14 2017
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 27 Jan 2017 14:58:14 +0000
Subject: [issue29100] datetime.fromtimestamp() doesn't check min/max year
 anymore: regression of Python 3.6
In-Reply-To: <1482972647.78.0.524140289263.issue29100@psf.upfronthosting.co.za>
Message-ID: <1485529094.21.0.113092311639.issue29100@psf.upfronthosting.co.za>


Changes by STINNER Victor <victor.stinner at gmail.com>:


----------
nosy: +Eli Collins, lemburg

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29100>
_______________________________________

From report at bugs.python.org  Fri Jan 27 09:58:33 2017
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 27 Jan 2017 14:58:33 +0000
Subject: [issue29100] datetime.fromtimestamp() doesn't check min/max year
 anymore: regression of Python 3.6
In-Reply-To: <1482972647.78.0.524140289263.issue29100@psf.upfronthosting.co.za>
Message-ID: <1485529113.92.0.837235254623.issue29100@psf.upfronthosting.co.za>


STINNER Victor added the comment:

The issue #29346 has been marked as a duplicate of this one.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29100>
_______________________________________

From report at bugs.python.org  Fri Jan 27 10:00:42 2017
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 27 Jan 2017 15:00:42 +0000
Subject: [issue29100] datetime.fromtimestamp() doesn't check min/max year
 anymore: regression of Python 3.6
In-Reply-To: <1482972647.78.0.524140289263.issue29100@psf.upfronthosting.co.za>
Message-ID: <1485529242.73.0.80124765106.issue29100@psf.upfronthosting.co.za>


STINNER Victor added the comment:

timestamp_limits.patch is waiting for your review! This issue is a major regression of Python 3.6. I'm going to push the patch in one week if I don't get any review.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29100>
_______________________________________

From report at bugs.python.org  Fri Jan 27 10:04:28 2017
From: report at bugs.python.org (Alexey Izbyshev)
Date: Fri, 27 Jan 2017 15:04:28 +0000
Subject: [issue29326] Blank lines in ._pth file are not ignored
In-Reply-To: <1484844872.11.0.819349056222.issue29326@psf.upfronthosting.co.za>
Message-ID: <1485529468.61.0.861314913103.issue29326@psf.upfronthosting.co.za>


Alexey Izbyshev added the comment:

I've encountered this issue too. (FYI, the official 3.6.0 embeddable zip from https://www.python.org/downloads/windows/ does contain a blank line in its ._pth, so all its users get an invalid entry in sys.path).

The patch is attached. I couldn't fix tests in more straightforward way because both of them were broken:
* Both of them generated ._pth with 'import site'
* Both of them checked for zero exit code instead of non-zero one, but both passed because predicates in sys.exit() were incorrect

I've strengthened the check in 'nosite' test because it is trivial to fully emulate sys.path calculation logic in this case. I've preserved weaker checks in the other test because I didn't want to emulate site.py logic.

----------
nosy: +izbyshev
Added file: http://bugs.python.org/file46433/underpth_blank_lines.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29326>
_______________________________________

From report at bugs.python.org  Fri Jan 27 10:05:15 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Fri, 27 Jan 2017 15:05:15 +0000
Subject: [issue29300] Modify the _struct module to use FASTCALL and Argument
 Clinic
In-Reply-To: <1484670705.59.0.0193443230555.issue29300@psf.upfronthosting.co.za>
Message-ID: <1485529515.63.0.790183272709.issue29300@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

I said not about the converter that converts format to bytes (it is needed only in Struct.__init__), but about the converter that converts format to the Struct object. It is used in all module-level functions.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29300>
_______________________________________

From report at bugs.python.org  Fri Jan 27 10:07:21 2017
From: report at bugs.python.org (Zachary Ware)
Date: Fri, 27 Jan 2017 15:07:21 +0000
Subject: [issue29380] A lot of warning while executing make install
In-Reply-To: <1485524855.82.0.489793661635.issue29380@psf.upfronthosting.co.za>
Message-ID: <1485529641.23.0.343185760547.issue29380@psf.upfronthosting.co.za>


Zachary Ware added the comment:

Building _ctypes with the bundled copy of libffi is deprecated since Python 3.6, and is not possible in 3.7. If you need the ctypes module, please install libffi separately and make sure the Python build process can find it. If you build libffi yourself, please report any warnings at github.com/libffi/libffi.

----------
nosy: +zach.ware
resolution:  -> third party
stage:  -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29380>
_______________________________________

From report at bugs.python.org  Fri Jan 27 10:20:33 2017
From: report at bugs.python.org (Kaeptm Blaubaer)
Date: Fri, 27 Jan 2017 15:20:33 +0000
Subject: [issue29378] Invalid example in documentation for PyErr_Fetch
In-Reply-To: <1485447904.37.0.693014852804.issue29378@psf.upfronthosting.co.za>
Message-ID: <1485530433.89.0.632395548495.issue29378@psf.upfronthosting.co.za>


Kaeptm Blaubaer added the comment:

I had an outdated version of the documentation.

----------
resolution:  -> out of date
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29378>
_______________________________________

From report at bugs.python.org  Fri Jan 27 10:22:21 2017
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 27 Jan 2017 15:22:21 +0000
Subject: [issue29300] Modify the _struct module to use FASTCALL and Argument
 Clinic
In-Reply-To: <1484670705.59.0.0193443230555.issue29300@psf.upfronthosting.co.za>
Message-ID: <1485530541.92.0.911652248603.issue29300@psf.upfronthosting.co.za>


STINNER Victor added the comment:

New patch version 5 without the "format_converter".


Serhiy Storchaka: I said not about the converter that converts format to bytes (it is needed only in Struct.__init__),

For me, it makes sense to check format type earlier than Struct.__init__(), but since you like the current code, I'm also ok to leave this part of the code unchanged :-)


Serhiy: but about the converter that converts format to the Struct object. It is used in all module-level functions.

Sorry, but I dislike this change. Even if multiple functions do the same, for me creating a Struct object is not something part of the argument parsing. I prefer to keep the following 3 lines in each C function *body* to ease debugging:

    PyObject *s_object = cache_struct(format);
    if (s_object == NULL)
        return NULL;

IMHO my struct_fastcall-5.patch is already big enough. I prefer incremental changes.

I have no strong opinion on cache_struct(), just a matter of taste :-) And others may like your change!

Feel free to write your own patch on top of mine, once mine is merged ;-)

----------
Added file: http://bugs.python.org/file46434/struct_fastcall-5.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29300>
_______________________________________

From report at bugs.python.org  Fri Jan 27 10:23:48 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Fri, 27 Jan 2017 15:23:48 +0000
Subject: [issue29301] decimal: Use FASTCALL and/or Argument Clinic
In-Reply-To: <1484671124.21.0.493200110844.issue29301@psf.upfronthosting.co.za>
Message-ID: <1485530628.31.0.158618789922.issue29301@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

I would dissuade from direct using of FASTCALL, but Argument Clinic is enough reliable. I would recommend to use Argument Clinic unless Stefan supports external fork of the decimal module and is limited by public API.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29301>
_______________________________________

From report at bugs.python.org  Fri Jan 27 10:31:32 2017
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 27 Jan 2017 15:31:32 +0000
Subject: [issue29301] decimal: Use FASTCALL and/or Argument Clinic
In-Reply-To: <1484671124.21.0.493200110844.issue29301@psf.upfronthosting.co.za>
Message-ID: <1485531092.34.0.495019479797.issue29301@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Oops, I forgot to attach decimal-2.patch, here you have.


> I would dissuade from direct using of FASTCALL, but Argument Clinic is enough reliable.

I wrote decimal-2.patch in a few minutes, when I was trying to find all functions still calling _PyStack_AsTuple() in my tp_fast{new,init,call} fork. I just wrote it to identify which parts of CPython can be optimized to make bm_telco faster.

I agree that Argument Clinic should be preferred over using directly METH_FASTCALL, especially for the _decimal module: Stefan already wrote that he wants to have the same code base on Python 3.5, 3.6 and 3.7.


> I would recommend to use Argument Clinic unless Stefan supports external fork of the decimal module and is limited by public API.

What is the status of Argument Clinic? Is it something "public" or not?

The status of _decimal is also unclear to me. Is it part of CPython or not? :-) I know that there is also a "backported" module for Python 2:

   https://pypi.python.org/pypi/cdecimal
   http://www.bytereef.org/mpdecimal/index.html

IMHO it's a great tool. Maybe it's time to make it usable outside CPython in Python 3.7? Or maybe we should wait until the remaining missing features are implemented? Issues #20291 and #29299 for example.

I'm now waiting for Stefan's feedback ;-)

----------
Added file: http://bugs.python.org/file46435/decimal-2.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29301>
_______________________________________

From report at bugs.python.org  Fri Jan 27 10:42:11 2017
From: report at bugs.python.org (Kaeptm Blaubaer)
Date: Fri, 27 Jan 2017 15:42:11 +0000
Subject: [issue29378] Invalid example in documentation for PyErr_Fetch
In-Reply-To: <1485447904.37.0.693014852804.issue29378@psf.upfronthosting.co.za>
Message-ID: <1485531731.48.0.307970846646.issue29378@psf.upfronthosting.co.za>


Kaeptm Blaubaer added the comment:

I had no outdated documentation, but the documentation of 3.5.

----------
resolution: out of date -> 
status: closed -> open
versions: +Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29378>
_______________________________________

From report at bugs.python.org  Fri Jan 27 10:43:04 2017
From: report at bugs.python.org (Stefan Krah)
Date: Fri, 27 Jan 2017 15:43:04 +0000
Subject: [issue29301] decimal: Use FASTCALL and/or Argument Clinic
In-Reply-To: <1484671124.21.0.493200110844.issue29301@psf.upfronthosting.co.za>
Message-ID: <1485531784.88.0.107042617978.issue29301@psf.upfronthosting.co.za>


Stefan Krah added the comment:

I'll take a look when I have the opportunity.

AC will not happen: It makes the module too large and unreadable.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29301>
_______________________________________

From report at bugs.python.org  Fri Jan 27 10:45:46 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Fri, 27 Jan 2017 15:45:46 +0000
Subject: [issue29301] decimal: Use FASTCALL and/or Argument Clinic
In-Reply-To: <1485531092.34.0.495019479797.issue29301@psf.upfronthosting.co.za>
Message-ID: <3225117.Hh1fgsxloD@raxxla>


Serhiy Storchaka added the comment:

> What is the status of Argument Clinic? Is it something "public" or not?

No, it is for for internal CPython use only. It lacks some features (support 
of var-positional and var-keyword parameters, optional parameters without 
default value), the syntax of positional-only parameters is not officially 
accepted still, and future optimizations can require incompatible changes.

Only when all CPython builtins and extensions be converted to Argument Clinic,
PEP 457 (or an alternative) be accepted, Argument Clinic issues be resolved, 
we could say it stable enough. For now even Argument Clinic tests are broken.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29301>
_______________________________________

From report at bugs.python.org  Fri Jan 27 10:46:27 2017
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 27 Jan 2017 15:46:27 +0000
Subject: [issue29300] Modify the _struct module to use FASTCALL and Argument
 Clinic
In-Reply-To: <1484670705.59.0.0193443230555.issue29300@psf.upfronthosting.co.za>
Message-ID: <1485531987.92.0.834730451814.issue29300@psf.upfronthosting.co.za>


STINNER Victor added the comment:

Updated patch, version 6:

* Rename "obj" variable to "iter"
* Fix unpack_from() signature: only the first parameter is positional only

----------
Added file: http://bugs.python.org/file46436/struct_fastcall-6.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29300>
_______________________________________

From report at bugs.python.org  Fri Jan 27 10:52:55 2017
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 27 Jan 2017 15:52:55 +0000
Subject: [issue29301] decimal: Use FASTCALL and/or Argument Clinic
In-Reply-To: <1485531784.88.0.107042617978.issue29301@psf.upfronthosting.co.za>
Message-ID: <CAMpsgwbR1bTRzxfpGySG6ckXACaU+=nCMUAFOwODSwLH7i17iQ@mail.gmail.com>


STINNER Victor added the comment:

Stefan Krah:
> AC will not happen: It makes the module too large and unreadable.

Ah you dislike the additional [clinic input] sections?

It's kind of strange when you have to convert existing code, when once
the code uses AC, I prefer AC to separated documentation variables. It
helps to keep docstrings more up to date, and it helps to enhance the
API (ex: allow keywords, rename parameters to better names, etc.). It
also helps to make the documentation closer to the code, which is IMHO
a good thing :-)

IMHO the PyArg_ParseXXX() calls and their "kwlist" static variable are
"unreadable", and I'm happy to be able to hide them!

FYI decimal-2.patch replaces PyArg_ParseTupleAndKeywords() with
_PyArg_ParseStackAndKeywords() with static _PyArg_Parser object. This
object only decides keyword names once and is more efficient to parse
arguments. It explains partially the speedup. Only partially because
bm_telco only calls the .quantize() method, and it only uses
positional arguments (no keyword arguments) ;-)

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29301>
_______________________________________

From report at bugs.python.org  Fri Jan 27 11:01:24 2017
From: report at bugs.python.org (Stefan Krah)
Date: Fri, 27 Jan 2017 16:01:24 +0000
Subject: [issue29301] decimal: Use FASTCALL and/or Argument Clinic
In-Reply-To: <CAMpsgwbR1bTRzxfpGySG6ckXACaU+=nCMUAFOwODSwLH7i17iQ@mail.gmail.com>
Message-ID: <20170127160107.GA4128@bytereef.org>


Stefan Krah added the comment:

> STINNER Victor added the comment:
> > AC will not happen: It makes the module too large and unreadable.
> 
> Ah you dislike the additional [clinic input] sections?

Yes, they tear apart the code.  I stopped reading many C files because
of this.  Brett asked why several people don't review, this is actually
*one* of the reasons for me.

> It's kind of strange when you have to convert existing code, when once
> the code uses AC, I prefer AC to separated documentation variables. It
> helps to keep docstrings more up to date, and it helps to enhance the
> API (ex: allow keywords, rename parameters to better names, etc.). It
> also helps to make the documentation closer to the code, which is IMHO
> a good thing :-)

Apparently it works for several people, but not me.

> IMHO the PyArg_ParseXXX() calls and their "kwlist" static variable are
> "unreadable", and I'm happy to be able to hide them!
> 
> FYI decimal-2.patch replaces PyArg_ParseTupleAndKeywords() with
> _PyArg_ParseStackAndKeywords() with static _PyArg_Parser object. This
> object only decides keyword names once and is more efficient to parse
> arguments. It explains partially the speedup. Only partially because
> bm_telco only calls the .quantize() method, and it only uses
> positional arguments (no keyword arguments) ;-)

Okay, thanks!

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29301>
_______________________________________

From report at bugs.python.org  Fri Jan 27 11:19:32 2017
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 27 Jan 2017 16:19:32 +0000
Subject: [issue29100] datetime.fromtimestamp() doesn't check min/max year
 anymore: regression of Python 3.6
In-Reply-To: <1482972647.78.0.524140289263.issue29100@psf.upfronthosting.co.za>
Message-ID: <1485533972.66.0.370168054654.issue29100@psf.upfronthosting.co.za>


STINNER Victor added the comment:

timestamp_limits-2.patch: catch also OverflowError in unit tests, and uses a context manager rather than self.assertRaises(..., func, ...) syntax.

----------
Added file: http://bugs.python.org/file46437/timestamp_limits-2.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29100>
_______________________________________

From report at bugs.python.org  Fri Jan 27 11:54:56 2017
From: report at bugs.python.org (Attila-Mihaly Balazs)
Date: Fri, 27 Jan 2017 16:54:56 +0000
Subject: [issue23163] pdb docs need to contain a statement on
 threads/multithreaded debugging
In-Reply-To: <1420366883.83.0.11703664982.issue23163@psf.upfronthosting.co.za>
Message-ID: <1485536096.09.0.380358446488.issue23163@psf.upfronthosting.co.za>


Attila-Mihaly Balazs added the comment:

Absolutely this. While it is good that there is at least some documentation (at sys.settrace), it is not very explicit nor is it self evident that one should look there (for example the PDB docs don't even mention settrace).

----------
nosy: +Attila-Mihaly Balazs

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23163>
_______________________________________

From report at bugs.python.org  Fri Jan 27 12:20:02 2017
From: report at bugs.python.org (Jim Fasarakis-Hilliard)
Date: Fri, 27 Jan 2017 17:20:02 +0000
Subject: [issue29381] Tutorial documentation contains undefined reference
Message-ID: <1485537602.74.0.811037724367.issue29381@psf.upfronthosting.co.za>


New submission from Jim Fasarakis-Hilliard:

After moving a certain chunk of the 'interpreter.rst' contents to 'appendix.rst' in issue16827 the reference to #! in the section '2.2.3. Source Code Encoding' is currently confusing for new readers.

Attached patches reword the sentence to remove that reference for 3.4+ and 2.7.

----------
assignee: docs at python
components: Documentation
files: interpreter_tut.patch
keywords: patch
messages: 286378
nosy: Jim Fasarakis-Hilliard, docs at python
priority: normal
severity: normal
status: open
title: Tutorial documentation contains undefined reference
versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7
Added file: http://bugs.python.org/file46438/interpreter_tut.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29381>
_______________________________________

From report at bugs.python.org  Fri Jan 27 12:20:26 2017
From: report at bugs.python.org (Jim Fasarakis-Hilliard)
Date: Fri, 27 Jan 2017 17:20:26 +0000
Subject: [issue29381] Tutorial documentation contains undefined reference to #!
In-Reply-To: <1485537602.74.0.811037724367.issue29381@psf.upfronthosting.co.za>
Message-ID: <1485537626.58.0.129570872063.issue29381@psf.upfronthosting.co.za>


Changes by Jim Fasarakis-Hilliard <d.f.hilliard at gmail.com>:


----------
title: Tutorial documentation contains undefined reference -> Tutorial documentation contains undefined reference to #!
Added file: http://bugs.python.org/file46439/interpreter_tut2.7.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29381>
_______________________________________

From report at bugs.python.org  Fri Jan 27 12:34:15 2017
From: report at bugs.python.org (Jim Fasarakis-Hilliard)
Date: Fri, 27 Jan 2017 17:34:15 +0000
Subject: [issue29371] Typo in doctest documentation
In-Reply-To: <1485351369.68.0.185486175403.issue29371@psf.upfronthosting.co.za>
Message-ID: <1485538455.75.0.934212559549.issue29371@psf.upfronthosting.co.za>


Jim Fasarakis-Hilliard added the comment:

I think it is fine as it is but, agree that it could be clearer. I'm simply not experienced enough to know if this change is warranted. 

The docs generally do a great job in being concise (balancing brevity and completeness). A core-dev would be best to judge if brevity should be sacrificed for more complete and clear wording here. 

I suggest waiting to see if any core-dev replies and, if no replies come in after a while, close it.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29371>
_______________________________________

From report at bugs.python.org  Fri Jan 27 13:07:04 2017
From: report at bugs.python.org (Alexander Mohr)
Date: Fri, 27 Jan 2017 18:07:04 +0000
Subject: [issue29302] add contextlib.AsyncExitStack
In-Reply-To: <1484693382.32.0.211448130443.issue29302@psf.upfronthosting.co.za>
Message-ID: <1485540424.62.0.4372507841.issue29302@psf.upfronthosting.co.za>


Changes by Alexander Mohr <thehesiod at gmail.com>:


Removed file: http://bugs.python.org/file46322/exit_stack.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29302>
_______________________________________

From report at bugs.python.org  Fri Jan 27 13:07:53 2017
From: report at bugs.python.org (Alexander Mohr)
Date: Fri, 27 Jan 2017 18:07:53 +0000
Subject: [issue29302] add contextlib.AsyncExitStack
In-Reply-To: <1484693382.32.0.211448130443.issue29302@psf.upfronthosting.co.za>
Message-ID: <1485540473.99.0.396844258576.issue29302@psf.upfronthosting.co.za>


Alexander Mohr added the comment:

created gist: https://gist.github.com/thehesiod/b8442ed50e27a23524435a22f10c04a0

I've now updated the imple to support both __aenter__/__aexit__ and __enter__/__exit__ so I don't need two ExitStacks

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29302>
_______________________________________

From report at bugs.python.org  Fri Jan 27 14:05:43 2017
From: report at bugs.python.org (Xavier de Gaye)
Date: Fri, 27 Jan 2017 19:05:43 +0000
Subject: [issue29264] sparc/ffi.c:440 error: 'asm' undeclared
In-Reply-To: <1484328459.12.0.655564355741.issue29264@psf.upfronthosting.co.za>
Message-ID: <1485543943.33.0.621179313323.issue29264@psf.upfronthosting.co.za>


Xavier de Gaye added the comment:

See also msg286365.

----------
nosy: +xdegaye
resolution:  -> third party
stage:  -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29264>
_______________________________________

From report at bugs.python.org  Fri Jan 27 14:17:26 2017
From: report at bugs.python.org (Terry J. Reedy)
Date: Fri, 27 Jan 2017 19:17:26 +0000
Subject: [issue29366] os.listdir has inconsistent behavior when run on a
 non-directory
In-Reply-To: <1485312192.81.0.934042138535.issue29366@psf.upfronthosting.co.za>
Message-ID: <1485544646.82.0.869058864889.issue29366@psf.upfronthosting.co.za>


Terry J. Reedy added the comment:

This is a feature-change 'enhancement' issue.  There is no violation of the doc at https://docs.python.org/3/library/os.html#os.listdir and as Raylu noted, the current behavior is accommodated in tests.

I presume that regardless of what system calls are made, FileNotFoundError and anything else could be caught and NotADirectoryError raised instead, with the message containing the Windows error message.  This would only break Windows-specific code that only catches FileNotFoundError and not NotADirectoryError.

I am not sure what our general policy is and whether this should be fixed at this point.  I personally like Python smoothing over OS differences when easily possible.

----------
nosy: +terry.reedy
stage:  -> test needed
type:  -> enhancement
versions: +Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29366>
_______________________________________

From report at bugs.python.org  Fri Jan 27 14:38:37 2017
From: report at bugs.python.org (Terry J. Reedy)
Date: Fri, 27 Jan 2017 19:38:37 +0000
Subject: [issue29374] Doc: signal.sig(timed)wait do not work outside the main
 thread
In-Reply-To: <1485365131.36.0.15146613889.issue29374@psf.upfronthosting.co.za>
Message-ID: <1485545917.5.0.973030246883.issue29374@psf.upfronthosting.co.za>


Changes by Terry J. Reedy <tjreedy at udel.edu>:


----------
title: Doc bug: signal.sigwait and signal.sigtimedwait do not work outside the Main thread -> Doc: signal.sig(timed)wait do not work outside the main thread

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29374>
_______________________________________

From report at bugs.python.org  Fri Jan 27 15:56:29 2017
From: report at bugs.python.org (Ivan Levkivskyi)
Date: Fri, 27 Jan 2017 20:56:29 +0000
Subject: [issue29377] Add the 'wrapper_descriptor' type to the types module
In-Reply-To: <1485425778.09.0.153066086326.issue29377@psf.upfronthosting.co.za>
Message-ID: <1485550589.84.0.86275258855.issue29377@psf.upfronthosting.co.za>


Ivan Levkivskyi added the comment:

Manuel, thank you for the new patch now everything works. I have few more comments:
1. I have found that there is one more built-in type: type(str.join) gives <class 'method_descriptor'>. Maybe it makes sense to add this one too?
2. Taking into account previous point I withdraw my idea of needing tests here. Sorry for changing opinion twice, but it looks like there are many built-in types that seem to exist just for historical reasons. They might change in future.
3. Please take a look at my review on documentation in Rietveld.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29377>
_______________________________________

From report at bugs.python.org  Fri Jan 27 16:06:01 2017
From: report at bugs.python.org (Gregory P. Smith)
Date: Fri, 27 Jan 2017 21:06:01 +0000
Subject: [issue15373] copy.copy() does not properly copy os.environment
In-Reply-To: <1342454937.95.0.386630033182.issue15373@psf.upfronthosting.co.za>
Message-ID: <1485551161.44.0.272302815072.issue15373@psf.upfronthosting.co.za>


Changes by Gregory P. Smith <greg at krypto.org>:


----------
keywords: +needs review
nosy: +gregory.p.smith
stage:  -> patch review
versions: +Python 3.7 -Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue15373>
_______________________________________

From report at bugs.python.org  Fri Jan 27 16:19:35 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Fri, 27 Jan 2017 21:19:35 +0000
Subject: [issue15373] copy.copy() does not properly copy os.environment
In-Reply-To: <1342454937.95.0.386630033182.issue15373@psf.upfronthosting.co.za>
Message-ID: <1485551975.73.0.0291656404063.issue15373@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

If add support for the copy protocol, consider also adding support for deepcopying and pickling.

----------
nosy: +serhiy.storchaka

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue15373>
_______________________________________

From report at bugs.python.org  Fri Jan 27 16:25:58 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Fri, 27 Jan 2017 21:25:58 +0000
Subject: [issue29371] Typo in doctest documentation
In-Reply-To: <1485351369.68.0.185486175403.issue29371@psf.upfronthosting.co.za>
Message-ID: <1485552358.58.0.400671215955.issue29371@psf.upfronthosting.co.za>


Changes by Raymond Hettinger <raymond.hettinger at gmail.com>:


----------
assignee: docs at python -> Mariatta
nosy: +Mariatta, rhettinger

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29371>
_______________________________________

From report at bugs.python.org  Fri Jan 27 16:26:58 2017
From: report at bugs.python.org (Saurav Singh)
Date: Fri, 27 Jan 2017 21:26:58 +0000
Subject: [issue968063] Add fileinput.islastline()
Message-ID: <1485552418.53.0.00474680112902.issue968063@psf.upfronthosting.co.za>


Saurav Singh added the comment:

last patch works fine i have applied and tested the patch.Its documented well.Can we please have it reviewed?

----------
nosy: +Saurav Singh

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue968063>
_______________________________________

From report at bugs.python.org  Fri Jan 27 16:58:31 2017
From: report at bugs.python.org (=?utf-8?q?Andreas_=C3=85kerlund?=)
Date: Fri, 27 Jan 2017 21:58:31 +0000
Subject: [issue3991] urllib.request.urlopen does not handle non-ASCII
 characters
In-Reply-To: <1222627636.82.0.587488225038.issue3991@psf.upfronthosting.co.za>
Message-ID: <1485554311.52.0.696101516191.issue3991@psf.upfronthosting.co.za>


Andreas ?kerlund added the comment:

Changed the patch after pointers from vadmium.
And quote_uri is changed to quote_iri as martin.panter thought it was more appropriate.

----------
Added file: http://bugs.python.org/file46440/issue3991_2017-01-27.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue3991>
_______________________________________

From report at bugs.python.org  Fri Jan 27 17:23:54 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Fri, 27 Jan 2017 22:23:54 +0000
Subject: [issue29371] Typo in doctest documentation
In-Reply-To: <1485351369.68.0.185486175403.issue29371@psf.upfronthosting.co.za>
Message-ID: <1485555834.77.0.977537279735.issue29371@psf.upfronthosting.co.za>


Raymond Hettinger added the comment:

Mariatta, would you like to look into this and opine about whether the docs are correct as-is.  If you think a change is warranted, propose a patch.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29371>
_______________________________________

From report at bugs.python.org  Fri Jan 27 17:25:16 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Fri, 27 Jan 2017 22:25:16 +0000
Subject: [issue29381] Tutorial documentation contains undefined reference to #!
In-Reply-To: <1485537602.74.0.811037724367.issue29381@psf.upfronthosting.co.za>
Message-ID: <1485555915.99.0.215320073574.issue29381@psf.upfronthosting.co.za>


Raymond Hettinger added the comment:

Mariatta, could you check this patch and tell us your recommendation about whether it should be applied, modified, or rejected?

----------
assignee: docs at python -> Mariatta
nosy: +Mariatta, rhettinger

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29381>
_______________________________________

From report at bugs.python.org  Fri Jan 27 17:27:01 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Fri, 27 Jan 2017 22:27:01 +0000
Subject: [issue1346238] A constant folding optimization pass for the AST
Message-ID: <1485556021.78.0.660401017904.issue1346238@psf.upfronthosting.co.za>


Raymond Hettinger added the comment:

I applaud any effort to move constant folding out of the peepholer and into AST.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue1346238>
_______________________________________

From report at bugs.python.org  Fri Jan 27 17:45:29 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Fri, 27 Jan 2017 22:45:29 +0000
Subject: [issue29237] Create enum for pstats sorting options
In-Reply-To: <1484098294.24.0.117553666266.issue29237@psf.upfronthosting.co.za>
Message-ID: <1485557129.19.0.193845965206.issue29237@psf.upfronthosting.co.za>


Raymond Hettinger added the comment:

Mariatta, would you like to prepare a patch for this.  Ethan and I will review it.

----------
assignee:  -> Mariatta
nosy: +Mariatta, rhettinger

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29237>
_______________________________________

From report at bugs.python.org  Fri Jan 27 19:19:45 2017
From: report at bugs.python.org (Greg Onufer)
Date: Sat, 28 Jan 2017 00:19:45 +0000
Subject: [issue28055] pyhash's siphash24 assumes alignment of the data pointer
In-Reply-To: <1473463742.19.0.416387824738.issue28055@psf.upfronthosting.co.za>
Message-ID: <1485562785.22.0.746842785973.issue28055@psf.upfronthosting.co.za>


Greg Onufer added the comment:

32-bit and 64-bit SPARC ABIs have 64-bit integer data types.

SPARC, like many RISC architectures, also has natural alignment requirements.  Attempting to dereference a pointer to a 4-byte-sized object requires 4-byte alignment, for example.  2-byte-sized objects require 2-byte alignment.  8-byte-sized objects require 8-byte alignment.

siphash24 is encountering this bug on modern SPARC (32-bit ABI currently, haven't tried compiling as 64-bit yet).  The code simply is not portable.

Benjamin's patch gets the failing self-test (test_plistlib) to pass as well as the simple test case in msg275493 above.

----------
nosy: +gco

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28055>
_______________________________________

From report at bugs.python.org  Fri Jan 27 20:02:28 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Sat, 28 Jan 2017 01:02:28 +0000
Subject: [issue15373] copy.copy() does not properly copy os.environment
In-Reply-To: <1342454937.95.0.386630033182.issue15373@psf.upfronthosting.co.za>
Message-ID: <1485565348.28.0.533197196221.issue15373@psf.upfronthosting.co.za>


Raymond Hettinger added the comment:

[Antoine]
> I agree with David that copy(os.environ) is rather ambiguous.
> I think the preferred idiom should be to call dict(os.environ).
> As for __copy__, I don't know what it should do: perhaps
> simply raise an error when copying is attempted?

FWIW, I concur with Antoine on every point.  Also, raising an error when copying seems reasonable ("refuse the temptation to guess" at what the user intended).

----------
nosy: +rhettinger

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue15373>
_______________________________________

From report at bugs.python.org  Fri Jan 27 20:03:15 2017
From: report at bugs.python.org (d s)
Date: Sat, 28 Jan 2017 01:03:15 +0000
Subject: [issue29382] IDLE search/replace/replace+find is (still?) using the
 'hilite' tag instead of 'hit'
Message-ID: <1485565395.44.0.960082340731.issue29382@psf.upfronthosting.co.za>


New submission from d s:

Searching for this issue turned up a bunch of results claiming it's fixed, but it isn't, at least not on my windows 10 machine. I tried replacing 'hit' with 'itdunwork' in colorizer.py, searchbase.py, replace.py, searchengine.py, config.py, search.py, configdialog.py, and of course config-highlight.cfg, but it didn't work.

----------
assignee: terry.reedy
components: IDLE, Windows
messages: 286393
nosy: d s2, paul.moore, steve.dower, terry.reedy, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: IDLE search/replace/replace+find is (still?) using the 'hilite' tag instead of 'hit'
type: behavior
versions: Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29382>
_______________________________________

From report at bugs.python.org  Fri Jan 27 22:04:15 2017
From: report at bugs.python.org (INADA Naoki)
Date: Sat, 28 Jan 2017 03:04:15 +0000
Subject: [issue29383] Reduce temporary unicode object while adding descriptors
Message-ID: <1485572655.18.0.389188121452.issue29383@psf.upfronthosting.co.za>


New submission from INADA Naoki:

add_methods(), add_members(), and add_getset() creates PyUnicode from C string 3 times, and calls PyUnicode_InternInplace 2 times.

1. PyDict_GetItemString() at first. (PyUnicode_FromString() is called).
2. In middle, descr_new() calls PyUnicode_InternFromString().
3. PyDict_SetItemString() at last. (creates unicode and intern it).

Skipping (2) is require adding new private APIs to pass PyUnicodeObject.
But I don't think it worth enough. (I'll try it later.)
So this patch only remove last temporary unicode.
(3 PyUnicode_FromString + 2 PyUnicode_InternInplace) becomes (2 PyUnicode_FromString + 2 PyUnicode_InternInplace).

It seems ~1% startup speedup (without site).

  $ ./python -m performance.benchmarks.bm_python_startup --no-site
  default: python_startup_no_site: Median +- std dev: 12.7 ms +- 0.1 ms
  patched: python_startup_no_site: Median +- std dev: 12.6 ms +- 0.1 ms

While speedup is small, this patch removes time to think "How large this
overhead of GetItemString + SetItemString pair?" while reading code :)

Additionally, this patch removes this comment in PyDict_SetItemString:

  -    PyUnicode_InternInPlace(&kv); /* XXX Should we really? */

SetItemString is used to add something to namespace.
Changing this behavior affects too widely.  So we should do it.

----------
assignee: inada.naoki
components: Interpreter Core
files: descr-remove-getitemstring.patch
keywords: patch
messages: 286394
nosy: inada.naoki
priority: normal
severity: normal
status: open
title: Reduce temporary unicode object while adding descriptors
versions: Python 3.7
Added file: http://bugs.python.org/file46441/descr-remove-getitemstring.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29383>
_______________________________________

From report at bugs.python.org  Fri Jan 27 22:20:46 2017
From: report at bugs.python.org (Terry J. Reedy)
Date: Sat, 28 Jan 2017 03:20:46 +0000
Subject: [issue29382] IDLE search and replace should use 'hit' tag instead of
 'hilite'
In-Reply-To: <1485565395.44.0.960082340731.issue29382@psf.upfronthosting.co.za>
Message-ID: <1485573646.49.0.526808654708.issue29382@psf.upfronthosting.co.za>


Terry J. Reedy added the comment:

The now-closed issues I remember were about hits not being visibly marked at all.  For me, on Win 10 with 3.6, they are being marked.  If you are referring to any discussion specifically about which tag to use, please post the issue numbers prefixed with '#', as in #29382, to make them become links.

Without a patch and more explanaton, I am not sure what you did and what you saw as a result.

I have considered changing the 'hit' tag, but I want to first investigate the behave of other editors (and apps with 'find' functions) on the major platforms.  When I have a proposed patch, I will solicit opinions on whether it is better.

----------
components:  -Windows
nosy:  -paul.moore, steve.dower, tim.golden, zach.ware
title: IDLE search/replace/replace+find is (still?) using the 'hilite' tag instead of 'hit' -> IDLE search and replace should use 'hit' tag instead of 'hilite'
type: behavior -> enhancement
versions: +Python 3.7 -Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29382>
_______________________________________

From report at bugs.python.org  Fri Jan 27 22:38:53 2017
From: report at bugs.python.org (Terry J. Reedy)
Date: Sat, 28 Jan 2017 03:38:53 +0000
Subject: [issue18590] Found text not always highlighted by Replace dialog on
 Windows
In-Reply-To: <1375128565.7.0.113615028691.issue18590@psf.upfronthosting.co.za>
Message-ID: <1485574733.2.0.402944801567.issue18590@psf.upfronthosting.co.za>


Terry J. Reedy added the comment:

Since the original bug for this issue has been fixed, I am closing it in favor of #29392

----------
resolution:  -> fixed
stage: test needed -> resolved
status: open -> closed
superseder:  -> IDLE search and replace should use 'hit' tag instead of 'hilite'

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue18590>
_______________________________________

From report at bugs.python.org  Fri Jan 27 22:45:25 2017
From: report at bugs.python.org (Terry J. Reedy)
Date: Sat, 28 Jan 2017 03:45:25 +0000
Subject: [issue29382] IDLE search and replace should use 'hit' tag instead of
 'hilite'
In-Reply-To: <1485565395.44.0.960082340731.issue29382@psf.upfronthosting.co.za>
Message-ID: <1485575125.73.0.317547776446.issue29382@psf.upfronthosting.co.za>


Terry J. Reedy added the comment:

After the original bug for #18590 was fixed, I left it open for working on the tags.  (Currently, Find and the find part of Replace work differently and at least one of them double tags found text.)  I decided to close it and use this for the tag choice issue.

My replace_find_hit.diff patch on #18590 should be considered transferred to this one.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29382>
_______________________________________

From report at bugs.python.org  Fri Jan 27 23:02:40 2017
From: report at bugs.python.org (Jelle Zijlstra)
Date: Sat, 28 Jan 2017 04:02:40 +0000
Subject: [issue29310] Document typing.NamedTuple default argument syntax
In-Reply-To: <1484755757.54.0.970081096642.issue29310@psf.upfronthosting.co.za>
Message-ID: <1485576160.05.0.51899968105.issue29310@psf.upfronthosting.co.za>


Jelle Zijlstra added the comment:

Thanks! Uploaded a revised patch.

----------
Added file: http://bugs.python.org/file46442/issue29310-2.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29310>
_______________________________________

From report at bugs.python.org  Fri Jan 27 23:07:40 2017
From: report at bugs.python.org (INADA Naoki)
Date: Sat, 28 Jan 2017 04:07:40 +0000
Subject: [issue29383] Reduce temporary unicode object while adding descriptors
In-Reply-To: <1485572655.18.0.389188121452.issue29383@psf.upfronthosting.co.za>
Message-ID: <CAEfz+TxGGd15V+vpzbATE5gD20mtrCBN2G5zyYbD5NRrG3K_ag@mail.gmail.com>


INADA Naoki added the comment:

I think I found better way.

Interned string can be get from descripter. Interning can be reduced
without adding private API.
Please don't review the first patch.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29383>
_______________________________________

From report at bugs.python.org  Sat Jan 28 00:13:03 2017
From: report at bugs.python.org (INADA Naoki)
Date: Sat, 28 Jan 2017 05:13:03 +0000
Subject: [issue29383] Reduce temporary unicode object while adding descriptors
In-Reply-To: <1485572655.18.0.389188121452.issue29383@psf.upfronthosting.co.za>
Message-ID: <1485580383.6.0.583897579564.issue29383@psf.upfronthosting.co.za>


INADA Naoki added the comment:

descr-remove-getitemstring-2.patch is more compact than first patch.
(3 unicode + 2 intern) becomes (2 unicode + 1 intern).

python_startup_no_site: Median +- std dev: 12.5 ms +- 0.1 ms

----------
Added file: http://bugs.python.org/file46443/descr-remove-getitemstring-2.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29383>
_______________________________________

From report at bugs.python.org  Sat Jan 28 00:21:34 2017
From: report at bugs.python.org (INADA Naoki)
Date: Sat, 28 Jan 2017 05:21:34 +0000
Subject: [issue29383] Reduce temporary unicode object while adding descriptors
In-Reply-To: <1485572655.18.0.389188121452.issue29383@psf.upfronthosting.co.za>
Message-ID: <1485580894.37.0.423546850499.issue29383@psf.upfronthosting.co.za>


Changes by INADA Naoki <songofacandy at gmail.com>:


Added file: http://bugs.python.org/file46444/dict-setitemstring.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29383>
_______________________________________

From report at bugs.python.org  Sat Jan 28 01:23:30 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sat, 28 Jan 2017 06:23:30 +0000
Subject: [issue29383] Reduce temporary unicode object while adding descriptors
In-Reply-To: <1485572655.18.0.389188121452.issue29383@psf.upfronthosting.co.za>
Message-ID: <1485584610.14.0.980108703824.issue29383@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

descr-remove-getitemstring-2.patch LGTM.

----------
nosy: +serhiy.storchaka

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29383>
_______________________________________

From report at bugs.python.org  Sat Jan 28 01:30:21 2017
From: report at bugs.python.org (Decorater)
Date: Sat, 28 Jan 2017 06:30:21 +0000
Subject: [issue21085] Cygwin does not provide siginfo_t.si_band
In-Reply-To: <1396019829.85.0.386935527155.issue21085@psf.upfronthosting.co.za>
Message-ID: <1485585021.37.0.916800951469.issue21085@psf.upfronthosting.co.za>


Decorater added the comment:

In 3.6 and 3.7 The Same Error is back on line 960 with gcc 6.3.0 on cygwin64. This issue is not entirely fixed even with the absolute latest commits.

$ make
gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-pro
totypes    -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missin
g-field-initializers   -I. -I./Include    -DPy_BUILD_CORE  -c ./Modules/signalmo
dule.c -o Modules/signalmodule.o
In file included from ./Include/Python.h:85:0,
                 from ./Modules/signalmodule.c:6:
./Modules/signalmodule.c: In function ?fill_siginfo?:
./Modules/signalmodule.c:960:60: error: ?siginfo_t {aka struct <anonymous>}? has
 no member named ?si_band?; did you mean ?si_pid??
     PyStructSequence_SET_ITEM(result, 6, PyLong_FromLong(si->si_band));
                                                            ^
./Include/tupleobject.h:62:75: note: in definition of macro ?PyTuple_SET_ITEM?
 #define PyTuple_SET_ITEM(op, i, v) (((PyTupleObject *)(op))->ob_item[i] = v)
                                                                           ^
./Modules/signalmodule.c:960:5: note: in expansion of macro ?PyStructSequence_SE
T_ITEM?
     PyStructSequence_SET_ITEM(result, 6, PyLong_FromLong(si->si_band));
     ^~~~~~~~~~~~~~~~~~~~~~~~~
make: *** [Makefile:1741: Modules/signalmodule.o] Error 1

----------
nosy: +Decorater

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue21085>
_______________________________________

From report at bugs.python.org  Sat Jan 28 02:06:58 2017
From: report at bugs.python.org (Senthil Kumaran)
Date: Sat, 28 Jan 2017 07:06:58 +0000
Subject: [issue29182] Remove the warning in urllib docs that it doesn't do
 certificate validate by default.
In-Reply-To: <1483722117.81.0.319313681571.issue29182@psf.upfronthosting.co.za>
Message-ID: <1485587218.3.0.439919638851.issue29182@psf.upfronthosting.co.za>


Senthil Kumaran added the comment:

Thank you for the review, Julian.
Addressed it and the change is committed at changeset b5125e971fdd

----------
resolution:  -> fixed
stage: commit review -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29182>
_______________________________________

From report at bugs.python.org  Sat Jan 28 02:35:54 2017
From: report at bugs.python.org (Roundup Robot)
Date: Sat, 28 Jan 2017 07:35:54 +0000
Subject: [issue29383] Reduce temporary unicode object while adding descriptors
In-Reply-To: <1485572655.18.0.389188121452.issue29383@psf.upfronthosting.co.za>
Message-ID: <20170128073551.127283.65928.1DD34F29@psf.io>


Roundup Robot added the comment:

New changeset d7ec72c1620c by INADA Naoki in branch 'default':
Issue #29383: reduce temporary interned unicode
https://hg.python.org/cpython/rev/d7ec72c1620c

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29383>
_______________________________________

From report at bugs.python.org  Sat Jan 28 02:39:33 2017
From: report at bugs.python.org (INADA Naoki)
Date: Sat, 28 Jan 2017 07:39:33 +0000
Subject: [issue29383] Reduce temporary unicode object while adding descriptors
In-Reply-To: <1485572655.18.0.389188121452.issue29383@psf.upfronthosting.co.za>
Message-ID: <1485589173.96.0.134367011653.issue29383@psf.upfronthosting.co.za>


Changes by INADA Naoki <songofacandy at gmail.com>:


----------
resolution:  -> fixed
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29383>
_______________________________________

From report at bugs.python.org  Sat Jan 28 03:46:16 2017
From: report at bugs.python.org (Eryk Sun)
Date: Sat, 28 Jan 2017 08:46:16 +0000
Subject: [issue29366] os.listdir has inconsistent behavior when run on a
 non-directory
In-Reply-To: <1485312192.81.0.934042138535.issue29366@psf.upfronthosting.co.za>
Message-ID: <1485593176.21.0.594884897354.issue29366@psf.upfronthosting.co.za>


Eryk Sun added the comment:

Python has no specific use for ERROR_PATH_NOT_FOUND (3) and just maps it to FileNotFoundError -- like the CRT maps it to ENOENT. Even if we wanted listdir() to specially handle this error, given that it's implemented via FindFirstFile, there's nothing that can be done after the fact that's not vulnerable to race conditions. 

Also, to reiterate this point, getting ERROR_PATH_NOT_FOUND instead of ERROR_DIRECTORY (267) when the target is a file seems to be pretty uncommon, much more so than I at first thought. I only see it with VirtualBox shared folders. I don't see it with Microsoft file systems such as NTFS, FAT32, or CDFS.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29366>
_______________________________________

From report at bugs.python.org  Sat Jan 28 04:20:46 2017
From: report at bugs.python.org (Ivan Levkivskyi)
Date: Sat, 28 Jan 2017 09:20:46 +0000
Subject: [issue29310] Document typing.NamedTuple default argument syntax
In-Reply-To: <1484755757.54.0.970081096642.issue29310@psf.upfronthosting.co.za>
Message-ID: <1485595246.12.0.380301802525.issue29310@psf.upfronthosting.co.za>


Ivan Levkivskyi added the comment:

I have found one typo (see Rietveld), otherwise LGTM.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29310>
_______________________________________

From report at bugs.python.org  Sat Jan 28 06:16:54 2017
From: report at bugs.python.org (Armin Rigo)
Date: Sat, 28 Jan 2017 11:16:54 +0000
Subject: [issue10544] yield expression inside generator expression does nothing
In-Reply-To: <1290799279.15.0.89740732651.issue10544@psf.upfronthosting.co.za>
Message-ID: <1485602214.48.0.510848976723.issue10544@psf.upfronthosting.co.za>


Armin Rigo added the comment:

Just to add my comment to this 7-years-old never-resolved issue: in PyPy 3.5, which behaves like Python 3.x in this respect, I made the following constructions give a warning.

    def wrong_listcomp():
        return [(yield 42) for i in j]
    def wrong_gencomp():
        return ((yield 42) for i in j)
    def wrong_dictcomp():
        return {(yield 42):2 for i in j}
    def wrong_setcomp():
        return {(yield 42) for i in j}

SyntaxWarning: 'yield' inside a list or generator comprehension behaves unexpectedly (http://bugs.python.org/issue10544)

The motivation is that none of the constructions above gives the "expected" result.  In more details:

- wrong_listcomp() doesn't even return a list at all.  It's possible to have a clue about why this occurs, but I would say that it is just plain wrong given the ``return [...]`` part of the syntax.  The same is true for wrong_dictcomp() and wrong_setcomp().

- wrong_gencomp() returns a generator as expected.  However, it is a generator that yields two elements for each i in j: first 42, and then whatever was ``send()`` into the generator.  I would say that it is in contradiction with the general idea that this syntax should give a generator that yields one item for each i in j.  In fact, when the user writes such code he might be expecting the "yield" to apply to the function level instead of the genexpr level---but none of the functions above end up being themselves generators.

For completeness, I think there is no problem with "await" instead of "yield" in Python 3.6.

How about fixing CPython to raise SyntaxWarning or even SyntaxError?

----------
nosy: +arigo

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue10544>
_______________________________________

From report at bugs.python.org  Sat Jan 28 06:28:57 2017
From: report at bugs.python.org (Ivan Levkivskyi)
Date: Sat, 28 Jan 2017 11:28:57 +0000
Subject: [issue10544] yield expression inside generator expression does nothing
In-Reply-To: <1290799279.15.0.89740732651.issue10544@psf.upfronthosting.co.za>
Message-ID: <1485602937.26.0.552045647376.issue10544@psf.upfronthosting.co.za>


Ivan Levkivskyi added the comment:

> How about fixing CPython to raise SyntaxWarning or even SyntaxError?

I think it is better to just fix the issue, i.e. make comprehensions be equivalent to for-loops even if they contain `yield`. (In particular this will lead to [(yield i) for i in range(5)] be SyntaxError outside function).

The example of `await` shows that it is possible without leaking the loop variable into enclosing scope.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue10544>
_______________________________________

From report at bugs.python.org  Sat Jan 28 06:55:04 2017
From: report at bugs.python.org (Armin Rigo)
Date: Sat, 28 Jan 2017 11:55:04 +0000
Subject: [issue10544] yield expression inside generator expression does nothing
In-Reply-To: <1290799279.15.0.89740732651.issue10544@psf.upfronthosting.co.za>
Message-ID: <1485604504.33.0.65807888689.issue10544@psf.upfronthosting.co.za>


Armin Rigo added the comment:

Let's see if the discussion goes anywhere or if this issue remains in limbo for the next 7 years.  In the meantime, if I may humbly make a suggestion: whether the final decision is to give SyntaxError or change the semantics, one or a few intermediate versions with a SyntaxWarning might be a good idea.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue10544>
_______________________________________

From report at bugs.python.org  Sat Jan 28 07:09:12 2017
From: report at bugs.python.org (Jim Fasarakis-Hilliard)
Date: Sat, 28 Jan 2017 12:09:12 +0000
Subject: [issue29381] Tutorial documentation contains undefined reference to #!
In-Reply-To: <1485537602.74.0.811037724367.issue29381@psf.upfronthosting.co.za>
Message-ID: <1485605352.82.0.657347462657.issue29381@psf.upfronthosting.co.za>


Jim Fasarakis-Hilliard added the comment:

Hi, Raymond. Is Mariatta responsible for reviewing Documentation submissions? If so, should I nosy Mariatta in any future documentation issues?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29381>
_______________________________________

From report at bugs.python.org  Sat Jan 28 09:16:22 2017
From: report at bugs.python.org (=?utf-8?b?VmVkcmFuIMSMYcSNacSH?=)
Date: Sat, 28 Jan 2017 14:16:22 +0000
Subject: [issue29302] add contextlib.AsyncExitStack
In-Reply-To: <1484693382.32.0.211448130443.issue29302@psf.upfronthosting.co.za>
Message-ID: <1485612982.87.0.994158940929.issue29302@psf.upfronthosting.co.za>


Vedran ?a?i? added the comment:

An example from "real life": I recently needed this when implementing a demo of dining philosophers using asyncio (when the order of locks should depend on comparison of fork ids, not be static). Of course, it wasn't hard to implement it directly, but still, it would be nice if I had it in stdlib.

Also, yes, it would probably be nicer if there was only one ExitStack with push_async and similar methods.

----------
nosy: +veky

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29302>
_______________________________________

From report at bugs.python.org  Sat Jan 28 10:01:24 2017
From: report at bugs.python.org (Antoine Pitrou)
Date: Sat, 28 Jan 2017 15:01:24 +0000
Subject: [issue28055] pyhash's siphash24 assumes alignment of the data pointer
In-Reply-To: <1473463742.19.0.416387824738.issue28055@psf.upfronthosting.co.za>
Message-ID: <1485615684.68.0.79016578105.issue28055@psf.upfronthosting.co.za>


Antoine Pitrou added the comment:

I agree with Stefan and Serhiy.  Unaligned memoryviews shouldn't trigger a copy when hashing.

----------
nosy: +pitrou

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28055>
_______________________________________

From report at bugs.python.org  Sat Jan 28 11:06:53 2017
From: report at bugs.python.org (Marco Buttu)
Date: Sat, 28 Jan 2017 16:06:53 +0000
Subject: [issue27200] make doctest in CPython has failures
In-Reply-To: <1464988408.0.0.840317371896.issue27200@psf.upfronthosting.co.za>
Message-ID: <1485619613.46.0.665830489125.issue27200@psf.upfronthosting.co.za>


Marco Buttu added the comment:

In order for the patches to work for every Python version (see #msg284622 ), I added a new feature to the Sphinx doctest extension, called pyversion. Starting from Sphinx 1.6, it will be possible to specify the Python version in order for the example to be tested. For instance, if I run the doctests with Python 3.3, then this test will be skipped:

.. doctest::
    :pyversion: > 3.4

    # Code example

It will be tested only when Sphinx is running on Python > 3.4. My question is: what about using this feature in the patches? I am asking this because Sphinx 1.6 will be released in about 7-10 months, so until then, if we want to use the :pyversion: feature we have to build the documentation using the master version of Sphinx. Please, let me know

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue27200>
_______________________________________

From report at bugs.python.org  Sat Jan 28 11:39:15 2017
From: report at bugs.python.org (Marco Buttu)
Date: Sat, 28 Jan 2017 16:39:15 +0000
Subject: [issue29381] Tutorial documentation contains undefined reference to #!
In-Reply-To: <1485537602.74.0.811037724367.issue29381@psf.upfronthosting.co.za>
Message-ID: <1485621555.71.0.804366646736.issue29381@psf.upfronthosting.co.za>


Marco Buttu added the comment:

You wrote: "It is also possible to specify a different encoding for source files. In order to do this, you can use a special comment line that defines the source file encoding::". I think that is not true, because the line have to be the first line, or right below a comment (just one, as in the case of the shebang). For instance, in this case Python apply the declared coding:

  $ cat foo.py
  # The first line is a comment
  # -*- coding: ascii -*-
  print('?')  # Encoded in UTF-8
  $ python foo.py
     ...
  SyntaxError: encoding problem: ascii


In this case it does not:

  $ cat foo.py
  # The first line is a comment
  # and also the sencond line
  # -*- coding: ascii -*-
  print('?')  # Encoded in UTF-8
  $ python foo.py 
  ?

But I think you are right that the current doc is confusing. Maybe yon can write something like this: 

"It is also possible to specify a different encoding for source files. In order to do this, put one special comment line to define the source file encoding:

# -*- coding: encoding -*-

This coding comment has to be the first line of the file, or the second line in case the first one is the #! line."

----------
nosy: +marco.buttu

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29381>
_______________________________________

From report at bugs.python.org  Sat Jan 28 11:55:13 2017
From: report at bugs.python.org (Jim Fasarakis-Hilliard)
Date: Sat, 28 Jan 2017 16:55:13 +0000
Subject: [issue29381] Tutorial documentation contains undefined reference to #!
In-Reply-To: <1485537602.74.0.811037724367.issue29381@psf.upfronthosting.co.za>
Message-ID: <1485622513.32.0.29575820223.issue29381@psf.upfronthosting.co.za>


Jim Fasarakis-Hilliard added the comment:

Indeed, this does create an issue. 

The last sentence in the documentation actually specifies where the encoding comment can be but doesn't strictly specify it can be on the second line if and only if preceded by `#!`.

I'm thinking the last sentence should contain the additional change, that is, change it from :

    The special encoding comment must be in the first or second line within the file.

to:

    The special encoding comment must be in the first line, or, if the first line is used to create an executable Python script, the second line within the file.

linking to (https://docs.python.org/3/tutorial/appendix.html#executable-python-scripts) accordingly. 

Let's see what Mariatta says about these proposals. Thanks for noticing.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29381>
_______________________________________

From report at bugs.python.org  Sat Jan 28 12:04:36 2017
From: report at bugs.python.org (Jelle Zijlstra)
Date: Sat, 28 Jan 2017 17:04:36 +0000
Subject: [issue29310] Document typing.NamedTuple default argument syntax
In-Reply-To: <1484755757.54.0.970081096642.issue29310@psf.upfronthosting.co.za>
Message-ID: <1485623076.72.0.432791999183.issue29310@psf.upfronthosting.co.za>


Jelle Zijlstra added the comment:

Fixed the typo

----------
Added file: http://bugs.python.org/file46445/issue29310-3.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29310>
_______________________________________

From report at bugs.python.org  Sat Jan 28 12:21:17 2017
From: report at bugs.python.org (Ivan Levkivskyi)
Date: Sat, 28 Jan 2017 17:21:17 +0000
Subject: [issue29310] Document typing.NamedTuple default argument syntax
In-Reply-To: <1484755757.54.0.970081096642.issue29310@psf.upfronthosting.co.za>
Message-ID: <1485624076.99.0.621773473375.issue29310@psf.upfronthosting.co.za>


Ivan Levkivskyi added the comment:

Thank you!

Guido, I think this is ready to be merged.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29310>
_______________________________________

From report at bugs.python.org  Sat Jan 28 14:23:15 2017
From: report at bugs.python.org (Shawn)
Date: Sat, 28 Jan 2017 19:23:15 +0000
Subject: [issue23287] ctypes.util.find_library needlessly call crle on Solaris
In-Reply-To: <1421800382.95.0.158394542432.issue23287@psf.upfronthosting.co.za>
Message-ID: <1485631395.94.0.714193309357.issue23287@psf.upfronthosting.co.za>


Shawn added the comment:

Could we get someone to evaluate this please?

----------
nosy: +swalker

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23287>
_______________________________________

From report at bugs.python.org  Sat Jan 28 16:45:11 2017
From: report at bugs.python.org (Ammar Askar)
Date: Sat, 28 Jan 2017 21:45:11 +0000
Subject: [issue29362] regrtest: don't fail immediately if a child does crash
In-Reply-To: <1485272121.72.0.342410390691.issue29362@psf.upfronthosting.co.za>
Message-ID: <1485639911.04.0.335649900855.issue29362@psf.upfronthosting.co.za>


Ammar Askar added the comment:

This patch makes it so that when a child runner crashes during a multiprocess test run, it is treated similar to a keyboard interrupt. That is, the currently running tests will continue to run and immediately after it will print out a summary.

I think the test suite should be interrupted when an interpreter crash happens because this indicates a fairly major failure and interrupting the suite brings a lot of attention to it.

----------
keywords: +patch
nosy: +ammar2
Added file: http://bugs.python.org/file46446/regrtest_crash.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29362>
_______________________________________

From report at bugs.python.org  Sat Jan 28 17:41:17 2017
From: report at bugs.python.org (Martin Panter)
Date: Sat, 28 Jan 2017 22:41:17 +0000
Subject: [issue23287] ctypes.util.find_library needlessly call crle on Solaris
In-Reply-To: <1421800382.95.0.158394542432.issue23287@psf.upfronthosting.co.za>
Message-ID: <1485643277.17.0.259976139332.issue23287@psf.upfronthosting.co.za>


Martin Panter added the comment:

Do you have an opinion of the proposal Shawn?

Judging by John?s ?[the constant] is what the system provides, though
users may customize as needed?, it sounds like the patch is more than a bug fix. Will it also mean find_library() will no longer search customized library directories? I?m not familiar with Solaris, but it seems this may not be intended or desired.

Why is this bug marked as ?resource usage??

----------
nosy: +martin.panter
stage:  -> patch review

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23287>
_______________________________________

From report at bugs.python.org  Sat Jan 28 18:00:08 2017
From: report at bugs.python.org (Martin Panter)
Date: Sat, 28 Jan 2017 23:00:08 +0000
Subject: [issue20664] _findLib_crle and _get_soname broken on latest SunOS 5.11
In-Reply-To: <1392668740.39.0.330236898998.issue20664@psf.upfronthosting.co.za>
Message-ID: <1485644408.33.0.481935141687.issue20664@psf.upfronthosting.co.za>


Martin Panter added the comment:

There is an Open Indiana buildbot <http://buildbot.python.org/all/buildslaves/cea-indiana-x86>, that was passing test_ctypes (until the buildbot went offline a few weeks ago). Would it be possible to include a regression test case?

Also, in <https://bugs.python.org/issue23287#msg247067>, John said that /usr/ccs/bin is a symbolic link to /usr/bin/, so I wonder if this really is a problem. Perhaps the symlink does not exist on some OS versions?

Regarding an environment variable for library searching, perhaps that is related to Issue 9998 (searched LD_LIBRARY_PATH env. var. on Linux). See also Issue 19317, discussing RPATH fom sys.executable.

----------
nosy: +martin.panter
stage:  -> patch review
versions: +Python 3.6, Python 3.7 -Python 3.1, Python 3.2, Python 3.3, Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue20664>
_______________________________________

From report at bugs.python.org  Sat Jan 28 18:14:00 2017
From: report at bugs.python.org (Ivan Levkivskyi)
Date: Sat, 28 Jan 2017 23:14:00 +0000
Subject: [issue29262] Provide a way to check for *real* typing.Union instances
In-Reply-To: <1484322347.31.0.5780880295.issue29262@psf.upfronthosting.co.za>
Message-ID: <1485645240.27.0.59539809313.issue29262@psf.upfronthosting.co.za>


Ivan Levkivskyi added the comment:

Cross-posting the link to upstream work on this: https://github.com/python/typing/pull/377

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29262>
_______________________________________

From report at bugs.python.org  Sat Jan 28 20:06:54 2017
From: report at bugs.python.org (Martin Panter)
Date: Sun, 29 Jan 2017 01:06:54 +0000
Subject: [issue3991] urllib.request.urlopen does not handle non-ASCII
 characters
In-Reply-To: <1222627636.82.0.587488225038.issue3991@psf.upfronthosting.co.za>
Message-ID: <1485652014.02.0.729738315483.issue3991@psf.upfronthosting.co.za>


Martin Panter added the comment:

I?m not really an expert on non-ASCII URLs / IRIs. Maybe it is obvious to other people that this is a good general implementation, but for me to thoroughly review it I would need time to research the relevant RFCs, other implementations, suitability for the URL schemes listed at <https://docs.python.org/dev/library/urllib.parse.html>, security implications, etc.

One problem problem with using urlunsplit() is it would strip empty URL components, e.g. quote_iri("http://example/file#") -> "http://example/file". See Issue 22852. This is highlighted by the file:///[. . .] ? file:/[. . .] test case.

FYI Martin Panter and vadmium are both just me, no need to get too excited. :) I just updated my settings for Rietveld (code review), so hopefully that is more obvious now.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue3991>
_______________________________________

From report at bugs.python.org  Sat Jan 28 20:25:25 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Sun, 29 Jan 2017 01:25:25 +0000
Subject: [issue29381] Tutorial documentation contains undefined reference to #!
In-Reply-To: <1485537602.74.0.811037724367.issue29381@psf.upfronthosting.co.za>
Message-ID: <1485653125.37.0.865688234551.issue29381@psf.upfronthosting.co.za>


Raymond Hettinger added the comment:

> Is Mariatta responsible for reviewing Documentation submissions?

No, she is a new core developer and this is a good issue for her to do the initial review and post any thoughts on the subject.  We've all got to start somewhere.

> If so, should I nosy Mariatta in any future documentation issues?

You did everything just right.  Just marking this as a doc patch is sufficient.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29381>
_______________________________________

From report at bugs.python.org  Sat Jan 28 20:40:25 2017
From: report at bugs.python.org (Guido van Rossum)
Date: Sun, 29 Jan 2017 01:40:25 +0000
Subject: [issue29310] Document typing.NamedTuple default argument syntax
In-Reply-To: <1485624076.99.0.621773473375.issue29310@psf.upfronthosting.co.za>
Message-ID: <CAP7+vJKQE5kF5qPgELmCT65WD2ZTqs4Q4=jVRp9K27QaTx9pYQ@mail.gmail.com>


Guido van Rossum added the comment:

Sorry, I'm still sick. You may have to find another core dev to help you.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29310>
_______________________________________

From report at bugs.python.org  Sat Jan 28 22:53:59 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Sun, 29 Jan 2017 03:53:59 +0000
Subject: [issue29310] Document typing.NamedTuple default argument syntax
In-Reply-To: <1484755757.54.0.970081096642.issue29310@psf.upfronthosting.co.za>
Message-ID: <1485662039.88.0.0666890480392.issue29310@psf.upfronthosting.co.za>


Changes by Raymond Hettinger <raymond.hettinger at gmail.com>:


----------
assignee:  -> rhettinger
nosy: +rhettinger

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29310>
_______________________________________

From report at bugs.python.org  Sat Jan 28 23:06:55 2017
From: report at bugs.python.org (Martin Panter)
Date: Sun, 29 Jan 2017 04:06:55 +0000
Subject: [issue29384] Unused beos build scripts
Message-ID: <1485662815.62.0.566203367253.issue29384@psf.upfronthosting.co.za>


New submission from Martin Panter:

Modules/ld_so_beos and Modules/ar_beos are no longer used since Be OS support was dropped from the configure script for Python 3:

https://hg.python.org/cpython/diff/a5e984eda45a/configure.in

I propose to also remove the scripts.

----------
components: Build
files: beos-scripts.patch
keywords: patch
messages: 286426
nosy: martin.panter
priority: normal
severity: normal
stage: patch review
status: open
title: Unused beos build scripts
type: enhancement
versions: Python 3.7
Added file: http://bugs.python.org/file46447/beos-scripts.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29384>
_______________________________________

From report at bugs.python.org  Sat Jan 28 23:17:16 2017
From: report at bugs.python.org (Roundup Robot)
Date: Sun, 29 Jan 2017 04:17:16 +0000
Subject: [issue29310] Document typing.NamedTuple default argument syntax
In-Reply-To: <1484755757.54.0.970081096642.issue29310@psf.upfronthosting.co.za>
Message-ID: <20170129041713.123199.2439.DB85CE84@psf.io>


Roundup Robot added the comment:

New changeset b33c3f26eefa by Raymond Hettinger in branch '3.6':
Issue 29310: Document typing.NamedTuple default argument syntax
https://hg.python.org/cpython/rev/b33c3f26eefa

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29310>
_______________________________________

From report at bugs.python.org  Sat Jan 28 23:18:59 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Sun, 29 Jan 2017 04:18:59 +0000
Subject: [issue29310] Document typing.NamedTuple default argument syntax
In-Reply-To: <1484755757.54.0.970081096642.issue29310@psf.upfronthosting.co.za>
Message-ID: <1485663539.59.0.55888209591.issue29310@psf.upfronthosting.co.za>


Raymond Hettinger added the comment:

Thanks for the patch.  It applies cleanly.  The docs build just fine.  The markup looks okay.  The example runs correctly.

Applied and marking as closed.

----------
resolution:  -> fixed
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29310>
_______________________________________

From report at bugs.python.org  Sun Jan 29 00:36:00 2017
From: report at bugs.python.org (Dhiraj)
Date: Sun, 29 Jan 2017 05:36:00 +0000
Subject: [issue29385] Sockets Crashes or Memory Corruption
Message-ID: <1485668160.59.0.222592776098.issue29385@psf.upfronthosting.co.za>


New submission from Dhiraj:

When Python Server receives a string of '/xff' (5 or more) from a page  after completing a handshake, the tab immediately crashes.  There are variations of this string with other characters mixed in that also cause an immediate close of python server.

To reproduce:

1. Run the websockets.py server 
2. Then open websockets.html in browser 

The PY server gets Close/Crash

----------
components: Windows
files: REPRO.ZIP
messages: 286429
nosy: Dhiraj_Mishra, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Sockets Crashes or Memory Corruption
versions: Python 2.7, Python 3.3, Python 3.4, Python 3.5, Python 3.6, Python 3.7
Added file: http://bugs.python.org/file46448/REPRO.ZIP

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29385>
_______________________________________

From report at bugs.python.org  Sun Jan 29 00:55:23 2017
From: report at bugs.python.org (Martin Panter)
Date: Sun, 29 Jan 2017 05:55:23 +0000
Subject: [issue23980] Documentation for format units starting with 'e' is
 inconsistent
In-Reply-To: <1429227526.01.0.733275706487.issue23980@psf.upfronthosting.co.za>
Message-ID: <1485669323.78.0.620846597719.issue23980@psf.upfronthosting.co.za>


Martin Panter added the comment:

The O! and O& units are in a similar situation. They just use a different font and descriptive name, rather than a specific type:

``O!`` (object) [*typeobject*, PyObject \*]
``O&`` (object) [*converter*, *anything*]

Following this lead, you could write:

``es`` (:class:`str`) [*encoding*, char \*buffer]
``et`` (. . .) [*encoding*, char \*buffer]
``es#`` (:class:`str`) [*encoding*, char \*buffer, int buffer_length]
``et#`` (. . .) [*encoding*, char \*buffer, int buffer_length]

The text description should explain what *encoding* is, but it appears it may already do that well enough.

----------
nosy: +martin.panter

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23980>
_______________________________________

From report at bugs.python.org  Sun Jan 29 01:28:41 2017
From: report at bugs.python.org (Gabriel McManus)
Date: Sun, 29 Jan 2017 06:28:41 +0000
Subject: [issue29386] select.epoll.poll may behave differently if timeout = -1
 vs timeout = None
Message-ID: <1485671321.44.0.0475790676768.issue29386@psf.upfronthosting.co.za>


New submission from Gabriel McManus:

The select module epoll.poll method takes a "timeout" parameter which is documented as having a default value of -1 [1]. If no timeout (or None) is passed to epoll.poll, then a value of -1 is passed to the epoll_wait system call. But if a timeout of -1 is passed to epoll.poll, then a value of -1000 is passed to epoll_wait. This is because the timeout is converted from seconds to milliseconds.

Before Python 3.5, if a negative timeout was passed to epoll.poll then -1 was passed to epoll_wait [2].

The Linux epoll_wait documentation doesn't specify the behaviour if timeout < -1. Linux itself behaves the same for all negative timeouts: never time out. But on Illumos, timeout < -1 currently times out immediately, and only timeout == -1 never times out [3].

Some code does pass -1 to select.epoll.poll expecting it to never time out. For example, selectors.EpollSelector [4].

I suggest restoring the pre-3.5 behaviour: epoll.poll should use -1 if the given timeout is negative.

I discovered this because ipython3 uses selectors.EpollSelector on Illumos,
and it was using 100% cpu while waiting for input because epoll_wait was returning immediately.

To demonstrate the issue you can run:

    strace python3.5 -c 'import select; select.epoll().poll(timeout=-1)' &

On Illumos this completes immediately, and the output contains the -1000 timeout:

    epoll_wait(3, [], 1023, -1000)          = 0

On Linux, it will block. If you then kill the python process with SIGTERM, strace should print the interrupted epoll_wait call, revealing the -1000 timeout:

    epoll_wait(3, 
    ...
    299a070, 1023, -100)        = -1 EINTR (Interrupted system call)

[1] https://github.com/python/cpython/blob/b9e40ed1bcce127893e40dd355087cda7187ac27/Modules/selectmodule.c#L1489
[2] https://github.com/python/cpython/commit/02e27d1301ea680dce9c3013010e3befedf9628a
[3] https://github.com/joyent/illumos-joyent/issues/136
[4] https://github.com/python/cpython/blob/8228a2b306844a213eddb4fb908c1925840ff67e/Lib/selectors.py#L428

----------
components: Extension Modules
messages: 286431
nosy: Gabriel McManus
priority: normal
severity: normal
status: open
title: select.epoll.poll may behave differently if timeout = -1 vs timeout = None
type: behavior
versions: Python 3.5, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29386>
_______________________________________

From report at bugs.python.org  Sun Jan 29 01:30:31 2017
From: report at bugs.python.org (Steve Dower)
Date: Sun, 29 Jan 2017 06:30:31 +0000
Subject: [issue29326] Blank lines in ._pth file are not ignored
In-Reply-To: <1484844872.11.0.819349056222.issue29326@psf.upfronthosting.co.za>
Message-ID: <1485671431.73.0.327954177696.issue29326@psf.upfronthosting.co.za>


Steve Dower added the comment:

I haven't had a chance to try the patch (underpth_blank_lines.diff) or apply it, but it looks right.

If one of the other core devs gets to apply this before I do, feel free.

Alexey, you don't need to sign a CLA for this fix, but if you're going to contribute more then you may want to: https://docs.python.org/devguide/committing.html#contributor-licensing-agreements It's all online, and it'll help save the core developers from having to evaluate whether you need it or not (which I can do, because it comes up often for me in my day job, but not everyone has that experience). And thanks for this patch!

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29326>
_______________________________________

From report at bugs.python.org  Sun Jan 29 02:56:42 2017
From: report at bugs.python.org (Senthil Kumaran)
Date: Sun, 29 Jan 2017 07:56:42 +0000
Subject: [issue29384] Unused beos build scripts
In-Reply-To: <1485662815.62.0.566203367253.issue29384@psf.upfronthosting.co.za>
Message-ID: <1485676602.55.0.121494335437.issue29384@psf.upfronthosting.co.za>


Senthil Kumaran added the comment:

@martin, the patch looks good to me. Please go ahead and commit this.

----------
nosy: +orsenthil

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29384>
_______________________________________

From report at bugs.python.org  Sun Jan 29 03:23:05 2017
From: report at bugs.python.org (Martin Panter)
Date: Sun, 29 Jan 2017 08:23:05 +0000
Subject: [issue18842] Add float.is_finite is_nan is_infinite to match Decimal
 methods
In-Reply-To: <1377532757.24.0.123652540945.issue18842@psf.upfronthosting.co.za>
Message-ID: <1485678185.45.0.74340746494.issue18842@psf.upfronthosting.co.za>


Martin Panter added the comment:

FWIW, here is an attempt to add Argument Clinic to the Objects/floatobject.c and Objects/longobject.c implementations:

https://bugs.python.org/file33943/issue20185_conglomerate_v4.diff
https://bugs.python.org/file33989/clinic_longobject_v3.patch

If the methods are rejected, the dead code should be removed to avoid wasting people?s time.

Why do you name the methods is_finite() etc with underscores, when the existing methods math.isfinite() etc do not have underscores? Seems it would add unnecessary confusion.

----------
nosy: +martin.panter

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue18842>
_______________________________________

From report at bugs.python.org  Sun Jan 29 04:28:15 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sun, 29 Jan 2017 09:28:15 +0000
Subject: [issue20185] Derby #17: Convert 49 sites to Argument Clinic across 13
 files
In-Reply-To: <1389140539.55.0.106381625615.issue20185@psf.upfronthosting.co.za>
Message-ID: <1485682095.07.0.379093372853.issue20185@psf.upfronthosting.co.za>


Changes by Serhiy Storchaka <storchaka+cpython at gmail.com>:


Added file: http://bugs.python.org/file46449/clinic_longobject_v5.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue20185>
_______________________________________

From report at bugs.python.org  Sun Jan 29 04:38:08 2017
From: report at bugs.python.org (Martin Panter)
Date: Sun, 29 Jan 2017 09:38:08 +0000
Subject: [issue20180] Derby #11: Convert 50 sites to Argument Clinic across 9
 files
In-Reply-To: <1389140162.94.0.578256004005.issue20180@psf.upfronthosting.co.za>
Message-ID: <1485682688.41.0.578667854578.issue20180@psf.upfronthosting.co.za>


Martin Panter added the comment:

For str.format_map(mapping), yes the parsing happens in Objects/stringlib/unicode_format.h, but I don?t see that as a big problem. Moving this back to ?needs patch?, assuming it is okay to convert format_map().

Other than from that, there are just tricky things left like the str() constructor, str.format(*args, **kwargs) (see Issue 20291), and the shared code in Objects/stringlib/find.h and Objects/stringlib/transmogrify.h. I don?t know what to do about those.

----------
nosy: +martin.panter
stage: commit review -> needs patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue20180>
_______________________________________

From report at bugs.python.org  Sun Jan 29 04:44:49 2017
From: report at bugs.python.org (Martin Panter)
Date: Sun, 29 Jan 2017 09:44:49 +0000
Subject: [issue20185] Derby #17: Convert 49 sites to Argument Clinic across 13
 files
In-Reply-To: <1389140539.55.0.106381625615.issue20185@psf.upfronthosting.co.za>
Message-ID: <1485683089.07.0.965973641961.issue20185@psf.upfronthosting.co.za>


Martin Panter added the comment:

longobject_v5 looks good to me

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue20185>
_______________________________________

From report at bugs.python.org  Sun Jan 29 05:34:26 2017
From: report at bugs.python.org (Roundup Robot)
Date: Sun, 29 Jan 2017 10:34:26 +0000
Subject: [issue29384] Unused beos build scripts
In-Reply-To: <1485662815.62.0.566203367253.issue29384@psf.upfronthosting.co.za>
Message-ID: <20170129103423.123333.78763.BB67747C@psf.io>


Roundup Robot added the comment:

New changeset aa9aef656fe0 by Martin Panter in branch 'default':
Issue #29384: Remove Be OS scripts from Modules/, unused in 3.0+
https://hg.python.org/cpython/rev/aa9aef656fe0

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29384>
_______________________________________

From report at bugs.python.org  Sun Jan 29 05:34:26 2017
From: report at bugs.python.org (Roundup Robot)
Date: Sun, 29 Jan 2017 10:34:26 +0000
Subject: [issue29349] Update old Python 2 code in
 Docs/tools/extensions/patchlevel.py
In-Reply-To: <1485164481.25.0.457345760711.issue29349@psf.upfronthosting.co.za>
Message-ID: <20170129103424.123333.6349.24324A77@psf.io>


Roundup Robot added the comment:

New changeset ecdc864884a9 by Martin Panter in branch '3.5':
Issue #29349: Fix Python 2 syntax in documentation build code
https://hg.python.org/cpython/rev/ecdc864884a9

New changeset cdcb33f37bf3 by Martin Panter in branch '3.6':
Issues #29349: Merge Py 2 fix 3.5
https://hg.python.org/cpython/rev/cdcb33f37bf3

New changeset db8b917bbfdd by Martin Panter in branch 'default':
Issues #29349: Merge Py 2 fix 3.6
https://hg.python.org/cpython/rev/db8b917bbfdd

New changeset ca7d2af9920e by Martin Panter in branch 'default':
Issues #29349: Add NEWS for 3.7; use ?with? statement
https://hg.python.org/cpython/rev/ca7d2af9920e

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29349>
_______________________________________

From report at bugs.python.org  Sun Jan 29 05:34:28 2017
From: report at bugs.python.org (Roundup Robot)
Date: Sun, 29 Jan 2017 10:34:28 +0000
Subject: [issue12067] Doc: remove errors about mixed-type comparisons.
In-Reply-To: <1305237573.86.0.646542413513.issue12067@psf.upfronthosting.co.za>
Message-ID: <20170129103424.123333.86772.8400E46E@psf.io>


Roundup Robot added the comment:

New changeset 8c9a86aa222e by Martin Panter in branch '3.5':
Issue #12067: Recommend that hash and equality be consistent
https://hg.python.org/cpython/rev/8c9a86aa222e

New changeset 9702c5f08df1 by Martin Panter in branch '3.6':
Issues #12067: Merge hash recommendation from 3.5
https://hg.python.org/cpython/rev/9702c5f08df1

New changeset 9dbb7bbc1449 by Martin Panter in branch 'default':
Issues #12067: Merge hash recommendation from 3.6
https://hg.python.org/cpython/rev/9dbb7bbc1449

New changeset 8a9904c5cb1d by Martin Panter in branch '2.7':
Issue #12067: Rewrite Comparisons section in the language reference
https://hg.python.org/cpython/rev/8a9904c5cb1d

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue12067>
_______________________________________

From report at bugs.python.org  Sun Jan 29 05:53:31 2017
From: report at bugs.python.org (Martin Panter)
Date: Sun, 29 Jan 2017 10:53:31 +0000
Subject: [issue29349] Update old Python 2 code in
 Docs/tools/extensions/patchlevel.py
In-Reply-To: <1485164481.25.0.457345760711.issue29349@psf.upfronthosting.co.za>
Message-ID: <1485687211.09.0.719648418291.issue29349@psf.upfronthosting.co.za>


Martin Panter added the comment:

I think the general rule is to clean up code if you are doing something else in nearby code, but don?t go out of your way with unnecessary cleanups to arbitrary code. Otherwise it adds too much noise to the repository history, review process, risks adding bugs, etc, for little gain.

Anyway, thanks for the patch!

----------
resolution:  -> fixed
stage: commit review -> resolved
status: open -> closed
versions:  -Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29349>
_______________________________________

From report at bugs.python.org  Sun Jan 29 06:01:18 2017
From: report at bugs.python.org (Steven D'Aprano)
Date: Sun, 29 Jan 2017 11:01:18 +0000
Subject: [issue18842] Add float.is_finite is_nan is_infinite to match Decimal
 methods
In-Reply-To: <1485678185.45.0.74340746494.issue18842@psf.upfronthosting.co.za>
Message-ID: <20170129110054.GM7345@ando.pearwood.info>


Steven D'Aprano added the comment:

On Sun, Jan 29, 2017 at 08:23:05AM +0000, Martin Panter wrote:
> Why do you name the methods is_finite() etc with underscores, when the 
> existing methods math.isfinite() etc do not have underscores? Seems it 
> would add unnecessary confusion.

The idea is to enable duck-typing between float and Decimal. Instead of:

if isinstance(x, float):
    math.isfinite(x)
else:
    x.is_finite()

we can just say

x.is_finite() 

on both floats and decimals.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue18842>
_______________________________________

From report at bugs.python.org  Sun Jan 29 06:04:15 2017
From: report at bugs.python.org (Martin Panter)
Date: Sun, 29 Jan 2017 11:04:15 +0000
Subject: [issue18842] Add float.is_finite is_nan is_infinite to match Decimal
 methods
In-Reply-To: <1377532757.24.0.123652540945.issue18842@psf.upfronthosting.co.za>
Message-ID: <1485687855.45.0.622490937224.issue18842@psf.upfronthosting.co.za>


Martin Panter added the comment:

Of course, somehow I missed that

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue18842>
_______________________________________

From report at bugs.python.org  Sun Jan 29 06:35:57 2017
From: report at bugs.python.org (irdb)
Date: Sun, 29 Jan 2017 11:35:57 +0000
Subject: [issue10399] AST Optimization: inlining of function calls
In-Reply-To: <1289593012.28.0.71054855235.issue10399@psf.upfronthosting.co.za>
Message-ID: <1485689757.22.0.977754206136.issue10399@psf.upfronthosting.co.za>


Changes by irdb <electro.nnn at gmail.com>:


----------
nosy: +irdb

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue10399>
_______________________________________

From report at bugs.python.org  Sun Jan 29 07:44:29 2017
From: report at bugs.python.org (Martin Panter)
Date: Sun, 29 Jan 2017 12:44:29 +0000
Subject: [issue29384] Unused beos build scripts
In-Reply-To: <1485662815.62.0.566203367253.issue29384@psf.upfronthosting.co.za>
Message-ID: <1485693869.88.0.661610941049.issue29384@psf.upfronthosting.co.za>


Martin Panter added the comment:

Thanks Senthil

----------
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29384>
_______________________________________

From report at bugs.python.org  Sun Jan 29 09:09:42 2017
From: report at bugs.python.org (R. David Murray)
Date: Sun, 29 Jan 2017 14:09:42 +0000
Subject: [issue3991] urllib.request.urlopen does not handle non-ASCII
 characters
In-Reply-To: <1222627636.82.0.587488225038.issue3991@psf.upfronthosting.co.za>
Message-ID: <1485698982.51.0.712299536907.issue3991@psf.upfronthosting.co.za>


R. David Murray added the comment:

I believe the last time this subject was discussed the conclusion was that we really needed a full IRI module that conformed to the relevant RFCs, and that putting something on pypi would be one way to get there.  

Someone should research the existing packages.  It might be that we need something simpler than what exists, but whatever we do should be informed by what exists, I think.

----------
nosy: +r.david.murray

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue3991>
_______________________________________

From report at bugs.python.org  Sun Jan 29 09:45:25 2017
From: report at bugs.python.org (Mark Lawrence)
Date: Sun, 29 Jan 2017 14:45:25 +0000
Subject: [issue10399] AST Optimization: inlining of function calls
In-Reply-To: <1289593012.28.0.71054855235.issue10399@psf.upfronthosting.co.za>
Message-ID: <1485701125.58.0.343568890826.issue10399@psf.upfronthosting.co.za>


Changes by Mark Lawrence <breamoreboy at gmail.com>:


----------
nosy:  -BreamoreBoy

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue10399>
_______________________________________

From report at bugs.python.org  Sun Jan 29 12:11:40 2017
From: report at bugs.python.org (Ned Deily)
Date: Sun, 29 Jan 2017 17:11:40 +0000
Subject: [issue29349] Update old Python 2 code in
 Docs/tools/extensions/patchlevel.py
In-Reply-To: <1485164481.25.0.457345760711.issue29349@psf.upfronthosting.co.za>
Message-ID: <1485709900.48.0.744621228596.issue29349@psf.upfronthosting.co.za>


Ned Deily added the comment:

These changes have broken some buildbots, for example, the OS X dmg buildbots which are still using Python 2 for sphinx builds of the docs.  They should be version agnostic as much as possible.

http://buildbot.python.org/all/builders/bolen-dmg-3.x/builds/59

----------
nosy: +ned.deily
resolution: fixed -> 
stage: resolved -> needs patch
status: closed -> open

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29349>
_______________________________________

From report at bugs.python.org  Sun Jan 29 13:02:42 2017
From: report at bugs.python.org (Mark Dickinson)
Date: Sun, 29 Jan 2017 18:02:42 +0000
Subject: [issue18842] Add float.is_finite is_nan is_infinite to match Decimal
 methods
In-Reply-To: <1377532757.24.0.123652540945.issue18842@psf.upfronthosting.co.za>
Message-ID: <1485712962.62.0.586992590966.issue18842@psf.upfronthosting.co.za>


Mark Dickinson added the comment:

@Martin: the dead code should definitely be removed from floatobject.c and longobject.c. (Though that's kinda independent of this issue.)

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue18842>
_______________________________________

From report at bugs.python.org  Sun Jan 29 14:31:33 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Sun, 29 Jan 2017 19:31:33 +0000
Subject: [issue18842] Add float.is_finite is_nan is_infinite to match Decimal
 methods
In-Reply-To: <1377532757.24.0.123652540945.issue18842@psf.upfronthosting.co.za>
Message-ID: <1485718293.72.0.725324101396.issue18842@psf.upfronthosting.co.za>


Raymond Hettinger added the comment:

I would like to caution against expansion of core APIs in cases where we already have a way to do it.  In almost every corner, Python's APIs are growing, resulting in the language becoming massive, where very few people on the planet can claim to know what is in it and it is becoming much harder to teach as it gets bigger.

Also, we should place very little value on anecdotal evidence from one core developer who says he once wrote a single line of code that would have benefitted from a single part of the proposed changes.  In general, core devs tend to write much more generic code than end users, so our needs are often atypical and do not reflect user needs (in this case, no non-core-dev numeric user has ever requested this behavior or had a real use case that couldn't be met by the existing APIs).  The bar for new APIs is much higher than "I wrote a line of code once".  

We should be looking for APIs additions that significantly reduce complexity or solve problems that can't easily be met with existing APIs.  Minor expansions create long term maintenance burdens, increase the size of docs making them less usable, cause other implementations to have to follow suit, cause subclassers and users of the numeric module to have to implement additional methods, and make Python more difficult to learn.  There is a reason that the zen expresses a preference for only one way to do it.

Replicating parts the decimal API should always be suspect as well.  In general, that module is much more difficult to use and learn than regular floats.  Many parts of the API are simply worthless and are there only because they were part of the spec.  We have no evidence that any of these proposed methods are actually being used and are of real benefit to real users.

Also, remember that Decimal is not registered as a Real for a reason.  Guido does not consider them to be interoperable with binary floats.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue18842>
_______________________________________

From report at bugs.python.org  Sun Jan 29 14:47:51 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Sun, 29 Jan 2017 19:47:51 +0000
Subject: [issue29301] decimal: Use FASTCALL and/or Argument Clinic
In-Reply-To: <1484671124.21.0.493200110844.issue29301@psf.upfronthosting.co.za>
Message-ID: <1485719271.76.0.467129996301.issue29301@psf.upfronthosting.co.za>


Raymond Hettinger added the comment:

>> Ah you dislike the additional [clinic input] sections?
>
> Yes, they tear apart the code.  I stopped reading many C files because
> of this.

I concur with Stefan.  AC is very impactful on modules, greatly affecting their readability and maintainability.  The existing PyArg_ParseXXX() calls and their "kwlist" static variable are very easy to work with, easy to modify, and easy to teach (I cover them effortlessly in the Python classes I teach).  In contrast, AC is much harder to learn, harder to get right, doesn't fit some existing APIs, harder to change, and it greatly expands the number of lines of code.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29301>
_______________________________________

From report at bugs.python.org  Sun Jan 29 15:53:03 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sun, 29 Jan 2017 20:53:03 +0000
Subject: [issue18842] Add float.is_finite is_nan is_infinite to match Decimal
 methods
In-Reply-To: <1377532757.24.0.123652540945.issue18842@psf.upfronthosting.co.za>
Message-ID: <1485723183.18.0.99950494371.issue18842@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

I concur with Raymond. This expands the API too much. Not just the float API, but the API of all numeric classes, including third-party classes. And since the existence of additional method in third-party classes (e.g. NumPy classes) can't be guarantied, this couldn't help in the case of msg286441.

The less harmful way is making math.isfinite() and like supporting non-float numeric types. Always return False for integer types and fall back to the is_finite() method. But even this can be too expensive.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue18842>
_______________________________________

From report at bugs.python.org  Sun Jan 29 16:33:12 2017
From: report at bugs.python.org (Martin Panter)
Date: Sun, 29 Jan 2017 21:33:12 +0000
Subject: [issue29349] Update old Python 2 code in
 Docs/tools/extensions/patchlevel.py
In-Reply-To: <1485164481.25.0.457345760711.issue29349@psf.upfronthosting.co.za>
Message-ID: <1485725592.73.0.186929268257.issue29349@psf.upfronthosting.co.za>


Martin Panter added the comment:

Thanks Ned. Do you know what version of Python Sphinx uses (which runs patchlevel.py)?

According to Issue 28039, David set up Python 2.7 so that ?make touch? would work. But the log also uses a python2.5 command, and apparently Python 2.3 also installed on that buildbot.

If we can rely on 2.6+, we just need to add ?from __future__ import print_function?. Otherwise, we may need something like

if sys.stderr is not None:
    sys.stderr.write(...)

and also do something about that ?with? statement.

----------
nosy: +db3l

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29349>
_______________________________________

From report at bugs.python.org  Sun Jan 29 16:39:50 2017
From: report at bugs.python.org (Ned Deily)
Date: Sun, 29 Jan 2017 21:39:50 +0000
Subject: [issue29349] Update old Python 2 code in
 Docs/tools/extensions/patchlevel.py
In-Reply-To: <1485164481.25.0.457345760711.issue29349@psf.upfronthosting.co.za>
Message-ID: <1485725990.13.0.146934139959.issue29349@psf.upfronthosting.co.za>


Ned Deily added the comment:

Dunno for sure, perhaps David can answer that.  But the Sphinx docs imply that Sphinx 1.4.6 requires at least 2.6 and I don't test the installer build with anything less than 2.7.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29349>
_______________________________________

From report at bugs.python.org  Sun Jan 29 16:40:30 2017
From: report at bugs.python.org (Martin Panter)
Date: Sun, 29 Jan 2017 21:40:30 +0000
Subject: [issue29349] Update old Python 2 code in
 Docs/tools/extensions/patchlevel.py
In-Reply-To: <1485164481.25.0.457345760711.issue29349@psf.upfronthosting.co.za>
Message-ID: <1485726030.01.0.924903197101.issue29349@psf.upfronthosting.co.za>


Martin Panter added the comment:

According to <https://bugs.python.org/issue17861#msg217417>, Ned says 2.6+ is already needed to build the Python 3.5 documentation, so maybe Sphinx uses that.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29349>
_______________________________________

From report at bugs.python.org  Sun Jan 29 17:23:09 2017
From: report at bugs.python.org (Armin Rigo)
Date: Sun, 29 Jan 2017 22:23:09 +0000
Subject: [issue16899] Add support for C99 complex type (_Complex) as
 ctypes.c_complex
In-Reply-To: <1357682763.46.0.590255439385.issue16899@psf.upfronthosting.co.za>
Message-ID: <1485728589.37.0.810634062653.issue16899@psf.upfronthosting.co.za>


Armin Rigo added the comment:

* Tom: the issue is unrelated to cffi, but both ctypes and cffi could proceed to support C complexes, now that libffi support has been added.

* Mark: the problem is that the text you quote from the C standard fixes the representation of a complex in memory, but doesn't say anything about directly passing a complex as argument or return value to a function call.  Platforms use custom ways to do that.  The text you quote says a complex is an array of two real numbers; but passing an array as argument to a function works by passing a pointer to the first element.  Typically, this is not how complexes are passed: instead, some pointerless form of "passing two real numbers" is used.

----------
nosy: +arigo

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue16899>
_______________________________________

From report at bugs.python.org  Sun Jan 29 17:28:46 2017
From: report at bugs.python.org (David Bolen)
Date: Sun, 29 Jan 2017 22:28:46 +0000
Subject: [issue29349] Update old Python 2 code in
 Docs/tools/extensions/patchlevel.py
In-Reply-To: <1485164481.25.0.457345760711.issue29349@psf.upfronthosting.co.za>
Message-ID: <1485728926.47.0.599208119029.issue29349@psf.upfronthosting.co.za>


David Bolen added the comment:

2.7.12 (/usr/local/bin) is used for the build slave and main external commands (such as hg and sphinx) and is in the buildslave path, but 2.5.1 is still the default in /usr/bin so can get used for processes with a restricted environment.  Tiger's original 2.3 is present but unused.

The DMG build-installer script itself runs under 2.5.1, because it specifies "python2.5" in its command to the slave.  I believe that was originally added once the system 2.3 couldn't be used.

During the DMG construction process, a particular step might, I suppose use 2.5.1 or 2.7.12 depending on whether it's internal to the build script, something using the same executable as the script, a subprocess with limited path, or an independent utility (like sphinx) referencing 2.7.

I believe Ned's comment in issue 17861 about 2.6+ is a reference to the sphinx step, which is covered by the fact that sphinx is using the newer 2.7.12.

What appears to break in this specific case is some of the preparation - specifically the Makefile referencing patchlevel is running "python".  That should work (2.7.12) if it inherits the overall build slave environment (which has /usr/local/bin first) but I think the install script filters that, so it's probably getting the /usr/bin/python 2.5.1 version.

I've just changed the default /usr/bin/python to be 2.7 so external processes run by build-installer should get 2.7.  I don't think that will break anything else currently done on the machine, but I'll deal with any fallout if needed.  I'm rerunning build 59 now.

As in issue 28039 there's no immediate need to change the installer script itself, but since it's clearly the last hold out, I'll find some time to test that I've got any pre-requisites for it set up at which point the build master could be changed to remove the python2.5 reference.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29349>
_______________________________________

From report at bugs.python.org  Sun Jan 29 17:38:09 2017
From: report at bugs.python.org (David Bolen)
Date: Sun, 29 Jan 2017 22:38:09 +0000
Subject: [issue29349] Update old Python 2 code in
 Docs/tools/extensions/patchlevel.py
In-Reply-To: <1485164481.25.0.457345760711.issue29349@psf.upfronthosting.co.za>
Message-ID: <1485729489.56.0.225283612547.issue29349@psf.upfronthosting.co.za>


David Bolen added the comment:

Whoops, I just realized that the patch still needs adjusting to be 2.x compatible, so obviously the extra build still won't work.

But at this point it should be safe to assume 2.6+ such as for the rest of the sphinx processing.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29349>
_______________________________________

From report at bugs.python.org  Sun Jan 29 17:43:39 2017
From: report at bugs.python.org (Berker Peksag)
Date: Sun, 29 Jan 2017 22:43:39 +0000
Subject: [issue29283] duplicate README in site-packages
In-Reply-To: <1484565285.81.0.453565653975.issue29283@psf.upfronthosting.co.za>
Message-ID: <1485729819.26.0.253212042158.issue29283@psf.upfronthosting.co.za>


Berker Peksag added the comment:

Good catch and thanks for the report! Renaming was done in issue 24633. It looks like it was added back in a merge commit: https://github.com/python/cpython/commit/ebe304e919b56d889313a2d15fb5c52189a32ef7

I will delete Lib/site-packages/README shortly.

----------
nosy: +berker.peksag
type: enhancement -> behavior

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29283>
_______________________________________

From report at bugs.python.org  Sun Jan 29 18:48:17 2017
From: report at bugs.python.org (Roundup Robot)
Date: Sun, 29 Jan 2017 23:48:17 +0000
Subject: [issue11670] configparser read_file now iterates over f,
 docs still say it calls readline
In-Reply-To: <1301047699.5.0.476245530251.issue11670@psf.upfronthosting.co.za>
Message-ID: <20170129234814.50657.5948.0CD014A4@psf.io>


Roundup Robot added the comment:

New changeset e70882558a96 by Martin Panter in branch '3.5':
Issue #11670: readfp(fp) parameter name is different to read_file(f)
https://hg.python.org/cpython/rev/e70882558a96

New changeset e06de6f9cfe5 by Martin Panter in branch '3.6':
Issues #11670: Merge configparser doc from 3.5
https://hg.python.org/cpython/rev/e06de6f9cfe5

New changeset 6db0a62b6aa6 by Martin Panter in branch 'default':
Issues #11670: Merge configparser doc from 3.6
https://hg.python.org/cpython/rev/6db0a62b6aa6

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue11670>
_______________________________________

From report at bugs.python.org  Sun Jan 29 18:48:17 2017
From: report at bugs.python.org (Roundup Robot)
Date: Sun, 29 Jan 2017 23:48:17 +0000
Subject: [issue29349] Update old Python 2 code in
 Docs/tools/extensions/patchlevel.py
In-Reply-To: <1485164481.25.0.457345760711.issue29349@psf.upfronthosting.co.za>
Message-ID: <20170129234814.50657.86680.670C576D@psf.io>


Roundup Robot added the comment:

New changeset 9880928ec962 by Martin Panter in branch '3.5':
Issue #29349: Use __future__ print_function; Sphinx may use Python 2.6+
https://hg.python.org/cpython/rev/9880928ec962

New changeset 1708afd284ff by Martin Panter in branch '3.6':
Issues #29349: Merge Py 2.6+ compatibility from 3.5
https://hg.python.org/cpython/rev/1708afd284ff

New changeset e50058ecd808 by Martin Panter in branch 'default':
Issues #29349: Merge Py 2.6+ compatibility from 3.6
https://hg.python.org/cpython/rev/e50058ecd808

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29349>
_______________________________________

From report at bugs.python.org  Sun Jan 29 18:48:18 2017
From: report at bugs.python.org (Roundup Robot)
Date: Sun, 29 Jan 2017 23:48:18 +0000
Subject: [issue28822] Fix indices handling in PyUnicode_FindChar
In-Reply-To: <1480350558.9.0.582516213925.issue28822@psf.upfronthosting.co.za>
Message-ID: <20170129234814.50657.42256.3619351A@psf.io>


Roundup Robot added the comment:

New changeset 2b5e5a3a805e by Martin Panter in branch 'default':
Issue #28822: Add susp-ignored entry for NEWS; fix grammar
https://hg.python.org/cpython/rev/2b5e5a3a805e

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28822>
_______________________________________

From report at bugs.python.org  Sun Jan 29 18:53:58 2017
From: report at bugs.python.org (Martin Panter)
Date: Sun, 29 Jan 2017 23:53:58 +0000
Subject: [issue29387] Tabs vs spaces FAQ out of date
Message-ID: <1485734038.81.0.0566253928841.issue29387@psf.upfronthosting.co.za>


New submission from Martin Panter:

The Windows FAQ <https://docs.python.org/release/3.5.2/faq/windows.html#how-do-i-keep-editors-from-inserting-tabs-into-my-python-source> mentions the ?python -t? command-line option, but in Python 3 this option is undocumented (and I understand has no effect):

/media/disk/home/proj/python/cpython/Doc/faq/windows.rst:303: WARNING: unknown option: -t

Also, the reference to the tabnanny script is wrong. It exists as a module (Lib/tabnanny.py), not in Tools/scripts/. This aspect also applies to 2.7.

----------
assignee: docs at python
components: Documentation, Windows
files: tabnanny.patch
keywords: patch
messages: 286460
nosy: docs at python, martin.panter, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
stage: patch review
status: open
title: Tabs vs spaces FAQ out of date
versions: Python 2.7, Python 3.5, Python 3.6, Python 3.7
Added file: http://bugs.python.org/file46450/tabnanny.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29387>
_______________________________________

From report at bugs.python.org  Sun Jan 29 18:57:29 2017
From: report at bugs.python.org (Martin Panter)
Date: Sun, 29 Jan 2017 23:57:29 +0000
Subject: [issue29349] Update old Python 2 code in
 Docs/tools/extensions/patchlevel.py
In-Reply-To: <1485164481.25.0.457345760711.issue29349@psf.upfronthosting.co.za>
Message-ID: <1485734249.88.0.183192352206.issue29349@psf.upfronthosting.co.za>


Martin Panter added the comment:

I pushed the simpler 2.6-compatible option. Keeping this open to check the buildbot is happy overnight.

----------
stage: needs patch -> resolved

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29349>
_______________________________________

From report at bugs.python.org  Sun Jan 29 19:06:12 2017
From: report at bugs.python.org (Martin Panter)
Date: Mon, 30 Jan 2017 00:06:12 +0000
Subject: [issue18842] Add float.is_finite is_nan is_infinite to match Decimal
 methods
In-Reply-To: <1377532757.24.0.123652540945.issue18842@psf.upfronthosting.co.za>
Message-ID: <1485734772.87.0.907672685041.issue18842@psf.upfronthosting.co.za>


Martin Panter added the comment:

While I don?t have much opinion either way, here is a patch to remove the existing dead code should that be the eventual outcome.

If the default implementations in the base class deferred to math.isfinite() etc, that may help with compatibility.

----------
versions: +Python 3.7 -Python 3.4
Added file: http://bugs.python.org/file46451/rm-finite.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue18842>
_______________________________________

From report at bugs.python.org  Sun Jan 29 19:08:24 2017
From: report at bugs.python.org (Brett Cannon)
Date: Mon, 30 Jan 2017 00:08:24 +0000
Subject: [issue27200] make doctest in CPython has failures
In-Reply-To: <1464988408.0.0.840317371896.issue27200@psf.upfronthosting.co.za>
Message-ID: <1485734904.51.0.561410459688.issue27200@psf.upfronthosting.co.za>


Brett Cannon added the comment:

I'm -1 on building the docs from Sphinx master as I don't want to complicate our development process to deal with potential broken tooling. Basically if the doctests don't pass on a compiled checkout for the  version of Python the docs are being built for then the doctest is broken (i.e. no need to care about compatibility with older Python versions for docs meant for master).

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue27200>
_______________________________________

From report at bugs.python.org  Sun Jan 29 21:05:32 2017
From: report at bugs.python.org (MaxR)
Date: Mon, 30 Jan 2017 02:05:32 +0000
Subject: [issue29388] regex mismatch in the simple cases
Message-ID: <1485741932.85.0.668510587664.issue29388@psf.upfronthosting.co.za>


New submission from MaxR:

When I compile and match, for example:
pattern = re.compile(r"""    ([a-z]|\d)+$         # ends with letter or number only    """, re.VERBOSE | re.IGNORECASE)
print(re.match(pattern, 'abc'))

result is correct:
<_sre.SRE_Match object; span=(0, 3), match='abc'>

but then I use the same pattern with another string:
print(re.match(pattern, 'abc.fds'))

result is: 
None

I tried to reformulate the pattern to the same, for example:
pattern = re.compile(r"""    (a|b|c|d|e|f|g|h|i|j|k|l|m|n|o|p|q|r|s|t|u|v|w|x|y|z|\d)+$         # ends with letter or number only    """, re.VERBOSE | re.IGNORECASE)

or to:
pattern = re.compile(r"""    [a-z\d]+$         # ends with letter or number only    """, re.VERBOSE | re.IGNORECASE)

or to:

pattern = = re.compile(r"""    ([a-z]|\d)+$         # ends with letter or number only    """, re.VERBOSE | re.IGNORECASE)

but the result is all the time the same - None
And ?hat is the double strange for the last three pattern - None is result also for 
print(re.match(pattern, 'abc'))

I checked this patterns on the site regex101.com
and all of them should work good and right
It's achtung!

----------
components: Regular Expressions
messages: 286464
nosy: MaxR, ezio.melotti, mrabarnett
priority: normal
severity: normal
status: open
title: regex mismatch in the simple cases
type: behavior
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29388>
_______________________________________

From report at bugs.python.org  Sun Jan 29 21:13:42 2017
From: report at bugs.python.org (Mariatta Wijaya)
Date: Mon, 30 Jan 2017 02:13:42 +0000
Subject: [issue29381] Tutorial documentation contains undefined reference to #!
In-Reply-To: <1485537602.74.0.811037724367.issue29381@psf.upfronthosting.co.za>
Message-ID: <1485742422.17.0.130984260474.issue29381@psf.upfronthosting.co.za>


Mariatta Wijaya added the comment:

There are certain rules about the encoding declaration line.
It has to be the first line of the source code, or in the case that the file starts with a unix "shebang" line, then the encoding declaration has to be on the second line. In fact, the first or the second lines of the source code has to follow a certain regular expression. This is all described in PEP 263, but we don't have to repeat all of those here. IMO we can just refer the reader to the PEP.

I've prepared a patch that addresses this. Please review, and let me know if you have additional feedback.

Thanks :)

----------
Added file: http://bugs.python.org/file46452/issue29381.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29381>
_______________________________________

From report at bugs.python.org  Sun Jan 29 21:20:31 2017
From: report at bugs.python.org (Berker Peksag)
Date: Mon, 30 Jan 2017 02:20:31 +0000
Subject: [issue29248] os.readlink fails on Windows
In-Reply-To: <1484208560.98.0.375529555783.issue29248@psf.upfronthosting.co.za>
Message-ID: <1485742831.37.0.0659058083589.issue29248@psf.upfronthosting.co.za>


Berker Peksag added the comment:

Thanks for the patch. I think it would be better to use the existing framework in Win32SymlinkTests to create a test file instead of using a hardcoded path like 'C:\Users\All Users' (e.g. use Win32SymlinkTests.filelink and Win32SymlinkTests.filelink_target to create a link)

----------
nosy: +berker.peksag
stage: needs patch -> patch review

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29248>
_______________________________________

From report at bugs.python.org  Sun Jan 29 21:23:37 2017
From: report at bugs.python.org (Steven D'Aprano)
Date: Mon, 30 Jan 2017 02:23:37 +0000
Subject: [issue29388] regex mismatch in the simple cases
In-Reply-To: <1485741932.85.0.668510587664.issue29388@psf.upfronthosting.co.za>
Message-ID: <1485743017.87.0.474667779669.issue29388@psf.upfronthosting.co.za>


Steven D'Aprano added the comment:

re.match only matches at the start of the string. If you use re.search instead, you will get the results you are expecting.

----------
nosy: +steven.daprano
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29388>
_______________________________________

From report at bugs.python.org  Sun Jan 29 22:03:02 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Mon, 30 Jan 2017 03:03:02 +0000
Subject: [issue29381] Tutorial documentation contains undefined reference to #!
In-Reply-To: <1485537602.74.0.811037724367.issue29381@psf.upfronthosting.co.za>
Message-ID: <1485745382.29.0.127450019319.issue29381@psf.upfronthosting.co.za>


Raymond Hettinger added the comment:

Suggestions:

* Please add a cross-link to from '''UNIX ?shebang? line"''' to 16.1.2. Executable Python Scripts.

* Instead of a link to a PEP (which is outside the main documentation), let's just add an example:

   # /usr/bin/env python3
   # -*- coding: cp-1252 -*-

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29381>
_______________________________________

From report at bugs.python.org  Sun Jan 29 22:29:11 2017
From: report at bugs.python.org (Eryk Sun)
Date: Mon, 30 Jan 2017 03:29:11 +0000
Subject: [issue29248] os.readlink fails on Windows
In-Reply-To: <1484208560.98.0.375529555783.issue29248@psf.upfronthosting.co.za>
Message-ID: <1485746951.6.0.770476482088.issue29248@psf.upfronthosting.co.za>


Eryk Sun added the comment:

os.symlink calls CreateSymbolicLink, which creates the reparse data buffer with the print name stored first, so the offset is always 0. Otherwise we would have noticed this problem already. For example:

    >>> os.symlink('C:\\', 'link', True)
    >>> os.system('fsutil reparsepoint query link')
    Reparse Tag Value : 0xa000000c
    Tag value: Microsoft
    Tag value: Name Surrogate
    Tag value: Symbolic Link

    Reparse Data Length: 0x00000020
    Reparse Data:
    0000:  06 00 0e 00 00 00 06 00  00 00 00 00 43 00 3a 00  ............C.:.
    0010:  5c 00 5c 00 3f 00 3f 00  5c 00 43 00 3a 00 5c 00  \.\.?.?.\.C.:.\.

As you can see above, CreateSymbolicLink stores the "PrintName" DOS path (e.g. "C:\") first, with an offset of 0, followed by the "SubstituteName" NT path (e.g. "\??\C:\").

The "All Users" link, on the other hand, seems to have been created manually with an inverted order. I have ctypes code to manually create a similar link (calling OpenProcessToken/AdjustTokenPrivileges to enable the required privilege and CreateFile/DeviceIoControl to set the reparse point), but I doubt it's worth adding it just to test this bug.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29248>
_______________________________________

From report at bugs.python.org  Sun Jan 29 22:33:40 2017
From: report at bugs.python.org (Mariatta Wijaya)
Date: Mon, 30 Jan 2017 03:33:40 +0000
Subject: [issue29381] Tutorial documentation contains undefined reference to #!
In-Reply-To: <1485537602.74.0.811037724367.issue29381@psf.upfronthosting.co.za>
Message-ID: <1485747220.53.0.614995865534.issue29381@psf.upfronthosting.co.za>


Mariatta Wijaya added the comment:

Thanks for the feedback, Raymond. I adjusted my patch.

----------
Added file: http://bugs.python.org/file46453/issue29381v2.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29381>
_______________________________________

From report at bugs.python.org  Mon Jan 30 00:01:28 2017
From: report at bugs.python.org (Martin Panter)
Date: Mon, 30 Jan 2017 05:01:28 +0000
Subject: [issue29070] Integration tests for pty.spawn on Linux and all other
 platforms
In-Reply-To: <1482684225.17.0.467768789806.issue29070@psf.upfronthosting.co.za>
Message-ID: <1485752488.68.0.348243937541.issue29070@psf.upfronthosting.co.za>


Martin Panter added the comment:

Can you explain your broken pipe situation? Are you talking about a real-world EPIPE operating on a pseudoterminal, or just a result of using a Unix socket to emulate a PTY in the tests? Usually a broken pipe is an asynchronous condition. You cannot predict exactly when it will happen without knowing the state of the other end, OS implementation, buffering, etc. It does not seem appropriate to change the _copy() loop around unless there is an real bug.

Regarding the buildbots, if I get this patch into a state that I am comfortable committing, the buildbots will are generally set to timeout in 15 or 20 minutes. See the ?make buildbottest? commands in Makefile.pre.in, test.regrtest --timeout option, etc. You can also see which platforms have buildbots, and the state of them etc: <https://www.python.org/dev/buildbot/>.

I left a bunch of comments in the code review. There is a lot of useful code in there, but also a lot of stuff that is hard to follow or that I want to clean up.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29070>
_______________________________________

From report at bugs.python.org  Mon Jan 30 02:09:58 2017
From: report at bugs.python.org (INADA Naoki)
Date: Mon, 30 Jan 2017 07:09:58 +0000
Subject: [issue29385] Sockets Crashes or Memory Corruption
In-Reply-To: <1485668160.59.0.222592776098.issue29385@psf.upfronthosting.co.za>
Message-ID: <1485760198.74.0.309875403681.issue29385@psf.upfronthosting.co.za>


INADA Naoki added the comment:

I can't reproduce it because of some reason.

1. While you report it affects Python 3.3~3.7, the script doesn't work on Python 3.7
2. I've fixed webserver.py to work on Python 3, but it uses very old (draft) websocket protocol.  Current browser doesn't send headers like "Sec-WebSocket-Key1".
3. Even if handshake succeed, you can't send `\xff`*10 directly.  You should wrap it with "frame", specified in RFC.

I can't understand what "Close/Crash" means.
But I propose you to use websocket libraries, or read protocol specification carefully.
WebSocket is not "raw TCP socket after handshake", it's framed protocol on top of TCP (or other stream protocol).

----------
nosy: +inada.naoki
resolution:  -> not a bug
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29385>
_______________________________________

From report at bugs.python.org  Mon Jan 30 02:51:50 2017
From: report at bugs.python.org (INADA Naoki)
Date: Mon, 30 Jan 2017 07:51:50 +0000
Subject: [issue15373] copy.copy() does not properly copy os.environment
In-Reply-To: <1342454937.95.0.386630033182.issue15373@psf.upfronthosting.co.za>
Message-ID: <1485762710.61.0.243162520517.issue15373@psf.upfronthosting.co.za>


INADA Naoki added the comment:

patch looks OK.
But I prefer `__deepcopy__ = __copy__ = copy`.

I don't know how to support pickling. (should unpickled object
reference os.environ, or copied dict?)
I feel it is different issue.

----------
nosy: +inada.naoki

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue15373>
_______________________________________

From report at bugs.python.org  Mon Jan 30 04:02:05 2017
From: report at bugs.python.org (Shmuel Amar)
Date: Mon, 30 Jan 2017 09:02:05 +0000
Subject: [issue29389] math.isclose signature contains incorrect start parameter
Message-ID: <1485766925.53.0.41080948643.issue29389@psf.upfronthosting.co.za>


New submission from Shmuel Amar:

documentation of math.isclose() signature on https://docs.python.org/3/library/math.html#math.isclose is as follows:

    math.isclose(a, b, *, rel_tol=1e-09, abs_tol=0.0)

the third star '*' argument is not allowed inside the function: 
https://hg.python.org/cpython/file/tip/Modules/clinic/mathmodule.c.h#l511

or mentioned in PEP485 here: https://www.python.org/dev/peps/pep-0485/#implementation

and does not work when trying provide more than 2 positional values:

>>> import math
>>> math.isclose(1,2,3, rel_tol=5.)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: Function takes at most 2 positional arguments (3 given)

So IMHO to solve this remove the positional argument on the signature of isclose() as it misleading.

----------
assignee: docs at python
components: Documentation
messages: 286474
nosy: Shmuel Amar, docs at python
priority: normal
severity: normal
status: open
title: math.isclose signature contains incorrect start parameter
versions: Python 3.5, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29389>
_______________________________________

From report at bugs.python.org  Mon Jan 30 04:03:45 2017
From: report at bugs.python.org (Shmuel Amar)
Date: Mon, 30 Jan 2017 09:03:45 +0000
Subject: [issue29389] math.isclose signature contains incorrect start parameter
In-Reply-To: <1485766925.53.0.41080948643.issue29389@psf.upfronthosting.co.za>
Message-ID: <1485767025.21.0.817398920861.issue29389@psf.upfronthosting.co.za>


Shmuel Amar added the comment:

documentation of math.isclose() signature on https://docs.python.org/3/library/math.html#math.isclose is as follows:

    math.isclose(a, b, *, rel_tol=1e-09, abs_tol=0.0)

the third star '*' argument is not allowed inside the function: 
https://hg.python.org/cpython/file/tip/Modules/clinic/mathmodule.c.h#l511

or mentioned in PEP485 here: https://www.python.org/dev/peps/pep-0485/#implementation

and does not work when trying provide more than 2 positional values:

>>> import math
>>> math.isclose(1,2,3, rel_tol=5.)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: Function takes at most 2 positional arguments (3 given)

So IMHO to solve this remove the positional argument on the signature of isclose() as it misleading.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29389>
_______________________________________

From report at bugs.python.org  Mon Jan 30 04:04:25 2017
From: report at bugs.python.org (Mark Dickinson)
Date: Mon, 30 Jan 2017 09:04:25 +0000
Subject: [issue18842] Add float.is_finite is_nan is_infinite to match Decimal
 methods
In-Reply-To: <1377532757.24.0.123652540945.issue18842@psf.upfronthosting.co.za>
Message-ID: <1485767065.73.0.46950063095.issue18842@psf.upfronthosting.co.za>


Mark Dickinson added the comment:

rm-finite.patch LGTM.

Once that's merged, let's close this issue as rejected. If decimal is ever added as a first-class type (literal support, fixed-width decimal128 or decimal64, etc.), we may want to reconsider, but that day is probably a long way off.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue18842>
_______________________________________

From report at bugs.python.org  Mon Jan 30 04:06:27 2017
From: report at bugs.python.org (Mark Dickinson)
Date: Mon, 30 Jan 2017 09:06:27 +0000
Subject: [issue29389] math.isclose signature contains incorrect start parameter
In-Reply-To: <1485766925.53.0.41080948643.issue29389@psf.upfronthosting.co.za>
Message-ID: <1485767187.73.0.152162258163.issue29389@psf.upfronthosting.co.za>


Mark Dickinson added the comment:

I think you're misreading the signature. The '*' is not a parameter in its own right; it's a piece of syntax marking the end of the positional parameters. Everything following that can only be passed by keyword.  See PEP 3102 for an explanation of the syntax.

----------
nosy: +mark.dickinson

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29389>
_______________________________________

From report at bugs.python.org  Mon Jan 30 04:09:54 2017
From: report at bugs.python.org (Shmuel Amar)
Date: Mon, 30 Jan 2017 09:09:54 +0000
Subject: [issue29389] math.isclose signature contains incorrect start parameter
In-Reply-To: <1485766925.53.0.41080948643.issue29389@psf.upfronthosting.co.za>
Message-ID: <1485767394.4.0.707683484013.issue29389@psf.upfronthosting.co.za>


Shmuel Amar added the comment:

ok thanks i think it can be closed

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29389>
_______________________________________

From report at bugs.python.org  Mon Jan 30 04:11:25 2017
From: report at bugs.python.org (Mark Dickinson)
Date: Mon, 30 Jan 2017 09:11:25 +0000
Subject: [issue29328] struct module should support variable-length strings
In-Reply-To: <1484850272.25.0.472309940141.issue29328@psf.upfronthosting.co.za>
Message-ID: <1485767485.69.0.128651149969.issue29328@psf.upfronthosting.co.za>


Changes by Mark Dickinson <dickinsm at gmail.com>:


----------
stage:  -> needs patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29328>
_______________________________________

From report at bugs.python.org  Mon Jan 30 04:55:36 2017
From: report at bugs.python.org (Gabriel McManus)
Date: Mon, 30 Jan 2017 09:55:36 +0000
Subject: [issue29386] select.epoll.poll may behave differently if timeout = -1
 vs timeout = None
In-Reply-To: <1485671321.44.0.0475790676768.issue29386@psf.upfronthosting.co.za>
Message-ID: <1485770136.18.0.711824613675.issue29386@psf.upfronthosting.co.za>


Gabriel McManus added the comment:

As mentioned in [1], Illumos will be fixed to match Linux's behaviour, so this problem will go away. It may still be worth changing epoll to just send -1 though, in case this causes similar issues in other operating systems.

[1] https://github.com/joyent/illumos-joyent/issues/136

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29386>
_______________________________________

From report at bugs.python.org  Mon Jan 30 05:05:10 2017
From: report at bugs.python.org (Michael Kesper)
Date: Mon, 30 Jan 2017 10:05:10 +0000
Subject: [issue29390] Python Tutorial should introduce iterators and generators
Message-ID: <1485770709.65.0.205484403801.issue29390@psf.upfronthosting.co.za>


New submission from Michael Kesper:

Please look at http://stackoverflow.com/questions/41932287/how-can-i-create-a-loop-for-these-if-statements/41932494?noredirect=1#41932494 

For beginners, it would be good to introduce the concepts of 'pythonic' dealing with sequences (iterators) and streams (generators) as soon as possible and everywhere where sequenceable data structures are discussed.

It is a common idiom in other languages to access members of sequences with counters, risking off-by-one errors or IndexErrors.

If beginners aren't introduced to the 'right' concepts soon enough, they might have a hard time figuring out the 'iterator' way.
Iterators should at least be mentioned in https://docs.python.org/3/tutorial/datastructures.html#looping-techniques.

----------
assignee: docs at python
components: Documentation
messages: 286480
nosy: docs at python, mkesper
priority: normal
severity: normal
status: open
title: Python Tutorial should introduce iterators and generators
type: enhancement

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29390>
_______________________________________

From report at bugs.python.org  Mon Jan 30 05:17:23 2017
From: report at bugs.python.org (Marco Buttu)
Date: Mon, 30 Jan 2017 10:17:23 +0000
Subject: [issue29381] Tutorial documentation contains undefined reference to #!
In-Reply-To: <1485537602.74.0.811037724367.issue29381@psf.upfronthosting.co.za>
Message-ID: <1485771443.4.0.133643776315.issue29381@psf.upfronthosting.co.za>


Marco Buttu added the comment:

The patch LGTM. I think there is just one type (see review). 

By the way, looking at the PEP 0263, the sentence "To define a source code encoding, a magic comment must be placed into the source files either as first or second line in the file" actually is not true. Also because I did not see any point in the PEP that clarifies that the fist line has to be a comment. Maybe it could be deduced by the examples (all using the shebang), but maybe not, and the user could think this is a valid declaration:

print('tr?s jolie')
# -*- coding: ascii -*-

If you think is worth to clarify, I open an issue with a patch for the PEP 0263.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29381>
_______________________________________

From report at bugs.python.org  Mon Jan 30 05:45:14 2017
From: report at bugs.python.org (Berker Peksag)
Date: Mon, 30 Jan 2017 10:45:14 +0000
Subject: [issue29247] Document return value of epoll.poll
In-Reply-To: <1484195034.62.0.718564451527.issue29247@psf.upfronthosting.co.za>
Message-ID: <1485773114.48.0.822987739457.issue29247@psf.upfronthosting.co.za>


Berker Peksag added the comment:

Thanks for the report, Nathaniel. I agree that the documentation could be improved. Here is a patch. I also moved the constant table to epoll.register() documentation (it's already done in https://docs.python.org/3.7/library/select.html#polling-objects)

It would be great if you could review the patch, thanks!

----------
keywords: +patch
nosy: +berker.peksag
stage:  -> patch review
type:  -> enhancement
Added file: http://bugs.python.org/file46454/issue29247.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29247>
_______________________________________

From report at bugs.python.org  Mon Jan 30 05:51:39 2017
From: report at bugs.python.org (Roundup Robot)
Date: Mon, 30 Jan 2017 10:51:39 +0000
Subject: [issue29283] duplicate README in site-packages
In-Reply-To: <1484565285.81.0.453565653975.issue29283@psf.upfronthosting.co.za>
Message-ID: <20170130105135.126528.50462.361DDAD8@psf.io>


Roundup Robot added the comment:

New changeset 23fcc0e51510 by Berker Peksag in branch '3.6':
Issue #29283: Delete accidentally added Lib/site-packages/README
https://hg.python.org/cpython/rev/23fcc0e51510

New changeset 3d328ee18612 by Berker Peksag in branch 'default':
Issue #29283: Merge from 3.6
https://hg.python.org/cpython/rev/3d328ee18612

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29283>
_______________________________________

From report at bugs.python.org  Mon Jan 30 05:52:36 2017
From: report at bugs.python.org (Berker Peksag)
Date: Mon, 30 Jan 2017 10:52:36 +0000
Subject: [issue29283] duplicate README in site-packages
In-Reply-To: <1484565285.81.0.453565653975.issue29283@psf.upfronthosting.co.za>
Message-ID: <1485773556.2.0.798591501806.issue29283@psf.upfronthosting.co.za>


Changes by Berker Peksag <berker.peksag at gmail.com>:


----------
resolution:  -> fixed
stage:  -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29283>
_______________________________________

From report at bugs.python.org  Mon Jan 30 05:52:43 2017
From: report at bugs.python.org (Berker Peksag)
Date: Mon, 30 Jan 2017 10:52:43 +0000
Subject: [issue29283] duplicate README in site-packages
In-Reply-To: <1484565285.81.0.453565653975.issue29283@psf.upfronthosting.co.za>
Message-ID: <1485773563.88.0.878318776917.issue29283@psf.upfronthosting.co.za>


Changes by Berker Peksag <berker.peksag at gmail.com>:


----------
components:  -Installation

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29283>
_______________________________________

From report at bugs.python.org  Mon Jan 30 06:08:16 2017
From: report at bugs.python.org (bli)
Date: Mon, 30 Jan 2017 11:08:16 +0000
Subject: [issue9914] trace/profile conflict with the use of
 sys.modules[__name__]
In-Reply-To: <1285093065.41.0.819526070425.issue9914@psf.upfronthosting.co.za>
Message-ID: <1485774496.7.0.897454069965.issue9914@psf.upfronthosting.co.za>


bli added the comment:

Just to report that this bug seems responsible for failures at cProfiling some code using multiprocessing:

http://stackoverflow.com/q/41892297/1878788
http://stackoverflow.com/q/11512499/1878788

----------
nosy: +bli

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue9914>
_______________________________________

From report at bugs.python.org  Mon Jan 30 06:37:44 2017
From: report at bugs.python.org (Jim Fasarakis-Hilliard)
Date: Mon, 30 Jan 2017 11:37:44 +0000
Subject: [issue29390] Python Tutorial should introduce iterators and generators
In-Reply-To: <1485770709.65.0.205484403801.issue29390@psf.upfronthosting.co.za>
Message-ID: <1485776264.88.0.50322214077.issue29390@psf.upfronthosting.co.za>


Jim Fasarakis-Hilliard added the comment:

What change do you have in mind for introducing these?

As for my personal opinion, dunno about this. I understand your concerns but dropping more terminology to a new learner early on wouldn't be the best idea in my view. 

>From what I am aware, Python is not becoming one of the *first* languages people learn due to its friendly nature. Adding concepts like generators and iterators when a new user is struggling with appends and mutability doesn't seem like it would help. 

BTW, iterators and generators are discussed in the Classes section of the tutorial, don't you think this suffices?

----------
nosy: +Jim Fasarakis-Hilliard

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29390>
_______________________________________

From report at bugs.python.org  Mon Jan 30 06:38:14 2017
From: report at bugs.python.org (Jim Fasarakis-Hilliard)
Date: Mon, 30 Jan 2017 11:38:14 +0000
Subject: [issue29390] Python Tutorial should introduce iterators and generators
In-Reply-To: <1485770709.65.0.205484403801.issue29390@psf.upfronthosting.co.za>
Message-ID: <1485776294.1.0.419220997034.issue29390@psf.upfronthosting.co.za>


Jim Fasarakis-Hilliard added the comment:

Typo, is *now* becoming :-)

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29390>
_______________________________________

From report at bugs.python.org  Mon Jan 30 06:55:50 2017
From: report at bugs.python.org (Marco Buttu)
Date: Mon, 30 Jan 2017 11:55:50 +0000
Subject: [issue29247] Document return value of epoll.poll
In-Reply-To: <1484195034.62.0.718564451527.issue29247@psf.upfronthosting.co.za>
Message-ID: <1485777350.67.0.554230348871.issue29247@psf.upfronthosting.co.za>


Marco Buttu added the comment:

The patch LGTM. There is just one typo, I think (see review).

----------
nosy: +marco.buttu

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29247>
_______________________________________

From report at bugs.python.org  Mon Jan 30 07:14:29 2017
From: report at bugs.python.org (Jim Fasarakis-Hilliard)
Date: Mon, 30 Jan 2017 12:14:29 +0000
Subject: [issue29381] Tutorial documentation contains undefined reference to #!
In-Reply-To: <1485537602.74.0.811037724367.issue29381@psf.upfronthosting.co.za>
Message-ID: <1485778469.49.0.114687305482.issue29381@psf.upfronthosting.co.za>


Jim Fasarakis-Hilliard added the comment:

Added a comment too. Other than that and the comment by Marco it looks fine to me too.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29381>
_______________________________________

From report at bugs.python.org  Mon Jan 30 07:22:15 2017
From: report at bugs.python.org (Manuel Krebber)
Date: Mon, 30 Jan 2017 12:22:15 +0000
Subject: [issue29377] Add the 'wrapper_descriptor' type to the types module
In-Reply-To: <1485425778.09.0.153066086326.issue29377@psf.upfronthosting.co.za>
Message-ID: <1485778935.35.0.452556436339.issue29377@psf.upfronthosting.co.za>


Manuel Krebber added the comment:

Okay, I added MethodDescriptorType to the types module and updated the docs. Hope this is okay now.

----------
Added file: http://bugs.python.org/file46455/slot-wrapper-types.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29377>
_______________________________________

From report at bugs.python.org  Mon Jan 30 08:19:17 2017
From: report at bugs.python.org (Marco Buttu)
Date: Mon, 30 Jan 2017 13:19:17 +0000
Subject: [issue27200] make doctest in CPython has failures
In-Reply-To: <1464988408.0.0.840317371896.issue27200@psf.upfronthosting.co.za>
Message-ID: <1485782357.44.0.911415974301.issue27200@psf.upfronthosting.co.za>


Marco Buttu added the comment:

Thanks Brett :-) Following the suggestion of Ezio to split the patch in more patches (msg284560), and according to msg284628, here is a first patch for just three files. To run their doctests:

$ sphinx-build -b doctest . build/doctest \
library/configparser.rst \
library/ctypes.rst \
library/copyreg.rst

----------
Added file: http://bugs.python.org/file46456/configparser_ctypes_copyreg.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue27200>
_______________________________________

From report at bugs.python.org  Mon Jan 30 09:08:10 2017
From: report at bugs.python.org (Davin Potts)
Date: Mon, 30 Jan 2017 14:08:10 +0000
Subject: [issue9914] trace/profile conflict with the use of
 sys.modules[__name__]
In-Reply-To: <1285093065.41.0.819526070425.issue9914@psf.upfronthosting.co.za>
Message-ID: <1485785290.55.0.377005994671.issue9914@psf.upfronthosting.co.za>


Davin Potts added the comment:

Though this issue is specifically concerned with runpy APIs and their impact especially in running unittest test scripts, it's worth commenting here for people who need a workaround in the short term:  code such as that shared in http://stackoverflow.com/q/41892297/1878788 can be made to run happily by creating a second script which imports the first and simply runs the test(s) from there.

In the specific case of the 'forkiter.py' from http://stackoverflow.com/q/41892297/1878788, one would create a 'run_my_tests.py' with the contents:

from forkiter import main

if __name__ == "__main__":
    exit(main())





Now this invocation of cProfile runs happily because pickle is able to see the module where all the needed classes/functions were defined:
python3.6 -m cProfile -o forkiter.prof ./run_my_tests.py

----------
nosy: +davin

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue9914>
_______________________________________

From report at bugs.python.org  Mon Jan 30 10:53:13 2017
From: report at bugs.python.org (Glenn Clark)
Date: Mon, 30 Jan 2017 15:53:13 +0000
Subject: [issue29391] Windows Defender finds trojan Spursint in Numpy for Py36
 Win64
Message-ID: <1485791593.43.0.325442187579.issue29391@psf.upfronthosting.co.za>


New submission from Glenn Clark:

Windows Defender trojan Spursint Numpy Python3.6 Python 3.4 Win64

----------
components: Windows
messages: 286492
nosy: Glenn Clark, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Windows Defender finds trojan Spursint in Numpy for Py36 Win64
type: security
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29391>
_______________________________________

From report at bugs.python.org  Mon Jan 30 11:28:25 2017
From: report at bugs.python.org (Eric V. Smith)
Date: Mon, 30 Jan 2017 16:28:25 +0000
Subject: [issue29391] Windows Defender finds trojan Spursint in Numpy for Py36
 Win64
In-Reply-To: <1485791593.43.0.325442187579.issue29391@psf.upfronthosting.co.za>
Message-ID: <1485793705.46.0.317299688697.issue29391@psf.upfronthosting.co.za>


Eric V. Smith added the comment:

You should report this to the Numpy tracker. It's a third party package.

----------
nosy: +eric.smith

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29391>
_______________________________________

From report at bugs.python.org  Mon Jan 30 11:28:43 2017
From: report at bugs.python.org (Terry J. Reedy)
Date: Mon, 30 Jan 2017 16:28:43 +0000
Subject: [issue4630] IDLE: add cursor noblink option
In-Reply-To: <1228982754.08.0.523311497339.issue4630@psf.upfronthosting.co.za>
Message-ID: <1485793723.89.0.928366329915.issue4630@psf.upfronthosting.co.za>


Terry J. Reedy added the comment:

SO request for same: https://stackoverflow.com/questions/41937748/stop-cursor-blinking-in-idle.

----------
assignee:  -> terry.reedy
versions: +Python 3.6, Python 3.7 -Python 2.7, Python 3.3, Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue4630>
_______________________________________

From report at bugs.python.org  Mon Jan 30 12:30:26 2017
From: report at bugs.python.org (Frazer McLean)
Date: Mon, 30 Jan 2017 17:30:26 +0000
Subject: [issue29392] msvcrt.locking crashes python
Message-ID: <1485797426.48.0.513516354213.issue29392@psf.upfronthosting.co.za>


New submission from Frazer McLean:

On 3.5.2 and 3.5.3, but not 3.4.3 or 2.7.10 (Windows 64-bit), the third line causes Python to crash with exit code 0xC0000417.

import msvcrt
fp = open('file.txt', 'w')
msvcrt.locking(fp.fileno(), msvcrt.LK_NBLCK, -1)
print('End')

I came across this in the portalocker module. I don't think -1 is a valid third argument, but it doesn't raise an exception on Python 3.4 or 2.7.

----------
components: Windows
messages: 286495
nosy: RazerM, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: msvcrt.locking crashes python
type: crash
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29392>
_______________________________________

From report at bugs.python.org  Mon Jan 30 13:08:11 2017
From: report at bugs.python.org (Ned Deily)
Date: Mon, 30 Jan 2017 18:08:11 +0000
Subject: [issue29349] Update old Python 2 code in
 Docs/tools/extensions/patchlevel.py
In-Reply-To: <1485164481.25.0.457345760711.issue29349@psf.upfronthosting.co.za>
Message-ID: <1485799691.17.0.345888385228.issue29349@psf.upfronthosting.co.za>


Ned Deily added the comment:

The fix looks good and the dmg buildbots are happy again.  Thanks, Martin and David.

----------
resolution:  -> fixed
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29349>
_______________________________________

From report at bugs.python.org  Mon Jan 30 13:40:32 2017
From: report at bugs.python.org (Brett Cannon)
Date: Mon, 30 Jan 2017 18:40:32 +0000
Subject: [issue22594] Add a link to the regex module in re documentation
In-Reply-To: <1412927285.57.0.392292696343.issue22594@psf.upfronthosting.co.za>
Message-ID: <1485801631.99.0.509771910332.issue22594@psf.upfronthosting.co.za>


Changes by Brett Cannon <brett at python.org>:


----------
assignee: docs at python -> brett.cannon
nosy: +brett.cannon

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue22594>
_______________________________________

From report at bugs.python.org  Mon Jan 30 14:09:29 2017
From: report at bugs.python.org (Steve Dower)
Date: Mon, 30 Jan 2017 19:09:29 +0000
Subject: [issue28231] zipfile does not support pathlib
In-Reply-To: <1474445503.11.0.219964117103.issue28231@psf.upfronthosting.co.za>
Message-ID: <1485803369.4.0.0362124688207.issue28231@psf.upfronthosting.co.za>


Steve Dower added the comment:

Speaking as a "regular user" who just ran into this, my main concern is that PathLike paths get used properly. For filenames being passed back out, if I really want them to be Path objects, I'll wrap them in Path() anyway.

Please don't let a full conversion to pathlib hold up fixing passing a PathLike into the constructor. That's the main use case, and given how nicely most of the rest of the stdlib handles Path objects now, it's an annoying wart.

----------
nosy: +steve.dower

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28231>
_______________________________________

From report at bugs.python.org  Mon Jan 30 15:41:11 2017
From: report at bugs.python.org (Ammar Askar)
Date: Mon, 30 Jan 2017 20:41:11 +0000
Subject: [issue29389] math.isclose signature contains incorrect start parameter
In-Reply-To: <1485766925.53.0.41080948643.issue29389@psf.upfronthosting.co.za>
Message-ID: <1485808871.35.0.574563511712.issue29389@psf.upfronthosting.co.za>


Changes by Ammar Askar <ammar at ammaraskar.com>:


----------
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29389>
_______________________________________

From report at bugs.python.org  Mon Jan 30 17:26:24 2017
From: report at bugs.python.org (Ezio Melotti)
Date: Mon, 30 Jan 2017 22:26:24 +0000
Subject: [issue2771] Test issue
In-Reply-To: <1210005645.74.0.283923986194.issue2771@psf.upfronthosting.co.za>
Message-ID: <1485815184.67.0.657412532969.issue2771@psf.upfronthosting.co.za>


Ezio Melotti added the comment:

test

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue2771>
_______________________________________

From report at bugs.python.org  Mon Jan 30 17:43:03 2017
From: report at bugs.python.org (Ivan Levkivskyi)
Date: Mon, 30 Jan 2017 22:43:03 +0000
Subject: [issue29377] Add the 'wrapper_descriptor' type to the types module
In-Reply-To: <1485425778.09.0.153066086326.issue29377@psf.upfronthosting.co.za>
Message-ID: <1485816183.42.0.687732494935.issue29377@psf.upfronthosting.co.za>


Ivan Levkivskyi added the comment:

Thank you!
The new patch LGTM.
(I combined two diffs in your patch into one so that it could be understood by Rietveld).

Guido, Yury, could one of you please take a look at this?

----------
nosy: +yselivanov
Added file: http://bugs.python.org/file46457/combined-patch.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29377>
_______________________________________

From report at bugs.python.org  Mon Jan 30 17:49:13 2017
From: report at bugs.python.org (Eryk Sun)
Date: Mon, 30 Jan 2017 22:49:13 +0000
Subject: [issue29391] Windows Defender finds trojan Spursint in Numpy for Py36
 Win64
In-Reply-To: <1485791593.43.0.325442187579.issue29391@psf.upfronthosting.co.za>
Message-ID: <1485816553.33.0.743419964999.issue29391@psf.upfronthosting.co.za>


Changes by Eryk Sun <eryksun at gmail.com>:


----------
resolution:  -> third party
stage:  -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29391>
_______________________________________

From report at bugs.python.org  Mon Jan 30 18:09:43 2017
From: report at bugs.python.org (Frazer McLean)
Date: Mon, 30 Jan 2017 23:09:43 +0000
Subject: [issue29392] msvcrt.locking crashes python
In-Reply-To: <1485797426.48.0.513516354213.issue29392@psf.upfronthosting.co.za>
Message-ID: <1485817783.05.0.0917207869186.issue29392@psf.upfronthosting.co.za>


Frazer McLean added the comment:

I ran it on AppVeyor to double check, the crash appears from 3.5 onwards as I saw on my computer.

https://ci.appveyor.com/project/RazerM/issue29392/build/3

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29392>
_______________________________________

From report at bugs.python.org  Mon Jan 30 18:40:32 2017
From: report at bugs.python.org (Eryk Sun)
Date: Mon, 30 Jan 2017 23:40:32 +0000
Subject: [issue29392] msvcrt.locking crashes python
In-Reply-To: <1485797426.48.0.513516354213.issue29392@psf.upfronthosting.co.za>
Message-ID: <1485819632.89.0.679929249173.issue29392@psf.upfronthosting.co.za>


Eryk Sun added the comment:

The old CRT doesn't do any parameter validation on the nbytes parameter. It just passes it directly to Windows LockFile as follows:

    LockFile((HANDLE)_get_osfhandle(fh), lockoffset, 0L, nbytes, 0L)

which is locking (DWORD)-1 bytes, i.e. 0xFFFFFFFF. This allows users to sneakily lock more than 2 GiB by passing a negative value. Python could do its own validation in 2.7 to raise an exception for negative values, but I think it's too late; that ship has sailed. 

The parameter is validated by the new CRT in 3.5+, which limits nbytes to non-negative values. There we need the _Py_BEGIN_SUPPRESS_IPH and _Py_END_SUPPRESS_IPH macros to handle the failed call without crashing the process.

----------
nosy: +eryksun

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29392>
_______________________________________

From report at bugs.python.org  Mon Jan 30 18:51:18 2017
From: report at bugs.python.org (Eryk Sun)
Date: Mon, 30 Jan 2017 23:51:18 +0000
Subject: [issue29392] msvcrt.locking crashes python
In-Reply-To: <1485797426.48.0.513516354213.issue29392@psf.upfronthosting.co.za>
Message-ID: <1485820278.02.0.823958043935.issue29392@psf.upfronthosting.co.za>


Changes by Eryk Sun <eryksun at gmail.com>:


----------
stage:  -> needs patch
versions: +Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29392>
_______________________________________

From report at bugs.python.org  Mon Jan 30 20:45:01 2017
From: report at bugs.python.org (Mariatta Wijaya)
Date: Tue, 31 Jan 2017 01:45:01 +0000
Subject: [issue29381] Tutorial documentation contains undefined reference to #!
In-Reply-To: <1485537602.74.0.811037724367.issue29381@psf.upfronthosting.co.za>
Message-ID: <1485827101.93.0.810237540249.issue29381@psf.upfronthosting.co.za>


Mariatta Wijaya added the comment:

Thanks all for the feedback. I updated the patch.

Marco, IMO it's not necessary to update the PEP.
This section of the documentation serves as a tutorial for it, and the update in this patch clarifies the situation.

----------
Added file: http://bugs.python.org/file46458/issue29381v3.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29381>
_______________________________________

From report at bugs.python.org  Mon Jan 30 20:51:50 2017
From: report at bugs.python.org (INADA Naoki)
Date: Tue, 31 Jan 2017 01:51:50 +0000
Subject: [issue11549] Build-out an AST optimizer,
 moving some functionality out of the peephole optimizer
In-Reply-To: <1300164402.79.0.766591131661.issue11549@psf.upfronthosting.co.za>
Message-ID: <1485827510.06.0.813974533687.issue11549@psf.upfronthosting.co.za>


INADA Naoki added the comment:

@haypo, how do you think about ast.Lit and ast.Constant?
Is this patch updated to use ast.Constant?
Or ast.Constant should be used only for some transform like constant folding?

----------
versions: +Python 3.7 -Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue11549>
_______________________________________

From report at bugs.python.org  Mon Jan 30 20:56:49 2017
From: report at bugs.python.org (Eric Lafontaine)
Date: Tue, 31 Jan 2017 01:56:49 +0000
Subject: [issue28879] smtplib send_message should add Date header if it is
 missing, per RFC5322
In-Reply-To: <1480952895.22.0.418354723664.issue28879@psf.upfronthosting.co.za>
Message-ID: <1485827809.37.0.191365167303.issue28879@psf.upfronthosting.co.za>


Eric Lafontaine added the comment:

Hi all,

Here you go :).

Regards,
Eric Lafontaine

----------
Added file: http://bugs.python.org/file46459/issue28879_v5.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28879>
_______________________________________

From report at bugs.python.org  Mon Jan 30 20:57:11 2017
From: report at bugs.python.org (Eric Lafontaine)
Date: Tue, 31 Jan 2017 01:57:11 +0000
Subject: [issue28879] smtplib send_message should add Date header if it is
 missing, per RFC5322
In-Reply-To: <1480952895.22.0.418354723664.issue28879@psf.upfronthosting.co.za>
Message-ID: <1485827831.2.0.481548160514.issue28879@psf.upfronthosting.co.za>


Eric Lafontaine added the comment:

Resent-heuristic

----------
Added file: http://bugs.python.org/file46460/Resent_heuristic.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28879>
_______________________________________

From report at bugs.python.org  Mon Jan 30 21:44:14 2017
From: report at bugs.python.org (Cooper Lees)
Date: Tue, 31 Jan 2017 02:44:14 +0000
Subject: [issue29393] Enabiling IPv6 by default
Message-ID: <1485830654.03.0.638312196963.issue29393@psf.upfronthosting.co.za>


New submission from Cooper Lees:

Can we discuss the possibility of enabling IPv6 by default now during build. I think enough of the world now NEED IPv6 so people should all be building their Python with it.

----------
components: Build
messages: 286506
nosy: cooperlees
priority: normal
severity: normal
status: open
title: Enabiling IPv6 by default
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29393>
_______________________________________

From report at bugs.python.org  Mon Jan 30 22:20:06 2017
From: report at bugs.python.org (Eryk Sun)
Date: Tue, 31 Jan 2017 03:20:06 +0000
Subject: [issue28164] _PyIO_get_console_type fails for various paths
In-Reply-To: <1473911463.84.0.00788793507165.issue28164@psf.upfronthosting.co.za>
Message-ID: <1485832806.7.0.480980268396.issue28164@psf.upfronthosting.co.za>


Eryk Sun added the comment:

I had reopened this issue with a suggestion for expanding the supported paths in msg276864, but it's languished for a while now. I've attached a patch implementing the change to _PyIO_get_console_type that I had suggested. Here are some example paths that this change enables (tested in Windows 10; need to check Windows 7):

    \\.\conin$
    \\.\conout$
    \\.\con
    //?/con
    C:/Temp/con
    C:/Temp/conout$
    C:/Temp/conin$

Paths such as "C:/Temp/con" are recognized by first converting to a device path such as r"\\.\con". It's fine if the directory doesn't exist because CreateFile will fail anyway in that case.

This patch also fixes an error in handling the return value of _get_osfhandle when it fails. 

It also adds a couple checks to __init__ to provide a better error message when io._WindowsConsoleIO is called directly for some reason instead of via open(). If the console type can't be determined it should error out immediately instead of printing an unrelated error about not being able to open the input/output buffer.

----------
keywords: +patch
Added file: http://bugs.python.org/file46461/issue_28164_01.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28164>
_______________________________________

From report at bugs.python.org  Mon Jan 30 22:22:15 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Tue, 31 Jan 2017 03:22:15 +0000
Subject: [issue29381] Tutorial documentation contains undefined reference to #!
In-Reply-To: <1485537602.74.0.811037724367.issue29381@psf.upfronthosting.co.za>
Message-ID: <1485832935.22.0.583137169133.issue29381@psf.upfronthosting.co.za>


Raymond Hettinger added the comment:

It looks like everyone is happy and the patch is ready for Mariatta to apply.

The procedure is:
 $ hg pull -u
 $ hg diff                      # Verify it has only your intended patch
 $ hg update 3.6
 $ cd Doc
 $ make html
 $ open build/html/index.html   # Now review generated HTML
 $ hg commit -m "Issue #29381: Clarify ordering of UNIX shebang line as source encoding line"
 $ hg update default
 $ hg merge 3.6
 $ hg diff                      # Verify it has only your intended patch
 $ hg commit -m "merge"
 $ hg push

Then return to this issue, verify the bot has posted the commit.

Write a comment thanking the OP and reviewers.  Then mark the status as "closed" and resolution as "fixed".

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29381>
_______________________________________

From report at bugs.python.org  Mon Jan 30 22:23:42 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Tue, 31 Jan 2017 03:23:42 +0000
Subject: [issue29381] Tutorial documentation contains undefined reference to #!
In-Reply-To: <1485537602.74.0.811037724367.issue29381@psf.upfronthosting.co.za>
Message-ID: <1485833022.45.0.707673420291.issue29381@psf.upfronthosting.co.za>


Raymond Hettinger added the comment:

I would only post this to 2.7, 3.6 and 3.7.   Also, no Misc/NEWS entry is needed.

----------
versions:  -Python 3.4, Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29381>
_______________________________________

From report at bugs.python.org  Mon Jan 30 22:36:13 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Tue, 31 Jan 2017 03:36:13 +0000
Subject: [issue27200] make doctest in CPython has failures
In-Reply-To: <1464988408.0.0.840317371896.issue27200@psf.upfronthosting.co.za>
Message-ID: <1485833773.74.0.362212574667.issue27200@psf.upfronthosting.co.za>


Raymond Hettinger added the comment:

Mariatta, would you please add your review comments to the proposed patches.  For the most part, we want the patches to make the minimal changes necessary to pass the doctests.  Also, clarity of the examples is more important than making the examples doctestable (i.e. perhaps added sorted() to the output makes it more predictable but it might also obfuscate the point being made).

In general, we would like to have the convenience of doc testing but should be unwilling to commit any atrocities along the way.  Ideally, the examples should be written in a way that doesn't garbage them up with <blankline> and whatnot (that is generally distracting to the reader and only serves doctest).  Similarly, make sure that clarity and cut-and-pastability don't get lost by adding ... PS2 prompts that uglify the examples.  In all cases, the intent of the example writer and the clarity of the example takes precedence over the doctests.

You should try out the patches and make sure the tests run, build the docs and make sure the examples are clean looking from a user point of view. If you find any issues or have questions, use rietveld for discussion (that is the "review" link to the right of the patch).

----------
assignee:  -> Mariatta
nosy: +Mariatta, rhettinger

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue27200>
_______________________________________

From report at bugs.python.org  Mon Jan 30 22:37:10 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Tue, 31 Jan 2017 03:37:10 +0000
Subject: [issue27200] make doctest in CPython has failures
In-Reply-To: <1464988408.0.0.840317371896.issue27200@psf.upfronthosting.co.za>
Message-ID: <1485833830.1.0.797844776671.issue27200@psf.upfronthosting.co.za>


Raymond Hettinger added the comment:

Also, please make this 3.7 only.  There is no point in going back in time unless an example is found that is actually broken.

----------
priority: normal -> low
versions:  -Python 2.7, Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue27200>
_______________________________________

From report at bugs.python.org  Mon Jan 30 22:41:57 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Tue, 31 Jan 2017 03:41:57 +0000
Subject: [issue29390] Python Tutorial should introduce iterators and generators
In-Reply-To: <1485770709.65.0.205484403801.issue29390@psf.upfronthosting.co.za>
Message-ID: <1485834117.67.0.643637751939.issue29390@psf.upfronthosting.co.za>


Raymond Hettinger added the comment:

IMO, the iterators are introduced at the correct point.  

There are many topics that one could argue should be introduced early, but of course they can't all go earlier (classes should be introduced earlier because everything in python is an object; namespaces should be introduced earlier because they are core concept that affects everything from function locals, to closures, classes, instances, and modules).

I concur with Jim that dropping more terminology earlier in the tutorial doesn't serve the user well.

----------
nosy: +rhettinger
resolution:  -> rejected
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29390>
_______________________________________

From report at bugs.python.org  Mon Jan 30 22:43:52 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Tue, 31 Jan 2017 03:43:52 +0000
Subject: [issue15373] copy.copy() does not properly copy os.environment
In-Reply-To: <1342454937.95.0.386630033182.issue15373@psf.upfronthosting.co.za>
Message-ID: <1485834232.4.0.980811655493.issue15373@psf.upfronthosting.co.za>


Raymond Hettinger added the comment:

That patch would be fine except that I really agree with Antoine that an error should be raised.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue15373>
_______________________________________

From report at bugs.python.org  Mon Jan 30 22:45:05 2017
From: report at bugs.python.org (Zachary Ware)
Date: Tue, 31 Jan 2017 03:45:05 +0000
Subject: [issue27200] make doctest in CPython has failures
In-Reply-To: <1485833773.74.0.362212574667.issue27200@psf.upfronthosting.co.za>
Message-ID: <CAKJDb-MSiBd1bmi_5OjS5VRODeaGULVOgkbwA0KwGCqBJtHQ8g@mail.gmail.com>


Zachary Ware added the comment:

Raymond Hettinger wrote:
> Similarly, make sure that clarity and cut-and-pastability don't get lost by adding ... PS2 prompts that uglify the examples.

Note that Sphinx adds a ">>>" button to the upper right of the example
that, when clicked, makes all of the prompts and output disappear so
that the code (and only the code) can be copied easily.  Not having
the ps2 prompts breaks that button.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue27200>
_______________________________________

From report at bugs.python.org  Mon Jan 30 23:01:30 2017
From: report at bugs.python.org (Raymond Hettinger)
Date: Tue, 31 Jan 2017 04:01:30 +0000
Subject: [issue18842] Add float.is_finite is_nan is_infinite to match Decimal
 methods
In-Reply-To: <1377532757.24.0.123652540945.issue18842@psf.upfronthosting.co.za>
Message-ID: <1485835290.66.0.491475779039.issue18842@psf.upfronthosting.co.za>


Raymond Hettinger added the comment:

Mariatta, please review the rm-finite patch.  We're already agreed that the removal makes sense.  What you're checking for is whether patch is complete (that there aren't any dangling references to the removed code and that it doesn't remove too much).  Also, try out the patch and run the test suite in in a debug mode.  

Whether you find any issues or are ready to bless the patch, make a comment in the tracker and assign back to Martin (the core dev who posts the patch is usually the one to apply it).  After merging, he will mark the class as closed/rejected.

----------
assignee:  -> Mariatta
nosy: +Mariatta

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue18842>
_______________________________________

From report at bugs.python.org  Mon Jan 30 23:57:54 2017
From: report at bugs.python.org (Steve Dower)
Date: Tue, 31 Jan 2017 04:57:54 +0000
Subject: [issue28164] _PyIO_get_console_type fails for various paths
In-Reply-To: <1473911463.84.0.00788793507165.issue28164@psf.upfronthosting.co.za>
Message-ID: <1485838674.4.0.88056343219.issue28164@psf.upfronthosting.co.za>


Steve Dower added the comment:

I'm okay with this patch. We move closer to being right without degrading the normal case, and I don't think the edge cases are important (and the behavior in those cases will be acceptable).

With a couple of tests to make sure the path comparisons don't get broken in the future, this will be ready to merge.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28164>
_______________________________________

From report at bugs.python.org  Tue Jan 31 02:19:43 2017
From: report at bugs.python.org (Maximilian Blochberger)
Date: Tue, 31 Jan 2017 07:19:43 +0000
Subject: [issue29394] Cannot tunnel TLS connection through TLS connection
Message-ID: <1485847183.69.0.499542154053.issue29394@psf.upfronthosting.co.za>


New submission from Maximilian Blochberger:

I have the following scenario: Client ? Proxy ? Target.

The following two scenarios are working perfectly fine:

1) Establishing a TLS-secured connection to the proxy and then tunnel traffic through that connection to the target. This results in the proxy being able to observe and manipulate the traffic in both directions. It protects against an adversary who has no control over the proxy, e.g. it prevents observers from learning that you are using a proxy (if the IP/port is not known) and from reading the actual traffic.

2) Establish a non-secured connection to the proxy and then tunnel TLS-secured traffic through that connection to the target. That prevents the proxy from being able to observe or manipulate the traffic. Although an observer could learn that you are using a proxy and what target you are connecting to.

Now what I tried was to establish a TLS-secured connection to the proxy and then to establish a TLS-secured tunnel to the target, effectively resulting in two layers of TLS in between the client and the proxy. This would protect from an observer learning that you are using a proxy and where you connect to (the proxy still knows) but preventing the proxy from observing and manipulating the actual traffic to the target.

This does not work in Python 3.6. The TLS-secured connection to the proxy is straight forward and can be easily done with ssl.SSLContext.wrap_socket(). The TCP connection between the proxy and the target can then be established by issuing an HTTP CONNECT request. The response can than be read without closing the connection as done in http.client.HTTPConnection._tunnel(). Now my idea was to call ssl.SSLContext.wrap_socket() again (with a different context for the target) and send traffic through that. Unfortunately the TLS handshake fails with the error message "unknown protocol". I looked into the actual traffic transmitted and realised that the handshake was performed in plain text ??so effectively stripping the TLS layer that was established already ? which results in the proxy server not knowing how to handle the traffic (as it is not TLS-secured) aborting the connection (and reporting a fatal TLS alert).

This leads to the conclusion that another call to ssl.SSLContext.wrap_socket() will override a previous call of the same function (different context object though). I think this is unexpected behaviour.

It might be easier to handle such scenarios if a tunnel would be a separate http.client.HTTP(S)Connection object, see issue #24964. This would also allow to handle ssl-specific calls such as ssl.SSLSocket.getpeercert() as each layer probably uses different certificates.

----------
assignee: christian.heimes
components: SSL
messages: 286517
nosy: Maximilian Blochberger, christian.heimes
priority: normal
severity: normal
status: open
title: Cannot tunnel TLS connection through TLS connection
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29394>
_______________________________________

From report at bugs.python.org  Tue Jan 31 04:03:23 2017
From: report at bugs.python.org (=?utf-8?q?Christian_W=C3=BCste?=)
Date: Tue, 31 Jan 2017 09:03:23 +0000
Subject: =?utf-8?q?=5Bissue29395=5D_Edit_with_IDLE_3=2E6_don=C2=B4t_work_if_Name_o?=
 =?utf-8?q?f_File_=22code=2Epy=22?=
Message-ID: <1485853403.69.0.0659896818047.issue29395@psf.upfronthosting.co.za>


New submission from Christian W?ste:

As soon as I specify a file "code.py", it can not be opened via the right click > Edit with IDLE 3.6 (64-Bit).
The content of the file is completely irrelevant.
All other Python files located in the same folder as "code.py" can not be opened.
If I rename the file in for example coder.py everything works.
Previously, Python version 3.5.3 was installed.
It is also possible that it is attached to my system.

----------
assignee: terry.reedy
components: IDLE
messages: 286518
nosy: terry.reedy, wueste87
priority: normal
severity: normal
status: open
title: Edit with IDLE 3.6 don?t work if Name of File "code.py"
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29395>
_______________________________________

From report at bugs.python.org  Tue Jan 31 04:23:43 2017
From: report at bugs.python.org (Christian Heimes)
Date: Tue, 31 Jan 2017 09:23:43 +0000
Subject: [issue29394] Cannot tunnel TLS connection through TLS connection
In-Reply-To: <1485847183.69.0.499542154053.issue29394@psf.upfronthosting.co.za>
Message-ID: <1485854623.69.0.151880430778.issue29394@psf.upfronthosting.co.za>


Christian Heimes added the comment:

You cannot use wrap_socket() to wrap a SSLSocket into another SSLSocket. Python uses SSL_set_fd() to wrap the socket's file descriptor. OpenSSL directly pulls and pushes data to the connection. Instead you have to use wrap_bio() and do I/O on your own.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29394>
_______________________________________

From report at bugs.python.org  Tue Jan 31 04:53:48 2017
From: report at bugs.python.org (Erik Bray)
Date: Tue, 31 Jan 2017 09:53:48 +0000
Subject: [issue21085] Cygwin does not provide siginfo_t.si_band
In-Reply-To: <1396019829.85.0.386935527155.issue21085@psf.upfronthosting.co.za>
Message-ID: <1485856428.79.0.727485242491.issue21085@psf.upfronthosting.co.za>


Erik Bray added the comment:

Decorater, since this issue was already closed, could you open a new one?  And when you do, please add me to the nosy list.

I'm still planning to get a Cygwin built bot up for Python, I've just had other various priorities.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue21085>
_______________________________________

From report at bugs.python.org  Tue Jan 31 05:53:07 2017
From: report at bugs.python.org (Silvio Ricardo Cordeiro)
Date: Tue, 31 Jan 2017 10:53:07 +0000
Subject: [issue29396] Re-opening /dev/tty breaks readline
Message-ID: <1485859987.68.0.026952717617.issue29396@psf.upfronthosting.co.za>


New submission from Silvio Ricardo Cordeiro:

The following code works on python2 (tested with 2.7.6), but breaks readline for python3 (tested with 3.4.3):

>>> import sys, readline
>>> sys.stdin = open('/dev/tty', 'r')
>>> input("> ")  # press "7<Enter>"
>>> input("> ")  # press "up" key

Re-opening /dev/tty as stdin is a very useful technique. For example, one could debug a script that takes a file as stdin by re-opening the stdin as /dev/tty and using pdb.set_trace().

See bug report for python2 in 2002 here: https://bugs.python.org/issue512981

----------
components: Library (Lib)
messages: 286521
nosy: silvioricardoc
priority: normal
severity: normal
status: open
title: Re-opening /dev/tty breaks readline
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29396>
_______________________________________

From report at bugs.python.org  Tue Jan 31 07:23:23 2017
From: report at bugs.python.org (Eryk Sun)
Date: Tue, 31 Jan 2017 12:23:23 +0000
Subject: [issue28164] _PyIO_get_console_type fails for various paths
In-Reply-To: <1473911463.84.0.00788793507165.issue28164@psf.upfronthosting.co.za>
Message-ID: <1485865403.46.0.462097289519.issue28164@psf.upfronthosting.co.za>


Eryk Sun added the comment:

I added some tests to ensure open() returns an instance of _WindowsConsoleIO for a few console paths. It also checks that opening a non-console file raises a ValueError, both for a path and an fd, as well for a negative file descriptor.

----------
Added file: http://bugs.python.org/file46462/issue_28164_02.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28164>
_______________________________________

From report at bugs.python.org  Tue Jan 31 07:28:52 2017
From: report at bugs.python.org (Eryk Sun)
Date: Tue, 31 Jan 2017 12:28:52 +0000
Subject: [issue28164] _PyIO_get_console_type fails for various paths
In-Reply-To: <1473911463.84.0.00788793507165.issue28164@psf.upfronthosting.co.za>
Message-ID: <1485865732.19.0.296350191096.issue28164@psf.upfronthosting.co.za>


Changes by Eryk Sun <eryksun at gmail.com>:


Removed file: http://bugs.python.org/file46462/issue_28164_02.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28164>
_______________________________________

From report at bugs.python.org  Tue Jan 31 07:29:50 2017
From: report at bugs.python.org (Eryk Sun)
Date: Tue, 31 Jan 2017 12:29:50 +0000
Subject: [issue28164] _PyIO_get_console_type fails for various paths
In-Reply-To: <1473911463.84.0.00788793507165.issue28164@psf.upfronthosting.co.za>
Message-ID: <1485865790.86.0.0231100176648.issue28164@psf.upfronthosting.co.za>


Changes by Eryk Sun <eryksun at gmail.com>:


Added file: http://bugs.python.org/file46463/issue_28164_02.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28164>
_______________________________________

From report at bugs.python.org  Tue Jan 31 07:30:07 2017
From: report at bugs.python.org (Matthias Klose)
Date: Tue, 31 Jan 2017 12:30:07 +0000
Subject: [issue29169] update zlib to 1.2.11
In-Reply-To: <1483629107.49.0.982374684244.issue29169@psf.upfronthosting.co.za>
Message-ID: <1485865807.71.0.814376119468.issue29169@psf.upfronthosting.co.za>


Matthias Klose added the comment:

plus the update to 1.2.11

----------
title: update zlib to 1.2.10 -> update zlib to 1.2.11
Added file: http://bugs.python.org/file46464/zlib-1.2.11.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29169>
_______________________________________

From report at bugs.python.org  Tue Jan 31 07:31:08 2017
From: report at bugs.python.org (Roundup Robot)
Date: Tue, 31 Jan 2017 12:31:08 +0000
Subject: [issue29169] update zlib to 1.2.11
In-Reply-To: <1483629107.49.0.982374684244.issue29169@psf.upfronthosting.co.za>
Message-ID: <20170131123105.85973.61137.A476E829@psf.io>


Roundup Robot added the comment:

New changeset 0136c99a9795 by doko in branch '2.7':
- Issue #29169: Update zlib to 1.2.11.
https://hg.python.org/cpython/rev/0136c99a9795

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29169>
_______________________________________

From report at bugs.python.org  Tue Jan 31 07:38:15 2017
From: report at bugs.python.org (Ma Lin)
Date: Tue, 31 Jan 2017 12:38:15 +0000
Subject: [issue28518] execute("begin immediate") throwing OperationalError
In-Reply-To: <1477307118.31.0.70785963389.issue28518@psf.upfronthosting.co.za>
Message-ID: <1485866295.69.0.658152607731.issue28518@psf.upfronthosting.co.za>


Ma Lin added the comment:

Let me give a summary.

Before 3.6.0 (before 284676cf2ac8):
implicit begin: st in (INSERT, UPDATE, DELETE, REPLACE)
implicit commit: st not in (SELECT, INSERT, UPDATE, DELETE, REPLACE)

In 3.6.0 (after 284676cf2ac8):
implicit begin: (not sqlite3_stmt_readonly(st))  and  (not in (CREATE, DROP, REINDEX))
implicit commit: no

With Palivoda's sqlite-ddl-dml-3.patch:
implicit begin: st in (INSERT, UPDATE, DELETE, REPLACE)
implicit commit: no
~~~~~~~~~~~~~~~~~~~~~~~~~~~

Some possible subsequent steps base on this issue:

1, remove sqlite3_stmt_readonly(), see issue29355.
Base on sqlite-ddl-dml-3.patch, only change a single line of code:
+ if (self->is_dml) {
- if (!sqlite3_stmt_readonly(self->statement->st)) {
Then restore the behavior before 3.6.0, this is even better than using sqlite3_stmt_readonly().

2, enhance backward compatibility.
Please read msg284585, msg286217, msg286293 in a row. 
If we implicitly commit before executing some statements (and print a warning), we can improve compatibility largely, the cost is almost free.

3, docments enhancement.
see issue8145, issue29121.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28518>
_______________________________________

From report at bugs.python.org  Tue Jan 31 07:51:47 2017
From: report at bugs.python.org (INADA Naoki)
Date: Tue, 31 Jan 2017 12:51:47 +0000
Subject: [issue15373] copy.copy() does not properly copy os.environment
In-Reply-To: <1342454937.95.0.386630033182.issue15373@psf.upfronthosting.co.za>
Message-ID: <1485867107.46.0.670012426135.issue15373@psf.upfronthosting.co.za>


INADA Naoki added the comment:

I agree.
os.environ is not dict, it's a proxy of "current" environment which
is not copyable.
So I agree to copy.copy() shouldn't copy os.environ implicitly.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue15373>
_______________________________________

From report at bugs.python.org  Tue Jan 31 07:53:52 2017
From: report at bugs.python.org (Roundup Robot)
Date: Tue, 31 Jan 2017 12:53:52 +0000
Subject: [issue29169] update zlib to 1.2.11
In-Reply-To: <1483629107.49.0.982374684244.issue29169@psf.upfronthosting.co.za>
Message-ID: <20170131125349.99465.11244.947527A0@psf.io>


Roundup Robot added the comment:

New changeset c8c1f08428cb by doko in branch '3.5':
- Issue #29169: Update zlib to 1.2.10.
https://hg.python.org/cpython/rev/c8c1f08428cb

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29169>
_______________________________________

From report at bugs.python.org  Tue Jan 31 07:53:54 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 31 Jan 2017 12:53:54 +0000
Subject: [issue11549] Build-out an AST optimizer,
 moving some functionality out of the peephole optimizer
In-Reply-To: <1300164402.79.0.766591131661.issue11549@psf.upfronthosting.co.za>
Message-ID: <1485867234.11.0.310269083958.issue11549@psf.upfronthosting.co.za>


STINNER Victor added the comment:

> @haypo, how do you think about ast.Lit and ast.Constant?

I already made two changes in Python 3.6 :-)

I added ast.Constant to Python 3.6. While it's not used by .py=>AST compiler, it is understood by the AST->bytecode compiler (ex: handle correctly special cases like docstrings).
https://docs.python.org/dev/whatsnew/3.6.html#ast
=> issue #26146

Moreover, I also modified the format of the co_lnotab structure to support negative line number delta. So a code transformer can move instructions and preserve the Python traceback feature.
https://docs.python.org/dev/whatsnew/3.6.html#changes-in-the-python-api
=> see the PEP 511

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue11549>
_______________________________________

From report at bugs.python.org  Tue Jan 31 07:57:27 2017
From: report at bugs.python.org (Matthias Klose)
Date: Tue, 31 Jan 2017 12:57:27 +0000
Subject: [issue29169] update zlib to 1.2.11
In-Reply-To: <1483629107.49.0.982374684244.issue29169@psf.upfronthosting.co.za>
Message-ID: <1485867447.3.0.797573621893.issue29169@psf.upfronthosting.co.za>


Matthias Klose added the comment:

now updated all active branches to 1.2.11

----------
resolution:  -> fixed
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29169>
_______________________________________

From report at bugs.python.org  Tue Jan 31 08:33:14 2017
From: report at bugs.python.org (Hugo Geoffroy)
Date: Tue, 31 Jan 2017 13:33:14 +0000
Subject: [issue11549] Build-out an AST optimizer,
 moving some functionality out of the peephole optimizer
In-Reply-To: <1300164402.79.0.766591131661.issue11549@psf.upfronthosting.co.za>
Message-ID: <1485869594.0.0.567361563353.issue11549@psf.upfronthosting.co.za>


Hugo Geoffroy added the comment:

I would like to point out that the changes in `ast.literal_eval` may have some security risk for code that do not expect this function to return an object with user-controlled length (for example, with `2**32*'X'`). AFAIK, this is not possible with the current version of `literal_eval`.

At least [this library](https://pypi.python.org/pypi/serpent) would have a serious risk of remote DoS :

> Because it only serializes literals and recreates the objects using ast.literal_eval(), the serialized data is safe to transport to other machines (over the network for instance) and de-serialize it there.

Sorry for the noise if this is a useless/incorrect consideration.

----------
nosy: +pstch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue11549>
_______________________________________

From report at bugs.python.org  Tue Jan 31 09:05:11 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 31 Jan 2017 14:05:11 +0000
Subject: [issue11549] Build-out an AST optimizer,
 moving some functionality out of the peephole optimizer
In-Reply-To: <1485869594.0.0.567361563353.issue11549@psf.upfronthosting.co.za>
Message-ID: <CAMpsgwaz67bKXkysG_Q-8xZaxNrnLC+DDj_pmUCckwV2EAJz3g@mail.gmail.com>


STINNER Victor added the comment:

Hugo Geoffroy added the comment:
> I would like to point out that the changes in `ast.literal_eval` may have some security risk for code that do not expect this function to return an object with user-controlled length (for example, with `2**32*'X'`). AFAIK, this is not possible with the current version of `literal_eval`.

Since the Python compiler doesn't produce ast.Constant, there is no
change in practice in ast.literal_eval(). If you found a bug, please
open a new issue.

> At least [this library](https://pypi.python.org/pypi/serpent) would have a serious risk of remote DoS :

I tried hard to implement a sandbox in Python and I failed:
https://lwn.net/Articles/574215/

I don't think that literal_eval() is safe *by design*.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue11549>
_______________________________________

From report at bugs.python.org  Tue Jan 31 09:19:58 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Tue, 31 Jan 2017 14:19:58 +0000
Subject: [issue11549] Build-out an AST optimizer,
 moving some functionality out of the peephole optimizer
In-Reply-To: <1300164402.79.0.766591131661.issue11549@psf.upfronthosting.co.za>
Message-ID: <1485872398.73.0.965051582759.issue11549@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

Good point Hugo. Yes, this should be taken into account when move constant folding to AST level. Thank you for the reminder.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue11549>
_______________________________________

From report at bugs.python.org  Tue Jan 31 09:21:40 2017
From: report at bugs.python.org (zhaw_munt)
Date: Tue, 31 Jan 2017 14:21:40 +0000
Subject: [issue29397] linux/random.h present but cannot be compiled
Message-ID: <1485872500.79.0.0659257887821.issue29397@psf.upfronthosting.co.za>


New submission from zhaw_munt:

configure: WARNING: linux/random.h: present but cannot be compiled
configure: WARNING: linux/random.h:     check for missing prerequisite headers?
configure: WARNING: linux/random.h: see the Autoconf documentation
configure: WARNING: linux/random.h:     section "Present But Cannot Be Compiled"
configure: WARNING: linux/random.h: proceeding with the compiler's result
configure: WARNING:     ## --------------------------------------- ##
configure: WARNING:     ## Report this to https://bugs.python.org/ ##
configure: WARNING:     ## --------------------------------------- ##

2.6.18-194.el5 #1 SMP Fri Apr 2 14:58:14 EDT 2010 x86_64 x86_64 x86_64 GNU/Linu (CentOS 5.5)

Buildings seems to work, at the end I have the binary but I've noticed that *at least* _socket is missing. Other libraries such as "_bz2" and "_sqlite" are there (as I've installed their corresponding devel package)

----------
messages: 286533
nosy: zhaw_munt
priority: normal
severity: normal
status: open
title: linux/random.h present but cannot be compiled
type: compile error
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29397>
_______________________________________

From report at bugs.python.org  Tue Jan 31 09:32:55 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Tue, 31 Jan 2017 14:32:55 +0000
Subject: [issue11549] Build-out an AST optimizer,
 moving some functionality out of the peephole optimizer
In-Reply-To: <1300164402.79.0.766591131661.issue11549@psf.upfronthosting.co.za>
Message-ID: <1485873175.69.0.317361553411.issue11549@psf.upfronthosting.co.za>


Serhiy Storchaka added the comment:

> Since the Python compiler doesn't produce ast.Constant, there is no
change in practice in ast.literal_eval(). If you found a bug, please
open a new issue.

Currently there is no a bug in ast.literal_eval() because the '**' operator is not accepted.

>>> ast.literal_eval("2**2**32")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/serhiy/py/cpython/Lib/ast.py", line 85, in literal_eval
    return _convert(node_or_string)
  File "/home/serhiy/py/cpython/Lib/ast.py", line 84, in _convert
    raise ValueError('malformed node or string: ' + repr(node))
ValueError: malformed node or string: <_ast.BinOp object at 0xb6f2fa4c>

But if move the optimization to AST level this can add a vulnerability to DOS attack. The optimizer should do additional checks first than execute operators that can return too large value or take too much CPU time. Currently this vulnerability have place in the peephole optimizer.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue11549>
_______________________________________

From report at bugs.python.org  Tue Jan 31 09:35:03 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 31 Jan 2017 14:35:03 +0000
Subject: [issue11549] Build-out an AST optimizer,
 moving some functionality out of the peephole optimizer
In-Reply-To: <1300164402.79.0.766591131661.issue11549@psf.upfronthosting.co.za>
Message-ID: <1485873303.5.0.884281588997.issue11549@psf.upfronthosting.co.za>


STINNER Victor added the comment:

> Currently there is no a bug in ast.literal_eval() because the '**' operator is not accepted.

The doc says "This can be used for safely evaluating strings containing Python values from untrusted sources without the need to parse the values oneself. It is not capable of evaluating arbitrarily complex expressions, for example involving operators or indexing."
https://docs.python.org/dev/library/ast.html#ast.literal_eval

I don't think that it's a bug, but a deliberate design choice. a**b is an obvious trick to DoS a server (high CPU and memory usage).

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue11549>
_______________________________________

From report at bugs.python.org  Tue Jan 31 09:56:34 2017
From: report at bugs.python.org (zeroinside)
Date: Tue, 31 Jan 2017 14:56:34 +0000
Subject: [issue29398] memory corruption in xxlimited
Message-ID: <1485874593.96.0.823310722961.issue29398@psf.upfronthosting.co.za>


New submission from zeroinside:

Hello
I found incorrect gc behavior in xxlimited module.
After an hour of investigation, I'm still not sure its security related problem.
I have a partial control on RBP register, depends of memory layout.



GDB:
Starting program: /usr/bin/python3.6 
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/usr/lib/libthread_db.so.1".
Python 3.6.0 (default, Jan 16 2017, 12:12:55) 
[GCC 6.3.1 20170109] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import xxlimited
>>> a=xxlimited.new()
>>> del a

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff73d612d in PyArena_Free () from /usr/lib/libpython3.6m.so.1.0
(gdb) info reg
rax            0x7ffff7457270   140737341911664
rbx            0x7ffff7f812b8   140737353618104
rcx            0x62aa00 6466048
rdx            0x7ffff7457270   140737341911664
rsi            0x1      1
rdi            0x7ffff7f81300   140737353618176
rbp            0x500000a29      0x500000a29
rsp            0x7fffffffe210   0x7fffffffe210
r8             0x7ffff7f81000   140737353617408
r9             0x1c     28
r10            0x1b     27
r11            0x12300  74496
r12            0x7ffff7f812b8   140737353618104
r13            0x6fafd0 7319504
r14            0x7ffff3e7b570   140737285436784
r15            0x7ffff3e7b5a0   140737285436832
rip            0x7ffff73d612d   0x7ffff73d612d <PyArena_Free+29>
eflags         0x10202  [ IF RF ]
cs             0x33     51
ss             0x2b     43
ds             0x0      0
es             0x0      0
fs             0x0      0
gs             0x0      0

root@/home/zeroinside/python/pyburst3.6/crash $ /usr/local/bin/python3.6 xlimited_poc.py 
ASAN:DEADLYSIGNAL
=================================================================
==5082==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000008 (pc 0x000000557469 bp 0x0fe0603ea23d sp 0x7fff76e98c20 T0)
==5082==The signal is caused by a WRITE memory access.
==5082==Hint: address points to the zero page.
    #0 0x557468 in PyObject_GC_UnTrack /home/zeroinside/Downloads/Python-3.6.0/Modules/gcmodule.c:1699:9
    #1 0x66d0af in subtype_dealloc /home/zeroinside/Downloads/Python-3.6.0/Objects/typeobject.c:1133:5
    #2 0x61e557 in _PyDict_DelItem_KnownHash /home/zeroinside/Downloads/Python-3.6.0/Objects/dictobject.c:1641:5
    #3 0x7970c0 in _PyEval_EvalFrameDefault /home/zeroinside/Downloads/Python-3.6.0/Python/ceval.c:2187:19
    #4 0x7aef44 in PyEval_EvalFrameEx /home/zeroinside/Downloads/Python-3.6.0/Python/ceval.c:718:12
    #5 0x7aef44 in _PyEval_EvalCodeWithName /home/zeroinside/Downloads/Python-3.6.0/Python/ceval.c:4119
    #6 0x79571c in PyEval_EvalCodeEx /home/zeroinside/Downloads/Python-3.6.0/Python/ceval.c:4140:12
    #7 0x79571c in PyEval_EvalCode /home/zeroinside/Downloads/Python-3.6.0/Python/ceval.c:695
    #8 0x5295e7 in run_mod /home/zeroinside/Downloads/Python-3.6.0/Python/pythonrun.c:980:9
    #9 0x5295e7 in PyRun_FileExFlags /home/zeroinside/Downloads/Python-3.6.0/Python/pythonrun.c:933
    #10 0x527e75 in PyRun_SimpleFileExFlags /home/zeroinside/Downloads/Python-3.6.0/Python/pythonrun.c:396:13
    #11 0x55340c in run_file /home/zeroinside/Downloads/Python-3.6.0/Modules/main.c:320:11
    #12 0x55340c in Py_Main /home/zeroinside/Downloads/Python-3.6.0/Modules/main.c:780
    #13 0x519776 in main /home/zeroinside/Downloads/Python-3.6.0/./Programs/python.c:69:11
    #14 0x7f0300e01290 in __libc_start_main (/usr/lib/libc.so.6+0x20290)
    #15 0x432179 in _start (/usr/local/bin/python3.6+0x432179)

AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV /home/zeroinside/Downloads/Python-3.6.0/Modules/gcmodule.c:1699:9 in PyObject_GC_UnTrack
==5082==ABORTING

----------
components: Build
files: xlimited.py
messages: 286536
nosy: zeroinside
priority: normal
severity: normal
status: open
title: memory corruption in xxlimited
type: security
versions: Python 3.6
Added file: http://bugs.python.org/file46465/xlimited.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29398>
_______________________________________

From report at bugs.python.org  Tue Jan 31 10:09:51 2017
From: report at bugs.python.org (Marco Buttu)
Date: Tue, 31 Jan 2017 15:09:51 +0000
Subject: [issue29387] Tabs vs spaces FAQ out of date
In-Reply-To: <1485734038.81.0.0566253928841.issue29387@psf.upfronthosting.co.za>
Message-ID: <1485875391.91.0.805629095667.issue29387@psf.upfronthosting.co.za>


Marco Buttu added the comment:

Hi Martin, why did you write "Python should report an error if mixed tabs and spaces are causing problems in leading whitespace."? 
Python for sure reports an error in that case. 

Maybe "Python reports an error if..." is a better choice. But my English is really poor, so I can be wrong... I have just this doubt, otherwise the patch is OK to me.

----------
nosy: +marco.buttu

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29387>
_______________________________________

From report at bugs.python.org  Tue Jan 31 10:25:03 2017
From: report at bugs.python.org (Ned Deily)
Date: Tue, 31 Jan 2017 15:25:03 +0000
Subject: [issue29393] Enabiling IPv6 by default
In-Reply-To: <1485830654.03.0.638312196963.issue29393@psf.upfronthosting.co.za>
Message-ID: <1485876303.13.0.382771061915.issue29393@psf.upfronthosting.co.za>


Ned Deily added the comment:

Checking for IPv6 *is* enabled by default and has been for quite some time:

$ ./configure
[...]
checking if --enable-ipv6 is specified... yes
checking if RFC2553 API is available... yes
checking ipv6 stack type... kame
[...]

----------
nosy: +ned.deily
resolution:  -> out of date
stage:  -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29393>
_______________________________________

From report at bugs.python.org  Tue Jan 31 10:59:23 2017
From: report at bugs.python.org (Nick Coghlan)
Date: Tue, 31 Jan 2017 15:59:23 +0000
Subject: [issue11549] Build-out an AST optimizer,
 moving some functionality out of the peephole optimizer
In-Reply-To: <1300164402.79.0.766591131661.issue11549@psf.upfronthosting.co.za>
Message-ID: <1485878363.46.0.461964364219.issue11549@psf.upfronthosting.co.za>


Nick Coghlan added the comment:

Hugo, Serhiy, and Victor: I think you're all agreeing with each other, but to make sure I'm understanding the point correctly:

1. ast.literal_eval() is currently safe from malicious code like "100000 ** 100000" or "1073741824 * 'a'" because it only traverses addition and subtraction nodes, so any such operations will just throw ValueError (As a point of interest: unary plus and minus are required to support positive and negative numeric literals, while binary addition and subtraction are required to support complex number literals. So the status quo isn't precisely the result of a conscious security decision, it's just a minimalist implementation of exactly what's necessary to support all of the builtin types, which also provides some highly desirable security properties when evaluating untrusted code)


2. an eager constant folding optimisation in the AST tier would happen *before* literal_eval filtered out the multiplication and exponentiation nodes, and hence would make literal_eval vulnerable to remote DOS attacks in cases where it is expected to be safe

However, that's not exactly how this patch works: if you pass "PyCF_ONLY_AST" as ast.parse does, it *doesn't* run the constant-folding step. Instead, the constant folding is run as an AST-to-AST transform during the AST-to-bytecode compilation step, *not* the initial source-to-AST step. (see http://bugs.python.org/issue11549#msg132361 )

This has a few nice properties:

- ast.literal_eval() remains safe
- source -> AST -> source transformation pipelines continue to preserve the original code structure
- externally generated AST structures still benefit from the AST optimisation pass
- we don't need a new flag to turn this optimisation pass off when generating the AST for a given piece of source code

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue11549>
_______________________________________

From report at bugs.python.org  Tue Jan 31 11:12:40 2017
From: report at bugs.python.org (Christopher Schramm)
Date: Tue, 31 Jan 2017 16:12:40 +0000
Subject: [issue27400] Datetime NoneType after calling Py_Finalize and
 Py_Initialize
In-Reply-To: <1467035358.55.0.678180807375.issue27400@psf.upfronthosting.co.za>
Message-ID: <1485879160.25.0.561464383269.issue27400@psf.upfronthosting.co.za>


Changes by Christopher Schramm <python at cschramm.eu>:


----------
nosy: +cschramm

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue27400>
_______________________________________

From report at bugs.python.org  Tue Jan 31 11:15:23 2017
From: report at bugs.python.org (Ammar Askar)
Date: Tue, 31 Jan 2017 16:15:23 +0000
Subject: [issue29329] Incorrect documentation for custom `hex()` support on
 Python 2
In-Reply-To: <1484852138.82.0.311515909069.issue29329@psf.upfronthosting.co.za>
Message-ID: <1485879323.11.0.32201853538.issue29329@psf.upfronthosting.co.za>


Ammar Askar added the comment:

Attached patch changes the py2.7 documentation to point out that the method name is __hex__ and its return type is a string.

----------
keywords: +patch
nosy: +ammar2
Added file: http://bugs.python.org/file46466/hex_method.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29329>
_______________________________________

From report at bugs.python.org  Tue Jan 31 11:49:39 2017
From: report at bugs.python.org (Mark Haase)
Date: Tue, 31 Jan 2017 16:49:39 +0000
Subject: =?utf-8?q?=5Bissue24875=5D_pyvenv_doesn=C2=B4t_install_PIP_inside_a_new_v?=
 =?utf-8?q?env_with_--system-site-package?=
In-Reply-To: <1439699040.07.0.199956094974.issue24875@psf.upfronthosting.co.za>
Message-ID: <1485881379.33.0.566699945472.issue24875@psf.upfronthosting.co.za>


Mark Haase added the comment:

Vinay, thanks for the patch! I just tested it on my Ubuntu 16.04.1 box with Python 3.5.2 and it works perfectly. Your patch is cleaner than mine, too.

Any idea how we get a CPython dev to look at this patch?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24875>
_______________________________________

From report at bugs.python.org  Tue Jan 31 12:04:48 2017
From: report at bugs.python.org (INADA Naoki)
Date: Tue, 31 Jan 2017 17:04:48 +0000
Subject: [issue11549] Build-out an AST optimizer,
 moving some functionality out of the peephole optimizer
In-Reply-To: <1300164402.79.0.766591131661.issue11549@psf.upfronthosting.co.za>
Message-ID: <1485882288.88.0.807759349986.issue11549@psf.upfronthosting.co.za>


INADA Naoki added the comment:

> 1. Changes to AST

I'm working on updating this part. There are some failing tests remains. 
But I doubt this stage is worth enough for now.

> a) Merge Num, Str, Bytes and Ellipsis constructors into a single Lit  (literal) that can hold anything. For one thing, this removes a good deal of copy-paste in the code, since these are always treated the same. (There were a couple of places where Bytes ctor was missing for no apparent reason, I think it was forgotten.) Otherwise, I would have to introduce at least 4 more node types: None, Bool, TupleConst, SetConst. This seemed excessive.

We have already Constant and NameConstant.  So it seems there are no need for
None, Bool, TupleConst, SetConst nodes.

I think converting Num, Str, Bytes, Ellipsis into Constant in folding stage
is easier than fixing all tests.


> b) Docstring is now an attribute of Module, FunctionDef and ClassDef, rather than a first statement. Docstring is a special syntactic construction, it's not an executable code, so it makes sense to separate it. Otherwise, optimizer would have to take extra care not to introduce, change or remove docstring.

Take docstring before constant folding isn't enough?
(I'm sorry if I'm wrong.  I haven't tried it yet.)


> c) 'None', 'True' and 'False' are parsed as literals instead of names, even without optimizations. Since they are not redefineable, I think it makes most sense to treat them as literals. This isn't strictly needed for folding, and I haven't removed all the artefacts, in case this turns out controversial.

They are all NameConstant already.

----------
Added file: http://bugs.python.org/file46467/change-ast.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue11549>
_______________________________________

From report at bugs.python.org  Tue Jan 31 12:08:31 2017
From: report at bugs.python.org (Vinay Sajip)
Date: Tue, 31 Jan 2017 17:08:31 +0000
Subject: =?utf-8?q?=5Bissue24875=5D_pyvenv_doesn=C2=B4t_install_PIP_inside_a_new_v?=
 =?utf-8?q?env_with_--system-site-package?=
In-Reply-To: <1439699040.07.0.199956094974.issue24875@psf.upfronthosting.co.za>
Message-ID: <1485882511.09.0.362303956814.issue24875@psf.upfronthosting.co.za>


Vinay Sajip added the comment:

> Any idea how we get a CPython dev to look at this patch?

Well, I'm one of them, so ... :-)

I will commit it soon, given that Thomas Nyberg has also given his feedback. It was just waiting for your comments ...

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24875>
_______________________________________

From report at bugs.python.org  Tue Jan 31 12:14:06 2017
From: report at bugs.python.org (Elli Pirelli)
Date: Tue, 31 Jan 2017 17:14:06 +0000
Subject: [issue29399] python3.dll export forwarders not resolved in all
 situations due to missing python3?.dll dependency and DLL search path
 behavior
Message-ID: <1485882846.24.0.14620001579.issue29399@psf.upfronthosting.co.za>


New submission from Elli Pirelli:

Let me first apologize if this issue has already been reported and discussed before. I would be surprised if there are not a few developers that had to deal with this issue before, but i was unable to find an existing bug report or discussion about this. So, here it goes...


Importing/using Py_* symbols from python3.dll instead directly from python3?.dll can be a problem when python3.dll and python3?.dll reside in a directory that is not part of the standard DLL search path. An example would be an application that tries loading Python from a sub directory of the application.

The functions exported in python3.dll are export forwarders referencing the respective functions in python3?.dll.
python3.dll itself has no import dependency on python3?.dll.

Without an import dependency on python3?.dll, this DLL will be loaded when one of the export forwarders in python3.dll will be resolved, for example when calling GetProcAddress("Py_Main").

However, in such a case an altered DLL search path will be completely ignored, and python3?.dll will only be looked for in the standard DLL search paths. Since the sub directory with the python DLLs is not part of the standard DLL search paths, the resolution of the export fowarder(s) fails.

Trying to alter the DLL search path has no effect. Loading python3.dll with LoadLibraryEx and the flag LOAD_WITH_ALTERED_SEARCH_PATH does not help. Neither does calling SetDllDirectory or AddDllDirectory before loading python3.dll and/or before invoking GetProcAddress.

Note that loading python3.dll itself works fine. However, resolving one of the exports with GetProcAddress will fail if python3?.dll cannot be found in the standard DLL search path.

I don't know, if there is a way to make it work properly...

As of now the only two options out of this dilemma are either to put the python DLLs (and assorted files) in the same directory as the application exe (ugly mess) or to load python3?.dll instead of python3.dll (which sort of defeats the purpose of having python3.dll, i guess).

My suggestion for future Python 3 versions is to let python3.dll have a DLL dependency on python3?.dll. Linking python3.dll to python3?.dll (i.e., loading python3.dll will cause python3?.dll being loaded) allows using altered DLL search paths.

----------
messages: 286544
nosy: Elli Pirelli
priority: normal
severity: normal
status: open
title: python3.dll export forwarders not resolved in all situations due to missing python3?.dll dependency and DLL search path behavior
type: behavior
versions: Python 3.5, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29399>
_______________________________________

From report at bugs.python.org  Tue Jan 31 12:14:44 2017
From: report at bugs.python.org (Elli Pirelli)
Date: Tue, 31 Jan 2017 17:14:44 +0000
Subject: [issue29399] python3.dll export forwarders not resolved in all
 situations due to missing python3?.dll dependency and DLL search path
 behavior
In-Reply-To: <1485882846.24.0.14620001579.issue29399@psf.upfronthosting.co.za>
Message-ID: <1485882884.49.0.423659366032.issue29399@psf.upfronthosting.co.za>


Changes by Elli Pirelli <gonzelmann at gmail.com>:


----------
components: +Windows
nosy: +paul.moore, steve.dower, tim.golden, zach.ware

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29399>
_______________________________________

From report at bugs.python.org  Tue Jan 31 12:20:19 2017
From: report at bugs.python.org (Mark Haase)
Date: Tue, 31 Jan 2017 17:20:19 +0000
Subject: =?utf-8?q?=5Bissue24875=5D_pyvenv_doesn=C2=B4t_install_PIP_inside_a_new_v?=
 =?utf-8?q?env_with_--system-site-package?=
In-Reply-To: <1439699040.07.0.199956094974.issue24875@psf.upfronthosting.co.za>
Message-ID: <1485883219.15.0.379697440566.issue24875@psf.upfronthosting.co.za>


Mark Haase added the comment:

Oh, sorry! I didn't realize you were a core dev or I would have replied sooner. Thank you for looking into this issue.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24875>
_______________________________________

From report at bugs.python.org  Tue Jan 31 13:20:52 2017
From: report at bugs.python.org (George King)
Date: Tue, 31 Jan 2017 18:20:52 +0000
Subject: [issue29400] Instruction level tracing via sys.settrace
Message-ID: <1485886852.16.0.854514177291.issue29400@psf.upfronthosting.co.za>


New submission from George King:

I have recently put some effort into developing a code coverage tool that shows correct results for intraline branches. In order to get intraline trace data, I patched CPython, adding an optional "trace instructions" flag to sys.settrace. The patch is fairly simple.

http://github.com/gwk/cpython/tree/inst-tracing

The crucial detail in ceval is here:

@@ -4521,6 +4523,9 @@ maybe_call_line_trace(Py_tracefunc func, PyObject *obj,
         frame->f_lineno = line;
         result = call_trace(func, obj, tstate, frame, PyTrace_LINE, Py_None);
     }
+    else if (tstate->inst_tracing) {
+        result = call_trace(func, obj, tstate, frame, PyTrace_INSTRUCTION, Py_None);
+    }

The call to maybe_call_line_trace is guarded by several checks, so I do not see how this would significantly impact performance, except for when tracing is enabled. I built python3.6 using clang -O3 -flto (I saw lots of test failures during the --with-optimizations PGO tests so I opted for a simpler build), with and without my patch. I then ran `python3 -m performance run --fast` and manually compared the results. The patched version was in most cases slightly slower but within the standard deviation of the normal version.

I would appreciate feedback before going any further with this. I looked at haypo's perf project but was unsure how to use it to run a more detailed comparison.

* First, will this feature be rejected on performance grounds or some other obvious reason?

* Second, does setting `tstate->inst_tracing` in PyEval_SetTraceInstructions require more elaborate guards similar to those in PyEval_SetTrace? I do not understand the intricacies of that function.

I am also curious about backwards compatibility. This adds an optional argument to settrace (haypo's recommendation) and some other things that appear to be under Py_LIMITED_API or internal (but I am not sure).
* Is a change like this eligible for 3.6.x or would it have to wait until 3.7?
* Would it be better to add a completely new function, e.g. sys.settrace_instructions instead of changing the signature of settrace?

Lastly, I have a few ideas for enhancing this patch that will require some more work on my end to verify their utility.
In particular:
* Remove the `else` to always trace with PyTrace_INSTRUCTION, rather than as an alternate to PyTrace_LINE as it does now.
* Pass the previous instruction offset as the trace function argument. This would give the tracer complete "edge" information. (In my coverage tool's trace function I track the previous edge as a nonlocal, but it would be better/faster to do it automatically in maybe_call_line_trace).
* Finally, if all of this works, it might also be useful (as a performance enhancement) to add a PyTrace_BRANCH option, which would omit runs of opcodes inbetween branches, as these edges tend not be be useful, and even counterproductive (at least for my use case). However it might be too tricky to get all the cases right.

----------
components: Interpreter Core
messages: 286546
nosy: George.King, haypo
priority: normal
severity: normal
status: open
title: Instruction level tracing via sys.settrace
type: enhancement
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29400>
_______________________________________

From report at bugs.python.org  Tue Jan 31 13:50:14 2017
From: report at bugs.python.org (Terry J. Reedy)
Date: Tue, 31 Jan 2017 18:50:14 +0000
Subject: [issue29287] IDLE needs syntax highlighting for f-strings
In-Reply-To: <1484605208.08.0.296182628472.issue29287@psf.upfronthosting.co.za>
Message-ID: <1485888614.26.0.434548847959.issue29287@psf.upfronthosting.co.za>


Terry J. Reedy added the comment:

This is 3 related but somewhat distinct proposals.

1. Special handling (normal syntax colorizing) of f-expressions (the grammatical term used at https://docs.python.org/3/reference/lexical_analysis.html#formatted-string-literals).  

2. Brace matching within the strings.  (For normal strings, this would be nice when the string is to be used as a format string, but there is no way to know how the string will be used.)

3. Normal identifier autocompletion (as opposed to within-string file name autocompletion).

I have two types of doubts about #1

1. Should it be done?
 a. I have not yet used f-strings, so I not know if I would want uncolorized holes within them.
 b. Do beginners use f-strings?  Would they find this more useful than confusing?
 c. Is this the sort of 'advanced feature' that Guido has said should *not* be copied from other editors?

2. Can it be done sensibly within the limits of IDLE's colorizer. 

IDLE's colorizer defines a giant regex that joins regexes that match keywords, builtins, comments, and strings (and newlines for synchronization).  Each of the latter is a named capturing group that joins alternatives for that group.  Keywords and built-in names are recognized when complete.  Partial comments and strings are recognized as soon as '#' or an open quote is typed.  There is a human-verified test of colorizing that could, I believe, be turned into a unit test of the re matching.  This would be needed for approach b. below.

The compiled re is used in a ColorDelegator instance that is part of a chain of delegators tied to a text widget.  The class code is not documented and I do not understand it well enough to modify it without adding tests.  But it was not designed for easy testing.

Sidenote: There are DEBUG prints in multiple methods (but not inrecolorize_main).  Some messages can come from multiple methods.  I should add message to the r...main method and prefix all messages with an indicator of the source so the control flow is easier to follow

I see two possible approaches to separately colorizing f-expressions within an f-string.

a. Follow the example of 'def' and 'class'.  They are recognized as a special case (of builting) and when they occur, a separate 'if' clause and re is used to colorize the following name.

The problem with doing this with f-strings is that we want to recursively apply the re...main function to a short substring, and the function is not designed for that.  We also want to do this separately for each embedded f-expression.  It might work to write a reduced version of recolorize_main as recolorize_fexp.

This approach would allow for {} matching once a closing quote is typed, but not identifier autocompletion.

b. Do the special-casing by writing special regexes to recognize a null f-string (no embedded f-expression), and beginning, middle, and ending string parts of an f-string.  But I don't know if it is possible to write an re that will *only* match null f-strings.

That aside, the f-expression would then be treated normally, and autocomplete should just work.  {} matching would be harder.  Without adding new state variables, I imagine that the end quote of the invalid f"a{b" would be seen as the beginning of a new string.

----------
stage: needs patch -> test needed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29287>
_______________________________________

From report at bugs.python.org  Tue Jan 31 14:22:57 2017
From: report at bugs.python.org (Michael Haubenwallner)
Date: Tue, 31 Jan 2017 19:22:57 +0000
Subject: [issue27435] ctypes library loading and AIX - also for 2.7.X (and
 later)
In-Reply-To: <1467383699.63.0.495142233463.issue27435@psf.upfronthosting.co.za>
Message-ID: <1485890577.64.0.440553958792.issue27435@psf.upfronthosting.co.za>


Michael Haubenwallner added the comment:

Feels like it is up to me to shed some light on various topics here:

AIX defines (static) "Object" files and "Shared Object" files. Both can (but do not need to) be members of an "Archive Library".

When talking about (dynamically loading) a "shared library" on non-AIX systems, the correct wording for AIX is (dynamically loading) a "Shared Object" - independent of whether it is a standalone file or an archive member.

As you already agreed on, ctypes.util.find_library() indeed should return the location of the Shared Object - as either "/path/to/file" or "/path/to/file(member)". And ctypes.CDLL() should add the RTLD_MEMBER flag to _ctypes.dlopen() if there is the "(member)" part.

However, that does not necessarily mean the RTLD_MEMBER value needs to be available through the ctypes API. Instead, RTLD_MEMBER feels more appropriate for the _ctypes API - and on AIX only.

Anyway:
Unfortunately, there is no concept of embedding something like ELF's DT_SONAME tag into the Shared Object. The very (PATH,BASE,MEMBER) value as (specified to and) discovered by the linker is recorded into the just-linked executable (or Shared Object). This implies that the runtime loader does search for the very same filename (and member eventually) as the linker (at linktime).

However, to still get some kind of shared library versioning, multiple versions of one Shared Object are put into one single Archive Library, where the "old" versions get the LOADONLY flag set (via 'strip -e') - so executables linked against an old version still can load their "old" Shared Object, while the linker discovers the "current" version only.

But this "single-filename" based shared library versioning is a huge burden for packages managers - either human or software, even when they maintain their packages in a private prefix (like /opt/freeware, /usr/local and similar). Neither can they put their "current" Shared Object's version into the single (system's) Archive Library, nor can they mark existing Shared Objects as LOADONLY.
On the other hand, they also do not want to pull all the system versions into their own Archive Library.

So for package managers it is crucial to have "multi-filename" based shared library versioning instead.

But there is help:
AIX also provides (plain text) "Import Files": They contain a list of symbols for the linker, and a (PATH,BASE,MEMBER) tuple naming the Shared Object that provides these symbols at runtime.

So yes, Import Files can be used for "multi-filename" based shared library versioning.

Thus, libtool now offers the "--with-aix-soname=svr4" configure flag, where "libNAME.so.X" really is created as an Archive Library, containing the Shared Object "shr.o" with the LOADONLY flag set, and the Import File "shr.imp" referring to "libNAME.so.X(shr.o)" for runtime. Combined with the symlink "libNAME.so", for the "-lNAME" argument the linker discovers the Import File "libNAME.so(shr.imp)" now, while recording "libNAME.so.X(shr.o)" into the executable for the runtime loader.
Note that for 64bit the Shared Object and Import File is named "shr_64.o" and "shr_64.imp", respectively.

While libtool's "--with-aix-soname=svr4" variant creates "libNAME.a" from static Objects only, both libtool's "--with-aix-soname=aix" and "--with-aix-soname=both" - for backwards compatibility - create "libNAME.a" for traditional "single-filename" based versioning: Without any static Object - as even Shared Objects can be linked statically(!). But to statically link the Shared Object (as an archive member), neither the LOADONLY flag must be set, nor an Import File must be found (as Import Files cannot serve for static linking).

And "--with-aix-soname=aix", libtool still creates standalone "libNAME.so.X.Y.Z" along the (versioning-wise useless) symlinks.

So it is up to the package manager which "aix-soname" variant to choose within its prefix: For example, in Gentoo Prefix I'm using "--with-aix-soname=svr4" only.

But still, how to get ctypes.find_library() working - ideally for each variant, is another story. Right now it does not work for any variant, but I guess that search algorithm should follow how the linker discovers the (PATH,BASE,MEMBER) values to write into just-linked executables, combined with how the runtime loader finds the Shared Object to actually load.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue27435>
_______________________________________

From report at bugs.python.org  Tue Jan 31 15:10:17 2017
From: report at bugs.python.org (Gerrit Holl)
Date: Tue, 31 Jan 2017 20:10:17 +0000
Subject: [issue29401] Python3.6 docs should not have a "what's new in Python
 3.7" page
Message-ID: <1485893417.06.0.864310985594.issue29401@psf.upfronthosting.co.za>


New submission from Gerrit Holl:

I believe this page is not supposed to exist:

https://docs.python.org/3.6/whatsnew/3.7.html

Nor this one:

https://docs.python.org/3.5/whatsnew/3.6.html

Neither are maintained, nor would I expect them to be.

----------
assignee: docs at python
components: Documentation
messages: 286549
nosy: Gerrit.Holl, docs at python
priority: normal
severity: normal
status: open
title: Python3.6 docs should not have a "what's new in Python 3.7" page
versions: Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29401>
_______________________________________

From report at bugs.python.org  Tue Jan 31 15:32:30 2017
From: report at bugs.python.org (Martin Panter)
Date: Tue, 31 Jan 2017 20:32:30 +0000
Subject: [issue29169] update zlib to 1.2.11
In-Reply-To: <1483629107.49.0.982374684244.issue29169@psf.upfronthosting.co.za>
Message-ID: <1485894750.12.0.468877969987.issue29169@psf.upfronthosting.co.za>


Martin Panter added the comment:

Misc/NEWS (and the commit message) say 1.2.10. Perhaps you meant 1.2.11?

----------
nosy: +martin.panter

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29169>
_______________________________________

From report at bugs.python.org  Tue Jan 31 15:39:13 2017
From: report at bugs.python.org (Ned Deily)
Date: Tue, 31 Jan 2017 20:39:13 +0000
Subject: [issue29401] Python3.6 docs should not have a "what's new in Python
 3.7" page
In-Reply-To: <1485893417.06.0.864310985594.issue29401@psf.upfronthosting.co.za>
Message-ID: <1485895153.34.0.835800179374.issue29401@psf.upfronthosting.co.za>


Ned Deily added the comment:

Thanks for the report.  Those two files should disappear soon.

----------
nosy: +ned.deily
resolution:  -> fixed
status: open -> pending

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29401>
_______________________________________

From report at bugs.python.org  Tue Jan 31 15:56:05 2017
From: report at bugs.python.org (Ronald Oussoren)
Date: Tue, 31 Jan 2017 20:56:05 +0000
Subject: [issue28747] Expose SSL_CTX_set_cert_verify_callback
In-Reply-To: <1479591372.9.0.198427018061.issue28747@psf.upfronthosting.co.za>
Message-ID: <1485896165.48.0.0908364225277.issue28747@psf.upfronthosting.co.za>


Changes by Ronald Oussoren <ronaldoussoren at mac.com>:


----------
nosy: +ronaldoussoren

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28747>
_______________________________________

From report at bugs.python.org  Tue Jan 31 16:15:42 2017
From: report at bugs.python.org (Ned Deily)
Date: Tue, 31 Jan 2017 21:15:42 +0000
Subject: [issue29401] Python3.6 docs should not have a "what's new in Python
 3.7" page
In-Reply-To: <1485893417.06.0.864310985594.issue29401@psf.upfronthosting.co.za>
Message-ID: <1485897342.84.0.515388478255.issue29401@psf.upfronthosting.co.za>


Changes by Ned Deily <nad at python.org>:


----------
stage:  -> resolved
status: pending -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29401>
_______________________________________

From report at bugs.python.org  Tue Jan 31 16:34:54 2017
From: report at bugs.python.org (Terry J. Reedy)
Date: Tue, 31 Jan 2017 21:34:54 +0000
Subject: [issue29402] Problem with Checkbutton and duplicate last name
 components
Message-ID: <1485898494.74.0.656128703551.issue29402@psf.upfronthosting.co.za>


New submission from Terry J. Reedy:

Report and code from George Trojan on python-list.

import tkinter

class GUI(tkinter.Tk):
    def __init__(self):
        tkinter.Tk.__init__(self)
        frame = tkinter.Frame(self)
        for tag in ('A', 'B'):
            w = tkinter.Checkbutton(frame, text=tag)
            w.pack(side='top', padx=10, pady=10)
            print(w)
        frame.pack(side='top')
        frame = tkinter.Frame(self)
        for tag in ('C', 'D'):
            w = tkinter.Checkbutton(frame, text=tag)
            w.pack(side='top', padx=10, pady=10)
            print(w)
        frame.pack(side='top')

gui = GUI()
gui.mainloop()

In 3.5, each Checkbutton has unique last name component.
.2028654224384.2028654606600
.2028654224384.2028654608224
.2028654606656.2028654606824
.2028654606656.2028654608336
Clicking any box checks or unchecks exactly that box.

In 3.6, last name components are duplicated
.!frame.!checkbutton
.!frame.!checkbutton2
.!frame2.!checkbutton
.!frame2.!checkbutton2
Clicking any box checks or unchecks both button with 'same name'.

I verified that the _w strings passed in tk.call are the full unique names.

MRAB reported that adding a tk variable attribute fixed the problem.  I notice that Brian Oakley said the same thing in the similar issue #25684, though that is marked for 2.7 and 3.5 also.

----------
keywords: 3.6regression
messages: 286552
nosy: serhiy.storchaka, terry.reedy
priority: normal
severity: normal
stage: test needed
status: open
title: Problem with Checkbutton and  duplicate last name components
type: behavior
versions: Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29402>
_______________________________________

From report at bugs.python.org  Tue Jan 31 16:56:41 2017
From: report at bugs.python.org (Eryk Sun)
Date: Tue, 31 Jan 2017 21:56:41 +0000
Subject: [issue29399] python3.dll export forwarders not resolved in all
 situations due to missing python3?.dll dependency and DLL search path
 behavior
In-Reply-To: <1485882846.24.0.14620001579.issue29399@psf.upfronthosting.co.za>
Message-ID: <1485899801.31.0.736308082995.issue29399@psf.upfronthosting.co.za>


Eryk Sun added the comment:

LoadLibraryEx with LOAD_WITH_ALTERED_SEARCH_PATH worked for me in Windows 10. I tested loading 64-bit python3.dll, for both 3.5 and 3.6, and getting the address of Py_Main. The DLLs were loaded from DLL35 and DLL36 subdirectories. It worked using both a relative and a fully-qualified path, but, per the docs, to be safe you should use the fully-qualified path to python3.dll.

----------
nosy: +eryksun

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29399>
_______________________________________

From report at bugs.python.org  Tue Jan 31 17:10:33 2017
From: report at bugs.python.org (Joshua Kinard)
Date: Tue, 31 Jan 2017 22:10:33 +0000
Subject: [issue27647] Update Windows build to Tcl/Tk 8.6.6
In-Reply-To: <1469737194.88.0.144648917966.issue27647@psf.upfronthosting.co.za>
Message-ID: <1485900633.66.0.351533621337.issue27647@psf.upfronthosting.co.za>


Joshua Kinard added the comment:

Hi,

I came across a StackOverflow question that apparently turned up a bug in Tk itself with regards to Spinboxes on Windows Aero themes.  The question is here:
http://stackoverflow.com/q/30783603/

And I did a deep-dive of the issue here as the accepted answer:
http://stackoverflow.com/a/36393680/

This led to someone from Tk noticing, and they patched the bug in Tk upstream in this commit:
http://core.tcl.tk/tk/info/f91b7065bf1bf655

I do not know if that commit is in a current Tk release (doubtful).  Could you guys look at picking it up for the next Python 2.7/3.x release?  It only affects Windows platforms using Aero.  Classic mode spinboxes or other Tk themes (e.g., clam) are unaffected.

It'd also be awesome if Tk-8.6 could go into the next Python 2.7 release under Windows.  That has PNG support for Tk's PhotoImage class, which would make working with icons and transparency a *lot* easier.

----------
nosy: +kumba

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue27647>
_______________________________________

From report at bugs.python.org  Tue Jan 31 17:10:49 2017
From: report at bugs.python.org (Terry J. Reedy)
Date: Tue, 31 Jan 2017 22:10:49 +0000
Subject: [issue27051] Create PIP gui
In-Reply-To: <1463550469.55.0.680424350053.issue27051@psf.upfronthosting.co.za>
Message-ID: <1485900649.51.0.735538767877.issue27051@psf.upfronthosting.co.za>


Terry J. Reedy added the comment:

Upendra, can you upload a zipped copy of your code as of the end of GSOC or later?  We should have a permanent copy of your work even if your github account goes away.

Since the idea of adding a pipgui to the CPython repository has been rejected, I believe this issue should be closed (as 'rejected').

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue27051>
_______________________________________

From report at bugs.python.org  Tue Jan 31 17:19:33 2017
From: report at bugs.python.org (Terry J. Reedy)
Date: Tue, 31 Jan 2017 22:19:33 +0000
Subject: [issue23551] IDLE to provide menu link to PIP gui.
In-Reply-To: <1425155752.55.0.0736033483828.issue23551@psf.upfronthosting.co.za>
Message-ID: <1485901173.84.0.53879180053.issue23551@psf.upfronthosting.co.za>


Terry J. Reedy added the comment:

Ned Deily and Nick Coughlin vetoed putting a pipgui in the CPython repository -- see #27051.  So, for the foreseeable future, there will not be anything to link to.

Note: I had to remove the open dependency because the tracker does not differentiate between closing something as fixed and closing it as unfixed.

----------
dependencies:  -Create PIP gui
resolution:  -> later
stage: needs patch -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23551>
_______________________________________

From report at bugs.python.org  Tue Jan 31 17:29:16 2017
From: report at bugs.python.org (Terry J. Reedy)
Date: Tue, 31 Jan 2017 22:29:16 +0000
Subject: [issue27647] Update Windows build to Tcl/Tk 8.6.6
In-Reply-To: <1469737194.88.0.144648917966.issue27647@psf.upfronthosting.co.za>
Message-ID: <1485901756.35.0.709524776187.issue27647@psf.upfronthosting.co.za>


Terry J. Reedy added the comment:

Changing tk in 2.7.z from 8.5 to 8.6 would add new features, which we don't usually do in bugfix releases.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue27647>
_______________________________________

From report at bugs.python.org  Tue Jan 31 17:29:54 2017
From: report at bugs.python.org (Larry Hastings)
Date: Tue, 31 Jan 2017 22:29:54 +0000
Subject: [issue27647] Update Windows build to Tcl/Tk 8.6.6
In-Reply-To: <1485900633.66.0.351533621337.issue27647@psf.upfronthosting.co.za>
Message-ID: <CAOBd=7=48ce2jv-zHX4wjgG8R4YWAv4LP-QZo=BkTyg2XJxb0A@mail.gmail.com>


Larry Hastings added the comment:

I don't think we should update it in 3.5.  That sounds destabilizing.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue27647>
_______________________________________

From report at bugs.python.org  Tue Jan 31 17:36:41 2017
From: report at bugs.python.org (Elli Pirelli)
Date: Tue, 31 Jan 2017 22:36:41 +0000
Subject: [issue29399] python3.dll export forwarders not resolved in all
 situations due to missing python3?.dll dependency and DLL search path
 behavior
In-Reply-To: <1485882846.24.0.14620001579.issue29399@psf.upfronthosting.co.za>
Message-ID: <1485902201.69.0.980422904545.issue29399@psf.upfronthosting.co.za>


Elli Pirelli added the comment:

It does not work on Windows 7 Pro x64.

And i am not the only one. The maintainer of the "python-embedded-launcher" also encountered the issue (see discussion here: https://github.com/zsquareplusc/python-embedded-launcher/issues/3)

Are you sure it really works on Windows 10? Perhaps you have the python3?.dll in the standard DLL search path already, making you mistakenly believe that it works properly? I would suggest to use a tool like SysInternal's Process Monitor to see and verify where the python3?.dll is really being loaded from.


The issue is not with incorrectly using LoadLibraryEx, SetDllDirectory and AddDllDirectory. The problem can be reduced to the python3.dll not having a dependcy on python3?.dll.

I did verify this with a little project, emulating how python3.dll uses export forwarders. I made a DLL (real.dll) with a dummy function Func  and another DLL (forwarder.dll) containing just an export forwarder Func=real.Func. 

As described, as long as forwarder.dll does not contain a import dependency on real.dll, loading forwarder.dll and doing GetProcAddress("Func") will fail, if real.dll is not in the standard DLL search path.

As soon as forwarder.dll includes a dependency to real.dll, the export forwarder can be resolved by GetProcAddress("Func") properly.

If i were using LoadLibraryEx/SetDllDirectory/AddDllDirectory incorrectly, attempting to load the forwarder.dll with the dependency on real.dll would just fail -- which it doesn't.

Unfortunately, the whole business with export forwarders is very poorly documented. It could perhaps be that MS did change something in Win8/2K12 or Win10 to fix this issue. Since i currently don't have access to either of these OS, i myself cannot confirm whether the problem would also occur or not on Win 10...

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29399>
_______________________________________

From report at bugs.python.org  Tue Jan 31 17:47:17 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 31 Jan 2017 22:47:17 +0000
Subject: [issue29400] Instruction level tracing via sys.settrace
In-Reply-To: <1485886852.16.0.854514177291.issue29400@psf.upfronthosting.co.za>
Message-ID: <CAMpsgwZ2B46hgPpKDxHZVP-4dKgH0+XZW0i_yziUG5i+0jr+zQ@mail.gmail.com>


STINNER Victor added the comment:

Can you please attach your patch?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29400>
_______________________________________

From report at bugs.python.org  Tue Jan 31 17:48:10 2017
From: report at bugs.python.org (Elli Pirelli)
Date: Tue, 31 Jan 2017 22:48:10 +0000
Subject: [issue29399] python3.dll export forwarders not resolved in all
 situations due to missing python3?.dll dependency and DLL search path
 behavior
In-Reply-To: <1485882846.24.0.14620001579.issue29399@psf.upfronthosting.co.za>
Message-ID: <1485902890.51.0.436027929505.issue29399@psf.upfronthosting.co.za>


Elli Pirelli added the comment:

I should perhaps clarify my sentence "As described, as long as forwarder.dll does not contain a import dependency on real.dll, loading forwarder.dll and doing GetProcAddress("Func") will fail, if real.dll is not in the standard DLL search path."

What i tried to say was:

As described, as long as forwarder.dll does not contain a import dependency on real.dll, loading forwarder.dll succeeds, but doing GetProcAddress("Func") will fail if real.dll is not in the standard DLL search path. If this forwarder.dll is put in one of the standard DLL search paths, GetProcAddress("Func") successfully resolves the address of the dummy function Func in real.dll.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29399>
_______________________________________

From report at bugs.python.org  Tue Jan 31 17:49:19 2017
From: report at bugs.python.org (Ammar Askar)
Date: Tue, 31 Jan 2017 22:49:19 +0000
Subject: [issue29400] Instruction level tracing via sys.settrace
In-Reply-To: <1485886852.16.0.854514177291.issue29400@psf.upfronthosting.co.za>
Message-ID: <1485902959.79.0.0549916137621.issue29400@psf.upfronthosting.co.za>


Ammar Askar added the comment:

A git style patch can be found at https://github.com/gwk/cpython/commit/071d17cedfdf2db8b405aab5adabebe2ac5ef67b.patch

----------
nosy: +ammar2

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29400>
_______________________________________

From report at bugs.python.org  Tue Jan 31 17:53:53 2017
From: report at bugs.python.org (Eryk Sun)
Date: Tue, 31 Jan 2017 22:53:53 +0000
Subject: [issue29399] python3.dll export forwarders not resolved in all
 situations due to missing python3?.dll dependency and DLL search path
 behavior
In-Reply-To: <1485882846.24.0.14620001579.issue29399@psf.upfronthosting.co.za>
Message-ID: <1485903233.06.0.782220605549.issue29399@psf.upfronthosting.co.za>


Eryk Sun added the comment:

> Are you sure it really works on Windows 10? 

Neither the DLL35 nor DLL36 directory was set in PATH, nor any directory containing python3x.dll. Each of the latter two directories contained only the minimal set python3.dll, python3x.dll, and vcruntime140.dll. I ran under a debugger to confirm the delayed loading of python3x.dll when GetProcAddress is called, and that the returned address of Py_Main is within the mapped range of python3x.dll. I'll try on Windows 7 in a little while.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29399>
_______________________________________

From report at bugs.python.org  Tue Jan 31 18:01:55 2017
From: report at bugs.python.org (Aaron Gallagher)
Date: Tue, 31 Jan 2017 23:01:55 +0000
Subject: [issue29403] mock's autospec's behavior on method-bound builtin
 functions is broken
Message-ID: <1485903715.23.0.569013669104.issue29403@psf.upfronthosting.co.za>


New submission from Aaron Gallagher:

Cython will, in the right circumstances, offer a MethodType instance where im_func is a builtin function. Any instance of MethodType is automatically assumed to be a python-defined function (more specifically, a function that has an inspectable signature), but _set_signature was still conservative in its assumptions. As a result _set_signature would return early with None instead of a mock since the im_func had no inspectable signature. This causes problems deeper inside mock, as _set_signature is assumed to always return a mock, and nothing checked its return value.

In similar corner cases, autospec will simply not check the spec of the function, so _set_signature is amended to now return early with the original, not-wrapped mock object.

There is another None early return in _set_signature, but it seems.. harder to trigger? It seems more appropriate to raise an exception there if this is being fixed.

----------
components: Library (Lib)
messages: 286565
nosy: Aaron Gallagher
priority: normal
severity: normal
status: open
title: mock's autospec's behavior on method-bound builtin functions is broken
type: behavior
versions: Python 2.7, Python 3.3, Python 3.4, Python 3.5, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29403>
_______________________________________

From report at bugs.python.org  Tue Jan 31 18:08:03 2017
From: report at bugs.python.org (Aaron Gallagher)
Date: Tue, 31 Jan 2017 23:08:03 +0000
Subject: [issue29403] mock's autospec's behavior on method-bound builtin
 functions is broken
In-Reply-To: <1485903715.23.0.569013669104.issue29403@psf.upfronthosting.co.za>
Message-ID: <1485904083.29.0.791498969732.issue29403@psf.upfronthosting.co.za>


Aaron Gallagher added the comment:

Patch adapted from https://github.com/testing-cabal/mock/pull/389

----------
keywords: +patch
Added file: http://bugs.python.org/file46468/mock.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29403>
_______________________________________

From report at bugs.python.org  Tue Jan 31 19:00:11 2017
From: report at bugs.python.org (Vincent Pelletier)
Date: Wed, 01 Feb 2017 00:00:11 +0000
Subject: [issue29404] "TypeError: 'int' does not have the buffer interface" on
 memoryview over bytearray
Message-ID: <1485907211.51.0.510192590668.issue29404@psf.upfronthosting.co.za>


New submission from Vincent Pelletier:

Quoting a github gist[1] (not mine, I found it while googling for this error) clearly illustrating the issue:

# Python 2
>>> b = bytearray(8)
>>> v = memoryview(b)
>>> v[0] = 42
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'int' does not have the buffer interface
>>> b
bytearray(b'\x00\x00\x00\x00\x00\x00\x00\x00')

# Python 3
>>> b = bytearray(8)
>>> v = memoryview(b)
>>> v[0] = 42
>>> b
bytearray(b'*\x00\x00\x00\x00\x00\x00\x00')

This is especially annoying as on python3 bytearray.__setitem__ only accepts integers, making working py2 code totally incompatible with py3 in a 2to3-undetectable way.

[1] https://gist.github.com/superbobry/b90c0cc7c44beaa51ef9

----------
messages: 286567
nosy: vpelletier
priority: normal
severity: normal
status: open
title: "TypeError: 'int' does not have the buffer interface" on memoryview over bytearray
type: behavior
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29404>
_______________________________________

From report at bugs.python.org  Tue Jan 31 19:07:43 2017
From: report at bugs.python.org (Eryk Sun)
Date: Wed, 01 Feb 2017 00:07:43 +0000
Subject: [issue29399] python3.dll export forwarders not resolved in all
 situations due to missing python3?.dll dependency and DLL search path
 behavior
In-Reply-To: <1485882846.24.0.14620001579.issue29399@psf.upfronthosting.co.za>
Message-ID: <1485907663.45.0.189991411093.issue29399@psf.upfronthosting.co.za>


Eryk Sun added the comment:

I can confirm that LoadLibraryEx w/ LOAD_WITH_ALTERED_SEARCH_PATH unfortunately does not work in Windows 7. Actually I almost looked into this before on python-list:

https://mail.python.org/pipermail/python-list/2016-September/thread.html#714622

but I got sidetracked in the manifest discussion and forgot to fully investigate the issue in different versions of Windows. Using the py3embed assembly that I described in that thread will work in Windows 7, if you want to give that a try. In some ways it's simpler.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29399>
_______________________________________

From report at bugs.python.org  Tue Jan 31 19:13:14 2017
From: report at bugs.python.org (Joshua Kinard)
Date: Wed, 01 Feb 2017 00:13:14 +0000
Subject: [issue27647] Update Windows build to Tcl/Tk 8.6.6
In-Reply-To: <1469737194.88.0.144648917966.issue27647@psf.upfronthosting.co.za>
Message-ID: <1485907993.99.0.548718426142.issue27647@psf.upfronthosting.co.za>


Joshua Kinard added the comment:

@Terry: This only applies on the Windows release of Python.  On both Linux and FreeBSD, you can wind up with Python 2.7 and Tk-8.6.x by default in some instances:

# pkg info | grep python27
python27-2.7.13_1              Interpreted object-oriented programming language
# pkg info | grep tk86
tk86-8.6.6                     Graphical toolkit for Tcl

That's a FreeBSD-11.0-RELEASE install on a laptop.

Is there guidance anywhere on which Tk version a given Python version prefers to be used with?  E.g., is Python 2.7 only supported against Tk-8.5.x, leaving 8.6.x users on their own?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue27647>
_______________________________________

From report at bugs.python.org  Tue Jan 31 19:15:50 2017
From: report at bugs.python.org (Brian Martin)
Date: Wed, 01 Feb 2017 00:15:50 +0000
Subject: [issue29398] memory corruption in xxlimited
In-Reply-To: <1485874593.96.0.823310722961.issue29398@psf.upfronthosting.co.za>
Message-ID: <1485908150.78.0.163848312362.issue29398@psf.upfronthosting.co.za>


Changes by Brian Martin <brian at opensecurityfoundation.org>:


----------
nosy: +osvdb

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29398>
_______________________________________

From report at bugs.python.org  Tue Jan 31 19:26:25 2017
From: report at bugs.python.org (Milt Epstein)
Date: Wed, 01 Feb 2017 00:26:25 +0000
Subject: [issue29405] improve csv.Sniffer().sniff() behavior
Message-ID: <1485908785.08.0.552892497325.issue29405@psf.upfronthosting.co.za>


New submission from Milt Epstein:

I'm trying to use csv.Sniffer().sniff(sample_data) to determine the delimiter on a number of input files.  Through some trial and error, many "Could not determine delimiter" errors, and analyzing how this routine works/behaves, I settled on sample_data being some number of lines of the input file, particularly 30.  This value seems to allow the routine to work more frequently, although not always, particularly on short input files.

I realize the way this routine works is somewhat idiosyncratic, and it won't be so easy to improve it generally, but there's one simple change that occurred to me that would help in some cases.  Currently the function _guess_delimiter() in csv.py contains the following lines:

            # build a list of possible delimiters
            modeList = modes.items()
            total = float(chunkLength * iteration)

So total is increased by chunkLength on each iteration.  The problem occurs when total becomes greater than the length of sample_data, that is, the iteration would go beyond the end of sample_data.  That reading is handled fine, it's truncated at the end of sample_data, but total is needlessly set too high.  My suggested change is to add the following two lines after the above:

            if total > len(data):
                total = float(len(data))

----------
components: Library (Lib)
messages: 286570
nosy: mepstein
priority: normal
severity: normal
status: open
title: improve csv.Sniffer().sniff() behavior
type: behavior
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29405>
_______________________________________

From report at bugs.python.org  Tue Jan 31 19:35:04 2017
From: report at bugs.python.org (Milt Epstein)
Date: Wed, 01 Feb 2017 00:35:04 +0000
Subject: [issue29405] improve csv.Sniffer().sniff() behavior
In-Reply-To: <1485908785.08.0.552892497325.issue29405@psf.upfronthosting.co.za>
Message-ID: <1485909304.4.0.845686952752.issue29405@psf.upfronthosting.co.za>


Milt Epstein added the comment:

FWIW, it might be more concise and more consistent with the existing code to change the one line to:

            total = min(float(chunkLength * iteration), float(len(data)))

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29405>
_______________________________________

From report at bugs.python.org  Tue Jan 31 19:37:15 2017
From: report at bugs.python.org (Guido van Rossum)
Date: Wed, 01 Feb 2017 00:37:15 +0000
Subject: [issue29377] Add the 'wrapper_descriptor' type to the types module
In-Reply-To: <1485425778.09.0.153066086326.issue29377@psf.upfronthosting.co.za>
Message-ID: <1485909435.6.0.722903497693.issue29377@psf.upfronthosting.co.za>


Guido van Rossum added the comment:

Seems the combined patch doesn't include the tests. Nor does the most recent slot-wrappre-types.patch.

Ivan, can you make a single truly combined patch and add a Misc/NEWS entry to it as well?

Other than that this looks fine.

Maybe we need to wait for the github migration to complete though?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29377>
_______________________________________

From report at bugs.python.org  Tue Jan 31 19:39:05 2017
From: report at bugs.python.org (Alexander Mohr)
Date: Wed, 01 Feb 2017 00:39:05 +0000
Subject: [issue29406] asyncio SSL contexts leak sockets after calling close
 with certain Apache servers
Message-ID: <1485909545.04.0.650924943503.issue29406@psf.upfronthosting.co.za>


New submission from Alexander Mohr:

with the attached code note how HttpClient.connection_lost callback is never called for the madis server. The madis server is an apache server, I tried with the OSX apache server and could not reproduce the issue so it seems something particular about their apache version or configuration.  This is a pretty critical issue as close() does not release the socket.

----------
components: asyncio
files: scratch_1.py
messages: 286573
nosy: gvanrossum, thehesiod, yselivanov
priority: normal
severity: normal
status: open
title: asyncio SSL contexts leak sockets after calling close with certain Apache servers
type: resource usage
versions: Python 3.5
Added file: http://bugs.python.org/file46469/scratch_1.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29406>
_______________________________________

From report at bugs.python.org  Tue Jan 31 19:42:01 2017
From: report at bugs.python.org (Alexander Mohr)
Date: Wed, 01 Feb 2017 00:42:01 +0000
Subject: [issue29406] asyncio SSL contexts leak sockets after calling close
 with certain Apache servers
In-Reply-To: <1485909545.04.0.650924943503.issue29406@psf.upfronthosting.co.za>
Message-ID: <1485909721.02.0.244551620152.issue29406@psf.upfronthosting.co.za>


Alexander Mohr added the comment:

updating to make default the error case (madis)

----------
Added file: http://bugs.python.org/file46470/scratch_1.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29406>
_______________________________________

From report at bugs.python.org  Tue Jan 31 19:42:25 2017
From: report at bugs.python.org (Alexander Mohr)
Date: Wed, 01 Feb 2017 00:42:25 +0000
Subject: [issue29406] asyncio SSL contexts leak sockets after calling close
 with certain Apache servers
In-Reply-To: <1485909545.04.0.650924943503.issue29406@psf.upfronthosting.co.za>
Message-ID: <1485909745.13.0.0474043405145.issue29406@psf.upfronthosting.co.za>


Changes by Alexander Mohr <thehesiod at gmail.com>:


Removed file: http://bugs.python.org/file46469/scratch_1.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29406>
_______________________________________

From report at bugs.python.org  Tue Jan 31 19:54:43 2017
From: report at bugs.python.org (Berker Peksag)
Date: Wed, 01 Feb 2017 00:54:43 +0000
Subject: [issue29344] sock_recv not detected a coroutine
In-Reply-To: <1485094185.41.0.725056014669.issue29344@psf.upfronthosting.co.za>
Message-ID: <1485910483.48.0.205647199571.issue29344@psf.upfronthosting.co.za>


Berker Peksag added the comment:

I'm not really familiar with asyncio terminology, but it looks like it returns a future:

    >>> asyncio.isfuture(loop.sock_recv(sock, 1024))
    True

There is a also a test in test_selector_events:

    f = self.loop.sock_recv(sock, 1024)
    self.assertIsInstance(f, asyncio.Future)

----------
nosy: +berker.peksag

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29344>
_______________________________________

From report at bugs.python.org  Tue Jan 31 20:09:07 2017
From: report at bugs.python.org (Berker Peksag)
Date: Wed, 01 Feb 2017 01:09:07 +0000
Subject: [issue29407] Remove redundant ensure_future() calls in factorial
 example
Message-ID: <1485911347.12.0.0497985806483.issue29407@psf.upfronthosting.co.za>


New submission from Berker Peksag:

Unless I'm missing something, ensure_future() calls at https://docs.python.org/3.6/library/asyncio-task.html#example-parallel-execution-of-tasks are redundant and can be removed.

Off-topic: Can we use the async/await keywords in asyncio docs now?

----------
assignee: docs at python
components: Documentation, asyncio
files: ensure_future.diff
keywords: patch
messages: 286576
nosy: berker.peksag, docs at python, gvanrossum, yselivanov
priority: normal
severity: normal
stage: patch review
status: open
title: Remove redundant ensure_future() calls in factorial example
type: enhancement
versions: Python 3.5, Python 3.6, Python 3.7
Added file: http://bugs.python.org/file46471/ensure_future.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29407>
_______________________________________

From report at bugs.python.org  Tue Jan 31 20:20:09 2017
From: report at bugs.python.org (Daniel Nunes)
Date: Wed, 01 Feb 2017 01:20:09 +0000
Subject: [issue29408] Add timestamp-based dependency system to build_clib
Message-ID: <1485912008.84.0.925111372259.issue29408@psf.upfronthosting.co.za>


New submission from Daniel Nunes:

Summary:
  - Added newer_pairwise_group function to distutils.dep_util
  - Added documentation and tests for previous function
  - Added timestamp-based dependency system to build_libraries in build_clib
  - Added tests for the above

build_clib is sorely missing one thing in distutils - checking outdated dependencies when compiling. This makes it so that, when building a static c library, all objects are always all compiled.

The function to create the static library itself does implement something of this sort, but since all objects have been recompiled it has no practical effect (everything still runs every time).

This is a small patch designed to fix that, with a couple of qol improvements along the way. Two new keys were added to the build_info dictionary:
    obj_deps - Allows specifying extra dependencies for each source so they can be recognized as being newer than the object being compiled. 
    cflags - This just allows the user to provide an extra set of flags to the compiler.

That is pretty much it, a simple (but important) quality of life improvement for those of us who need to compile static libs with python.

Example:
setup(
...
libraries=[(
    'examplelib', 
    {'sources': 'example.c', 
     'obj_deps': {'': 'global.h',  # use an empty string to define global (for all sources) dependencies
                  'example.c': 'example.h'}
    }
)]
...
)


Documentation for the new function is done but there is no documentation for build_clib itself. After submitting this I'll start working on it and will wait to submit it until this patch is accepted/rejected (so I can know what to include).

Thanks!

----------
components: Distutils
files: build_clib.patch
keywords: patch
messages: 286577
nosy: daniel.nunes, dstufft, eric.araujo
priority: normal
severity: normal
status: open
title: Add timestamp-based dependency system to build_clib
type: enhancement
versions: Python 3.7
Added file: http://bugs.python.org/file46472/build_clib.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29408>
_______________________________________

From report at bugs.python.org  Tue Jan 31 20:38:00 2017
From: report at bugs.python.org (Roundup Robot)
Date: Wed, 01 Feb 2017 01:38:00 +0000
Subject: [issue29218] distutils: Remove unused install_misc class
In-Reply-To: <1484002873.56.0.81761190557.issue29218@psf.upfronthosting.co.za>
Message-ID: <20170201013755.25455.20963.47EAFF3A@psf.io>


Roundup Robot added the comment:

New changeset 19c4528f53b0 by Berker Peksag in branch 'default':
Issue #29218: Remove unused install_misc command
https://hg.python.org/cpython/rev/19c4528f53b0

----------
nosy: +python-dev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29218>
_______________________________________

From report at bugs.python.org  Tue Jan 31 20:41:58 2017
From: report at bugs.python.org (Berker Peksag)
Date: Wed, 01 Feb 2017 01:41:58 +0000
Subject: [issue29218] distutils: Remove unused install_misc class
In-Reply-To: <1484002873.56.0.81761190557.issue29218@psf.upfronthosting.co.za>
Message-ID: <1485913318.15.0.29559645272.issue29218@psf.upfronthosting.co.za>


Berker Peksag added the comment:

Thanks! Since Greg has given his LGTM and I couldn't find any usage of it in the wild, I've committed the patch with a NEWS entry.

----------
nosy: +berker.peksag
resolution:  -> fixed
stage:  -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29218>
_______________________________________

From report at bugs.python.org  Tue Jan 31 21:00:22 2017
From: report at bugs.python.org (Roundup Robot)
Date: Wed, 01 Feb 2017 02:00:22 +0000
Subject: [issue29218] distutils: Remove unused install_misc class
In-Reply-To: <1484002873.56.0.81761190557.issue29218@psf.upfronthosting.co.za>
Message-ID: <1485914422.55.0.387565278148.issue29218@psf.upfronthosting.co.za>


Roundup Robot added the comment:


New changeset ef6bf4c28f78c1f9ad01ff466c9c84d4513a506b by Berker Peksag in branch 'master':
Issue #29218: Remove unused install_misc command
https://github.com/python/cpython/commit/ef6bf4c28f78c1f9ad01ff466c9c84d4513a506b


----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29218>
_______________________________________

From report at bugs.python.org  Tue Jan 31 21:00:22 2017
From: report at bugs.python.org (Roundup Robot)
Date: Wed, 01 Feb 2017 02:00:22 +0000
Subject: [issue29218] distutils: Remove unused install_misc class
In-Reply-To: <1484002873.56.0.81761190557.issue29218@psf.upfronthosting.co.za>
Message-ID: <1485914422.65.0.757021742688.issue29218@psf.upfronthosting.co.za>


Changes by Roundup Robot <devnull at psf.upfronthosting.co.za>:


_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29218>
_______________________________________

From report at bugs.python.org  Tue Jan 31 21:00:22 2017
From: report at bugs.python.org (Roundup Robot)
Date: Wed, 01 Feb 2017 02:00:22 +0000
Subject: [issue29218] distutils: Remove unused install_misc class
In-Reply-To: <1484002873.56.0.81761190557.issue29218@psf.upfronthosting.co.za>
Message-ID: <1485914422.71.0.80196022367.issue29218@psf.upfronthosting.co.za>


Changes by Roundup Robot <devnull at psf.upfronthosting.co.za>:


_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29218>
_______________________________________

From report at bugs.python.org  Tue Jan 31 21:00:22 2017
From: report at bugs.python.org (Roundup Robot)
Date: Wed, 01 Feb 2017 02:00:22 +0000
Subject: [issue29218] distutils: Remove unused install_misc class
In-Reply-To: <1484002873.56.0.81761190557.issue29218@psf.upfronthosting.co.za>
Message-ID: <1485914422.76.0.0428207084025.issue29218@psf.upfronthosting.co.za>


Changes by Roundup Robot <devnull at psf.upfronthosting.co.za>:


_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29218>
_______________________________________

From report at bugs.python.org  Tue Jan 31 21:22:42 2017
From: report at bugs.python.org (paul j3)
Date: Wed, 01 Feb 2017 02:22:42 +0000
Subject: [issue23298] Add ArgumentParser.add_mutually_dependence_group
In-Reply-To: <1421905312.49.0.407798450784.issue23298@psf.upfronthosting.co.za>
Message-ID: <1485915762.35.0.907095303294.issue23298@psf.upfronthosting.co.za>


paul j3 added the comment:

I propose closing this with reference to http://bugs.python.org/issue11588

----------
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23298>
_______________________________________

From report at bugs.python.org  Tue Jan 31 21:24:54 2017
From: report at bugs.python.org (paul j3)
Date: Wed, 01 Feb 2017 02:24:54 +0000
Subject: [issue11588] Add "necessarily inclusive" groups to argparse
In-Reply-To: <1300377092.03.0.159307937388.issue11588@psf.upfronthosting.co.za>
Message-ID: <1485915894.68.0.934936285946.issue11588@psf.upfronthosting.co.za>


paul j3 added the comment:

Another issue requesting a 'mutually dependent group'
http://bugs.python.org/issue23298

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue11588>
_______________________________________

From report at bugs.python.org  Tue Jan 31 21:32:58 2017
From: report at bugs.python.org (paul j3)
Date: Wed, 01 Feb 2017 02:32:58 +0000
Subject: [issue24736] argparse add_mutually_exclusive_group do not print help
In-Reply-To: <1438001526.73.0.776043102355.issue24736@psf.upfronthosting.co.za>
Message-ID: <1485916378.81.0.573760947878.issue24736@psf.upfronthosting.co.za>


paul j3 added the comment:

I'm proposing closing this with reference to http://bugs.python.org/issue22047 

That focuses on the issue of adding mutually exclusive group to another exclusive group, but adding Argument Group has the same problems.

Solutions, short of the big http://bugs.python.org/issue11588 fix, include code that raises an error the user attempts to nest groups and/or adding a documentation note that nesting does not work.

There is one kind of nesting that does work - a mutually exclusive group can be nested inside an Argument Group.  The effect is to give the mutually exclusive group a title and help line grouping.

----------
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24736>
_______________________________________

From report at bugs.python.org  Tue Jan 31 21:34:00 2017
From: report at bugs.python.org (paul j3)
Date: Wed, 01 Feb 2017 02:34:00 +0000
Subject: [issue22047] argparse improperly prints mutually exclusive options
 when they are in a group
In-Reply-To: <1406131424.14.0.0235697032218.issue22047@psf.upfronthosting.co.za>
Message-ID: <1485916440.41.0.249521478107.issue22047@psf.upfronthosting.co.za>


paul j3 added the comment:

A similar issue about nesting an Argument Group inside a Mutually Exclusive Group.

http://bugs.python.org/issue24736

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue22047>
_______________________________________

From report at bugs.python.org  Tue Jan 31 22:06:20 2017
From: report at bugs.python.org (Eryk Sun)
Date: Wed, 01 Feb 2017 03:06:20 +0000
Subject: [issue29409] Implement PEP 529 for io.FileIO
Message-ID: <1485918380.71.0.564330613462.issue29409@psf.upfronthosting.co.za>


New submission from Eryk Sun:

PEP 529 isn't implemented for io.FileIO, and I think it should be. If a UTF-8 path is passed to open(), it ends up calling C _open instead of decoding the path and calling C _wopen. Also, if a pathlike object is passed to io.FileIO, it calls PyUnicode_FSConverter on it, converting it to UTF-8 and then passing it to _open. For example:

    >>> p = r'C:\Temp\????'
    >>> os.path.exists(p)
    True

    >>> open(p.encode())
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    FileNotFoundError: [Errno 2] No such file or directory: b'C:\\Temp\\\xce\xb1\xce\xb2\xcf\x88\xce\xb4'

    >>> io.FileIO(pathlib.Path(p))
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    FileNotFoundError: [Errno 2] No such file or directory: WindowsPath('C:/Temp/????')

The Windows implementation should mirror the POSIX implementation via PyUnicode_FSDecoder.

----------
components: IO, Windows
messages: 286585
nosy: eryksun, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
stage: needs patch
status: open
title: Implement PEP 529 for io.FileIO
type: behavior
versions: Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29409>
_______________________________________

From report at bugs.python.org  Tue Jan 31 22:08:09 2017
From: report at bugs.python.org (Eryk Sun)
Date: Wed, 01 Feb 2017 03:08:09 +0000
Subject: [issue29409] Implement PEP 529 for io.FileIO
In-Reply-To: <1485918380.71.0.564330613462.issue29409@psf.upfronthosting.co.za>
Message-ID: <1485918489.43.0.629080925511.issue29409@psf.upfronthosting.co.za>


Changes by Eryk Sun <eryksun at gmail.com>:


----------
keywords: +patch
Added file: http://bugs.python.org/file46473/issue_29409_01.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29409>
_______________________________________

From report at bugs.python.org  Tue Jan 31 22:08:32 2017
From: report at bugs.python.org (Eryk Sun)
Date: Wed, 01 Feb 2017 03:08:32 +0000
Subject: [issue29409] Implement PEP 529 for io.FileIO
In-Reply-To: <1485918380.71.0.564330613462.issue29409@psf.upfronthosting.co.za>
Message-ID: <1485918512.28.0.121648390351.issue29409@psf.upfronthosting.co.za>


Changes by Eryk Sun <eryksun at gmail.com>:


----------
stage: needs patch -> test needed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29409>
_______________________________________

From report at bugs.python.org  Tue Jan 31 23:35:25 2017
From: report at bugs.python.org (Nikolay Kim)
Date: Wed, 01 Feb 2017 04:35:25 +0000
Subject: [issue29406] asyncio SSL contexts leak sockets after calling close
 with certain Apache servers
In-Reply-To: <1485909545.04.0.650924943503.issue29406@psf.upfronthosting.co.za>
Message-ID: <1485923725.08.0.730113121415.issue29406@psf.upfronthosting.co.za>


Nikolay Kim added the comment:

madis-data.ncep.noaa.gov side does not complete ssl shutdown process.

----------
keywords: +patch
nosy: +fafhrd91
Added file: http://bugs.python.org/file46474/ssl_shutdown_timeout.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29406>
_______________________________________

From report at bugs.python.org  Tue Jan 31 23:35:54 2017
From: report at bugs.python.org (Nikolay Kim)
Date: Wed, 01 Feb 2017 04:35:54 +0000
Subject: [issue29406] asyncio SSL contexts leak sockets after calling close
 with certain Apache servers
In-Reply-To: <1485909545.04.0.650924943503.issue29406@psf.upfronthosting.co.za>
Message-ID: <1485923754.51.0.830571572218.issue29406@psf.upfronthosting.co.za>


Changes by Nikolay Kim <fafhrd91 at gmail.com>:


----------
versions: +Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29406>
_______________________________________

From report at bugs.python.org  Tue Jan 31 23:36:18 2017
From: report at bugs.python.org (George King)
Date: Wed, 01 Feb 2017 04:36:18 +0000
Subject: [issue29400] Instruction level tracing via sys.settrace
In-Reply-To: <1485886852.16.0.854514177291.issue29400@psf.upfronthosting.co.za>
Message-ID: <1485923778.82.0.308744762474.issue29400@psf.upfronthosting.co.za>


George King added the comment:

Here is the patch from git; if you need a patch for hg I can work on that tomorrow!

----------
keywords: +patch
Added file: http://bugs.python.org/file46475/inst-tracing.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29400>
_______________________________________