From report at bugs.python.org  Wed Apr  1 00:26:26 2015
From: report at bugs.python.org (Dan)
Date: Tue, 31 Mar 2015 22:26:26 +0000
Subject: [New-bugs-announce] [issue23833] email.header.Header folding
	modifies headers that	include semi-colons
Message-ID: <1427840786.4.0.777280344269.issue23833@psf.upfronthosting.co.za>


New submission from Dan:

When adding or replacing a header with email.header.Header, folding of long lines will add a space after any semi-colon encountered in the string. 

Setting maxlinelen to something longer than the string (no folding required), no spaces are added after semi-colons.

----------
components: email
messages: 239755
nosy: barry, dseg, r.david.murray
priority: normal
severity: normal
status: open
title: email.header.Header folding modifies headers that include semi-colons
type: behavior
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23833>
_______________________________________

From report at bugs.python.org  Wed Apr  1 00:55:06 2015
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 31 Mar 2015 22:55:06 +0000
Subject: [New-bugs-announce] [issue23834] socketmodule.c: add sock_call() to
	fix how the timeout	is recomputed
Message-ID: <1427842506.28.0.704301607448.issue23834@psf.upfronthosting.co.za>


New submission from STINNER Victor:

With the PEP 475, the BEGIN_SELECT_LOOP and END_SELECT_LOOP macros of Modules/socketmodule.c became complex. Moreover, they are misused to handle EINTR. Most functions currently reimplemented their own loop inside the existing "select loop", and so the timeout is not recomputed correctly.

Attached patch replaces the two macros with a new sock_call() function which takes a function as a parameter. IMO, the new code (sock_xxx_impl functions) is simpler to read, understand and debug.

I hate debugging code (especially in gdb) using complex macros.

Copy of sock_call() documentation:

/* Call a socket function.

   If the socket has a timeout, wait until the socket is ready before calling
   the function: wait until the socket is writable if writing is nonzero, wait
   until the socket received data otherwise.

   If the function is interrupted by a signal (failed with EINTR): retry the
   function, except if the signal handler raised an exception (PEP 475).

   When the function is retried, recompute the timeout using a monotonic clock.

   Raise an exception and return -1 on error, return 0 on success. */

I was surprised by the number of lines of code. It probably means that we are solving a non trivial problem: calling correctly socket functions with a timeout and retrying when interrupted by a signal.

----------
files: sock_call.patch
keywords: patch
messages: 239758
nosy: haypo, neologix, pitrou, serhiy.storchaka
priority: normal
severity: normal
status: open
title: socketmodule.c: add sock_call() to fix how the timeout is recomputed
versions: Python 3.5
Added file: http://bugs.python.org/file38769/sock_call.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23834>
_______________________________________

From report at bugs.python.org  Wed Apr  1 06:23:08 2015
From: report at bugs.python.org (James Tocknell)
Date: Wed, 01 Apr 2015 04:23:08 +0000
Subject: [New-bugs-announce] [issue23835] configparser does not convert
	defaults to strings
Message-ID: <1427862188.44.0.00358189102447.issue23835@psf.upfronthosting.co.za>


New submission from James Tocknell:

ConfigParser(defaults={1:2.4}) and ConfigParser(defaults={"a":5.2}) cause an exception when configparser tries to perform string operations on 1 and 5.2. I didn't see it documented that defaults must only contain strings, and using ConfigParser['DEFAULT'] = {1:2.4} does the necessary string conversion, so it looks like there should be some conversion done to the defaults.

----------
components: Library (Lib)
messages: 239768
nosy: aragilar
priority: normal
severity: normal
status: open
title: configparser does not convert defaults to strings
type: crash
versions: Python 2.7, Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23835>
_______________________________________

From report at bugs.python.org  Wed Apr  1 11:43:59 2015
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 01 Apr 2015 09:43:59 +0000
Subject: [New-bugs-announce] [issue23836] PEP 475: Enhance faulthandler to
	handle better reentrant	calls
Message-ID: <1427881439.29.0.554454019096.issue23836@psf.upfronthosting.co.za>


New submission from STINNER Victor:

While working on the PEP 475, I noticed that the faulthandler module doesn't handle well reentrant calls to its signal handlers. If a signal is received while faulthandler is displaying the traceback, faulthandler restarts to display a new traceback (and then finish to display the previous traceback).

Moreover, faulthandler uses the C function write() to display the traceback and it ignores write() errors. It should retry write() if write() is interrupted by a signal (fails with EINTR).

Attached py_write_noraise.patch:

- Add a new _Py_write_noraise() function
- Document functions releasing the GIL

Attached traceback_eintr.patch:

- Modify functions displaying traceback to use _Py_write_noraise() instead of write()

Attached faulthandler_eintr.patch:

- Add a reentrant flag to the two signal handlers of faulthandler
- Modify faulthandler.dump_traceback() to call PyErr_CheckSignals(), to call the Python signal handler if a signal was received while faulthandler was displaying the traceback of all Python threads

----------
files: py_write_noraise.patch
keywords: patch
messages: 239776
nosy: haypo
priority: normal
severity: normal
status: open
title: PEP 475: Enhance faulthandler to handle better reentrant calls
versions: Python 3.4, Python 3.5
Added file: http://bugs.python.org/file38775/py_write_noraise.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23836>
_______________________________________

From report at bugs.python.org  Wed Apr  1 12:06:02 2015
From: report at bugs.python.org (=?utf-8?q?Mathias_Fr=C3=B6jdman?=)
Date: Wed, 01 Apr 2015 10:06:02 +0000
Subject: [New-bugs-announce] [issue23837] read pipe transport tries to
	resume reading after loop	is gone
Message-ID: <1427882762.68.0.794378524046.issue23837@psf.upfronthosting.co.za>


New submission from Mathias Fr?jdman:

Script attached which reproduces the issue.

Steps to reproduce:
0) Use python 3.4.3 on Linux. Does not repro with 3.4.0 or 3.4.2.
1) Create a child process with asyncio.create_child_exec and stdout=PIPE
2) loop yield from child.read(n) (i used n=2048, any other positive value < StreamRead._limit ought to work too)
3) Write >= StreamRead._limit * 2 + 1 (by default 2**17+1) bytes from child process and exit

File referenced below: asyncio/streams.py

feed_data is called when data arrives from the child process. Having more than 2 * self._limit bytes in self._buffer will lead to StreamReader pausing reading on line 372.

feed_eof sets self._eof to True, but that value is not checked in relevant sections later.

Child process exits, which will lead to self._loop = None being set apparently on line 405 of _UnixReadPipeTransport._call_connection_lost in asyncio/unix_events.py (could not find any other location where it would be set to None).

After a number of read()s, self._maybe_resume_transport() is called when len(self._buffer) <= self._limit (ie. <= 64k left in buffer). self._paused will evaluate true too, so self._transport.resume_reading() is called on line 349.

That will call self._loop.add_reader(self._fileno, self._read_ready) on line 364 of asyncio/unix_events.py. self._loop is None, so and AttributeError is raised when None has no add_reader attribute:

Traceback (most recent call last):
  File "child_reader.py", line 29, in <module>
    print('read {} bytes from child'.format(loop.run_until_complete(fail())))
  File "/home/frojma/python-3.4.3/lib/python3.4/asyncio/base_events.py", line 316, in run_until_complete
    return future.result()
  File "/home/frojma/python-3.4.3/lib/python3.4/asyncio/futures.py", line 275, in result
    raise self._exception
  File "/home/frojma/python-3.4.3/lib/python3.4/asyncio/tasks.py", line 238, in _step
    result = next(coro)
  File "child_reader.py", line 15, in fail
    chunk = yield from child.stdout.read(2048)
  File "/home/frojma/python-3.4.3/lib/python3.4/asyncio/streams.py", line 478, in read
    self._maybe_resume_transport()
  File "/home/frojma/python-3.4.3/lib/python3.4/asyncio/streams.py", line 354, in _maybe_resume_transport
    self._transport.resume_reading()
  File "/home/frojma/python-3.4.3/lib/python3.4/asyncio/unix_events.py", line 364, in resume_reading
    self._loop.add_reader(self._fileno, self._read_ready)

----------
components: asyncio
files: child_reader.py
messages: 239779
nosy: gvanrossum, haypo, mwf, yselivanov
priority: normal
severity: normal
status: open
title: read pipe transport tries to resume reading after loop is gone
type: crash
versions: Python 3.4
Added file: http://bugs.python.org/file38778/child_reader.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23837>
_______________________________________

From report at bugs.python.org  Wed Apr  1 13:24:24 2015
From: report at bugs.python.org (Serhiy Storchaka)
Date: Wed, 01 Apr 2015 11:24:24 +0000
Subject: [New-bugs-announce] [issue23838] linecache and MemoryError
Message-ID: <1427887464.14.0.986179672993.issue23838@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka:

When format a traceback of MemoryError, linecache tries to read all lines of source file and likely raises a MemoryError. For example:

http://buildbot.python.org/all/builders/x86%20OpenIndiana%203.x/builds/9712/steps/test/logs/stdio

Traceback (most recent call last):
  File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/test/regrtest.py", line 1280, in runtest_inner
    test_runner()
  File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/test/test_socket.py", line 5340, in test_main
    support.run_unittest(*tests)
  File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/test/support/__init__.py", line 1808, in run_unittest
    _run_suite(suite)
  File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/test/support/__init__.py", line 1774, in _run_suite
    result = runner.run(suite)
  File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/unittest/runner.py", line 176, in run
    test(result)
  File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/unittest/suite.py", line 84, in __call__
    return self.run(*args, **kwds)
  File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/unittest/suite.py", line 122, in run
    test(result)
  File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/unittest/suite.py", line 84, in __call__
    return self.run(*args, **kwds)
  File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/unittest/suite.py", line 122, in run
    test(result)
  File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/unittest/case.py", line 625, in __call__
    return self.run(*args, **kwds)
  File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/unittest/case.py", line 585, in run
    self._feedErrorsToResult(result, outcome.errors)
  File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/unittest/case.py", line 518, in _feedErrorsToResult
    result.addError(test, exc_info)
  File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/unittest/runner.py", line 67, in addError
    super(TextTestResult, self).addError(test, err)
  File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/unittest/result.py", line 17, in inner
    return method(self, *args, **kw)
  File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/unittest/result.py", line 115, in addError
    self.errors.append((test, self._exc_info_to_string(err, test)))
  File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/unittest/result.py", line 186, in _exc_info_to_string
    exctype, value, tb, limit=length, capture_locals=self.tb_locals)
  File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/traceback.py", line 456, in __init__
    capture_locals=capture_locals)
  File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/traceback.py", line 341, in extract
    f.line
  File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/traceback.py", line 270, in line
    self._line = linecache.getline(self.filename, self.lineno).strip()
  File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/linecache.py", line 16, in getline
    lines = getlines(filename, module_globals)
  File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/linecache.py", line 44, in getlines
    return updatecache(filename, module_globals)
  File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/linecache.py", line 134, in updatecache
    lines = fp.readlines()
MemoryError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/test/regrtest.py", line 532, in main
    result = runtest(*args, **kwargs)
  File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/test/regrtest.py", line 967, in runtest
    display_failure=False)
  File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/test/regrtest.py", line 1304, in runtest_inner
    msg = traceback.format_exc()
  File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/traceback.py", line 161, in format_exc
    return "".join(format_exception(*sys.exc_info(), limit=limit, chain=chain))
  File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/traceback.py", line 115, in format_exception
    type(value), value, tb, limit=limit).format(chain=chain))
  File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/traceback.py", line 456, in __init__
    capture_locals=capture_locals)
  File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/traceback.py", line 341, in extract
    f.line
  File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/traceback.py", line 270, in line
    self._line = linecache.getline(self.filename, self.lineno).strip()
  File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/linecache.py", line 16, in getline
    lines = getlines(filename, module_globals)
  File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/linecache.py", line 44, in getlines
    return updatecache(filename, module_globals)
  File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/linecache.py", line 134, in updatecache
    lines = fp.readlines()
  File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/codecs.py", line 321, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
MemoryError
Traceback (most recent call last):
  File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/runpy.py", line 170, in _run_module_as_main
    "__main__", mod_spec)
  File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/test/__main__.py", line 3, in <module>
    regrtest.main_in_temp_cwd()
  File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/test/regrtest.py", line 1564, in main_in_temp_cwd
    main()
  File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/test/regrtest.py", line 738, in main
    raise Exception("Child error on {}: {}".format(test, result[1]))
Exception: Child error on test_socket: 
make: *** [buildbottest] Error 1
program finished with exit code 2

Proposed patch clears the cache and repeats an attempt to read source file, and returns empty result if the second attempt fails (as when the source file can't be read).

I hesitate how to classify this issue. On one hand, it looks as new feature, on other hand it is related to testing on all versions.

----------
components: Library (Lib)
files: linecache_memoryerror.patch
keywords: patch
messages: 239781
nosy: ezio.melotti, haypo, michael.foord, pitrou, serhiy.storchaka
priority: normal
severity: normal
stage: patch review
status: open
title: linecache and MemoryError
Added file: http://bugs.python.org/file38779/linecache_memoryerror.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23838>
_______________________________________

From report at bugs.python.org  Wed Apr  1 13:30:41 2015
From: report at bugs.python.org (Serhiy Storchaka)
Date: Wed, 01 Apr 2015 11:30:41 +0000
Subject: [New-bugs-announce] [issue23839] Clear caches after every test
Message-ID: <1427887841.84.0.393130432881.issue23839@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka:

Some caches can grow unlimitedly during running the testsuite (in particular linecache). On some machines with limited resources this can cause multiple MemoryErrors. Many of these MemoryErrors can be avoided if clear caches between tests. Lib/test/regrtest.py already contains a code for cleaning caches, but it is used only with the -R option.

----------
components: Tests
messages: 239782
nosy: ezio.melotti, haypo, michael.foord, pitrou, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Clear caches after every test
type: resource usage
versions: Python 2.7, Python 3.4, Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23839>
_______________________________________

From report at bugs.python.org  Wed Apr  1 13:52:33 2015
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 01 Apr 2015 11:52:33 +0000
Subject: [New-bugs-announce] [issue23840] tokenize.open() leaks an open
	binary file on	TextIOWrapper error
Message-ID: <1427889153.26.0.649877902408.issue23840@psf.upfronthosting.co.za>


New submission from STINNER Victor:

If TextIOWrapper constructor fails in tokenize.open(), the open binary file is not closed and a ResourceWarning is emited.

Try attached patch:

$ python3 -Wd ~/tokenize_open_bug.py
tokenize.open failed: unknown encoding for 'test.py': xxx
/home/haypo/tokenize_open_bug.py:13: ResourceWarning: unclosed file <_io.BufferedReader name='test.py'>
  print("tokenize.open failed: %s" % err)

The fix is quite simple: add "try: ... except: buffer.close(); raise". If someone wants to fix this issue, an unit test must be added, test based on my script and ensures that the buffer is closed (ex: mock tokenize._builtin_open and checks that close() was called).

----------
files: tokenize_open_bug.py
keywords: easy
messages: 239786
nosy: haypo
priority: normal
severity: normal
status: open
title: tokenize.open() leaks an open binary file on TextIOWrapper error
versions: Python 3.4, Python 3.5
Added file: http://bugs.python.org/file38781/tokenize_open_bug.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23840>
_______________________________________

From report at bugs.python.org  Wed Apr  1 14:01:47 2015
From: report at bugs.python.org (sorin)
Date: Wed, 01 Apr 2015 12:01:47 +0000
Subject: [New-bugs-announce] [issue23841] py34 OrderedDict is using weakref
	for root reference
Message-ID: <1427889707.75.0.323875541647.issue23841@psf.upfronthosting.co.za>


New submission from sorin:

The standard library implementation of an OrderedDict (which is included in Python 3.4) uses a weakref for the root reference. 
https://hg.python.org/cpython/file/534b26837a13/Lib/collections/__init__.py#l74

This was reported initially on https://github.com/kennethreitz/requests/issues/2303 and at the moment there are no known workarounds.

----------
components: Library (Lib)
messages: 239789
nosy: sorin
priority: normal
severity: normal
status: open
title: py34 OrderedDict is using weakref for root reference
type: compile error
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23841>
_______________________________________

From report at bugs.python.org  Wed Apr  1 18:17:23 2015
From: report at bugs.python.org (Matthias Klose)
Date: Wed, 01 Apr 2015 16:17:23 +0000
Subject: [New-bugs-announce] [issue23842] SystemError:
	../Objects/longobject.c:998: bad argument	to internal function
Message-ID: <1427905043.2.0.434822440378.issue23842@psf.upfronthosting.co.za>


New submission from Matthias Klose:

an unreleased version of cinder fails a test when run with python 2.7 from the branch.  I don't have a test case yet, but looking at the error message and list of fixed issues after the 2.7.9 release, the fix for issue #23098 could be the culprit.

FAIL: cinder.tests.test_utils.GetBlkdevMajorMinorTestCase.test_get_blkdev_major_minor_file
cinder.tests.test_utils.GetBlkdevMajorMinorTestCase.test_get_blkdev_major_minor_file
----------------------------------------------------------------------
_StringException: Traceback (most recent call last):
  File "/build/buildd/cinder-2015.1~b3/cinder/tests/test_utils.py", line 721, in test_get_blkdev_major_minor_file
    dev = self._test_get_blkdev_major_minor_file('/dev/made_up_disk1')
  File "/usr/lib/python2.7/dist-packages/mock.py", line 1210, in patched
    return func(*args, **keywargs)
  File "/build/buildd/cinder-2015.1~b3/cinder/tests/test_utils.py", line 712, in _test_get_blkdev_major_minor_file
    dev = utils.get_blkdev_major_minor(test_file)
  File "/build/buildd/cinder-2015.1~b3/cinder/utils.py", line 656, in get_blkdev_major_minor
    return get_blkdev_major_minor(devpath, False)
  File "/build/buildd/cinder-2015.1~b3/cinder/utils.py", line 645, in get_blkdev_major_minor
    return '%d:%d' % (os.major(st.st_rdev), os.minor(st.st_rdev))
SystemError: ../Objects/longobject.c:998: bad argument to internal function
Traceback (most recent call last):
_StringException: Empty attachments:
  stderr
  stdout

----------
components: Extension Modules
messages: 239826
nosy: doko, serhiy.storchaka
priority: normal
severity: normal
status: open
title: SystemError: ../Objects/longobject.c:998: bad argument to internal function
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23842>
_______________________________________

From report at bugs.python.org  Wed Apr  1 20:32:18 2015
From: report at bugs.python.org (John Nagle)
Date: Wed, 01 Apr 2015 18:32:18 +0000
Subject: [New-bugs-announce] [issue23843] ssl.wrap_socket doesn't handle
	virtual TLS hosts
Message-ID: <1427913138.81.0.946174755254.issue23843@psf.upfronthosting.co.za>


New submission from John Nagle:

ssl.wrap_socket() always uses the SSL certificate associated with the raw IP address, rather than using the server_host feature of TLS. Even when wrap_socket is used before calling "connect(port, host)", the "host" parameter isn't used by TLS.

To get proper TLS behavior (which only works in recent Python versions), it's necessary to create an SSLContext, then use

context.wrap_socket(sock, server_hostname="example.com")

This behavior is backwards-compatible (the SSL module didn't talk TLS until very recently) but confusing.  The documentation does not reflect this difference.  There's a lot of old code and online advice which suggests using ssl.wrap_socket().  It works until you hit a virtual host with TLS support. Then you get the wrong server cert and an unexpected "wrong host" SSL error.

Possible fixes:

1. Deprecate ssl.wrap_socket(), and modify the documentation to tell users to always use context.wrap_socket().

2. Add a "server_hostname" parameter to ssl.wrap_socket().  It doesn't accept that parameter; only context.wrap_socket() does.  Modify documentation accordingly.

----------
assignee: docs at python
components: Documentation, Library (Lib)
messages: 239834
nosy: docs at python, nagle
priority: normal
severity: normal
status: open
title: ssl.wrap_socket doesn't handle virtual TLS hosts
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23843>
_______________________________________

From report at bugs.python.org  Wed Apr  1 22:53:33 2015
From: report at bugs.python.org (=?utf-8?q?C=C3=A9dric_Krier?=)
Date: Wed, 01 Apr 2015 20:53:33 +0000
Subject: [New-bugs-announce] [issue23844] test_ssl: fails on recent libressl
	version with	BAD_DH_P_LENGTH
Message-ID: <1427921613.92.0.742190312016.issue23844@psf.upfronthosting.co.za>


New submission from C?dric Krier:

Since [1], libressl fails if DH keys are too small (<1024).
Here is a patch to increase the dh keys to 1024 for the tests.

[1] http://cvsweb.openbsd.org/cgi-bin/cvsweb/src/lib/libssl/src/ssl/s3_clnt.c?rev=1.108&content-type=text/x-cvsweb-markup

----------
files: dh1024.patch
keywords: patch
messages: 239846
nosy: ced
priority: normal
severity: normal
status: open
title: test_ssl: fails on recent libressl version with BAD_DH_P_LENGTH
Added file: http://bugs.python.org/file38791/dh1024.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23844>
_______________________________________

From report at bugs.python.org  Thu Apr  2 00:21:02 2015
From: report at bugs.python.org (=?utf-8?q?C=C3=A9dric_Krier?=)
Date: Wed, 01 Apr 2015 22:21:02 +0000
Subject: [New-bugs-announce] [issue23845] test_ssl: fails on recent libressl
	with	SSLV3_ALERT_HANDSHAKE_FAILURE
Message-ID: <1427926862.2.0.277939784676.issue23845@psf.upfronthosting.co.za>


New submission from C?dric Krier:

SSLv3 has been deactivated by default [1], as stated in the commit message it can be reactivated by clearing the option. So here is a patch that reactivate it in the test when needed.

[1] http://marc.info/?l=openbsd-cvs&m=141339479327258&w=2

----------
files: sslv3.patch
keywords: patch
messages: 239856
nosy: ced
priority: normal
severity: normal
status: open
title: test_ssl: fails on recent libressl with SSLV3_ALERT_HANDSHAKE_FAILURE
Added file: http://bugs.python.org/file38794/sslv3.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23845>
_______________________________________

From report at bugs.python.org  Thu Apr  2 06:00:18 2015
From: report at bugs.python.org (SeungHyun.Hwang)
Date: Thu, 02 Apr 2015 04:00:18 +0000
Subject: [New-bugs-announce] [issue23846] asyncio : ProactorEventLoop raised
	BlockingIOError when	ThreadPoolExecutor has many workers
Message-ID: <1427947218.56.0.749215014794.issue23846@psf.upfronthosting.co.za>


New submission from SeungHyun.Hwang:

I tested on python3.4.2, windows 7

1. Only proactorEvent issued.
2. ThreadPoolExecutor has many workers (in the attached example file, worker count 20,000)
3. The loop does run_in_executor more call than worker count (in the attached example file, 40,000 calls)
4. After some random seconds, raise BlockingIOError

BlockingIOError: [WinError 10035] A non-blocking socket operation could not be completed immediately.
exception calling callback for <Future at 0x1ab89ef0 state=finished returned NoneType>
Traceback (most recent call last):
  File "c:\Python342\Lib\concurrent\futures\_base.py", line 297, in _invoke_callbacks
    callback(self)
  File "c:\Python342\Lib\asyncio\futures.py", line 410, in <lambda>
    new_future._copy_state, fut))
  File "c:\Python342\Lib\asyncio\base_events.py", line 403, in call_soon_threadsafe
    self._write_to_self()
  File "c:\Python342\Lib\asyncio\proactor_events.py", line 449, in _write_to_self
    self._csock.send(b'\0')


I guess that proactor's _write_to_self method misses exception handle.

proactor_events.py
def _write_to_self(self):
        self._csock.send(b'\0')


selector_events.py
def _write_to_self(self):
        # This may be called from a different thread, possibly after
        # _close_self_pipe() has been called or even while it is
        # running.  Guard for self._csock being None or closed.  When
        # a socket is closed, send() raises OSError (with errno set to
        # EBADF, but let's not rely on the exact error code).
        csock = self._csock
        if csock is not None:
            try:
                csock.send(b'\0')
            except OSError:
                if self._debug:
                    logger.debug("Fail to write a null byte into the "
                                 "self-pipe socket",
                                 exc_info=True)


Ps: It's my first publication. Hope you understand my poor comment..

----------
components: asyncio
files: example_thread_executor.py
messages: 239872
nosy: gvanrossum, haypo, kernel0, yselivanov
priority: normal
severity: normal
status: open
title: asyncio : ProactorEventLoop raised BlockingIOError when ThreadPoolExecutor has many workers
type: crash
versions: Python 3.4
Added file: http://bugs.python.org/file38801/example_thread_executor.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23846>
_______________________________________

From report at bugs.python.org  Thu Apr  2 09:01:50 2015
From: report at bugs.python.org (Raymond Hettinger)
Date: Thu, 02 Apr 2015 07:01:50 +0000
Subject: [New-bugs-announce] [issue23847] Add xml pretty print option to
	ElementTree
Message-ID: <1427958110.33.0.788273316254.issue23847@psf.upfronthosting.co.za>


New submission from Raymond Hettinger:

The xml.dom.minidom package has as a xml.toprettyxml() function that has the problem of altering the whitespace of the text of elements.   The ElementTree module needs a prettify option that works better, perhaps something based of Effbot's code at:

http://effbot.org/zone/element-lib.htm#prettyprint

The lxml package also provides an API for this purpose:

    print(etree.tostring(root, pretty_print=True))

----------
messages: 239879
nosy: rhettinger
priority: normal
severity: normal
status: open
title: Add xml pretty print option to ElementTree
type: enhancement
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23847>
_______________________________________

From report at bugs.python.org  Thu Apr  2 10:15:59 2015
From: report at bugs.python.org (STINNER Victor)
Date: Thu, 02 Apr 2015 08:15:59 +0000
Subject: [New-bugs-announce] [issue23848] faulthandler: setup an exception
	handler on Windows
Message-ID: <1427962559.63.0.750474204561.issue23848@psf.upfronthosting.co.za>


New submission from STINNER Victor:

Attached patch setup an exception handler on Windows.

I wrote it when investigating a failure of test_exceptions related to the issue #22977. faulthandler was not trigerred while the program crashed.

I didn't test the patch yet (I tested a previous attempt, a little bit different). Maybe we can avoid setting signal handlers on Windows if an exception handler is set?

----------
components: Windows
files: faulthandler_exc_handler.patch
keywords: patch
messages: 239890
nosy: haypo, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: faulthandler: setup an exception handler on Windows
versions: Python 3.5
Added file: http://bugs.python.org/file38802/faulthandler_exc_handler.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23848>
_______________________________________

From report at bugs.python.org  Thu Apr  2 11:37:20 2015
From: report at bugs.python.org (Serhiy Storchaka)
Date: Thu, 02 Apr 2015 09:37:20 +0000
Subject: [New-bugs-announce] [issue23849] Leaks in test_deque
Message-ID: <1427967440.19.0.462388571219.issue23849@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka:

$ ./python -m test.regrtest -R 3:3:reflog test_deque
[1/1] test_deque
beginning 6 repetitions
123456
......
test_deque leaked [91, 91, 91] references, sum=273
test_deque leaked [21, 23, 23] memory blocks, sum=67
1 test failed:
    test_deque

----------
components: Tests
messages: 239894
nosy: rhettinger, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Leaks in test_deque
type: resource usage
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23849>
_______________________________________

From report at bugs.python.org  Thu Apr  2 12:09:33 2015
From: report at bugs.python.org (Giacomo Alzetta)
Date: Thu, 02 Apr 2015 10:09:33 +0000
Subject: [New-bugs-announce] [issue23850] Missing documentation for
	Py_TPFLAGS_HAVE_NEWBUFFER
Message-ID: <1427969373.72.0.801261679654.issue23850@psf.upfronthosting.co.za>


New submission from Giacomo Alzetta:

Python2.7 documentation is missing critical information regarding the backporting of the new-buffer protocol.
There is no mention whatsoever of the Py_TPFLAGS_HAVE_NEWBUFFER flag which is required to implement it. The only way to discover it is by reading the sources of the built-ins or stdlib.

----------
messages: 239895
nosy: Giacomo.Alzetta
priority: normal
severity: normal
status: open
title: Missing documentation for Py_TPFLAGS_HAVE_NEWBUFFER

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23850>
_______________________________________

From report at bugs.python.org  Thu Apr  2 13:54:34 2015
From: report at bugs.python.org (STINNER Victor)
Date: Thu, 02 Apr 2015 11:54:34 +0000
Subject: [New-bugs-announce] [issue23851] PEP 475: _posixsubprocess retries
	close() on EINTR
Message-ID: <1427975674.16.0.24545541866.issue23851@psf.upfronthosting.co.za>


New submission from STINNER Victor:

According to the PEP 475, the close() function must *not* be retried if it fails with EINTR:
- http://alobbs.com/post/54503240599/close-and-eintr
- http://lwn.net/Articles/576478/
- http://linux.derkeiler.com/Mailing-Lists/Kernel/2005-09/3000.html
- http://alobbs.com/post/54503240599/close-and-eintr

The _posixsubprocess retries close() when it fails with EINTR. Example:

    while (close(fd_num) < 0 && errno == EINTR);

It should be fixed.

----------
messages: 239907
nosy: haypo
priority: normal
severity: normal
status: open
title: PEP 475: _posixsubprocess retries close() on EINTR
versions: Python 2.7, Python 3.4, Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23851>
_______________________________________

From report at bugs.python.org  Thu Apr  2 17:13:52 2015
From: report at bugs.python.org (=?utf-8?q?C=C3=A9dric_Krier?=)
Date: Thu, 02 Apr 2015 15:13:52 +0000
Subject: [New-bugs-announce] [issue23852] Wrong FD_DIR file name on OpenBSD
Message-ID: <1427987632.4.0.790792340938.issue23852@psf.upfronthosting.co.za>


New submission from C?dric Krier:

The test_subprocess fails since issue21618 on OpenBSD because the FD_DIR is wrong (/dev/fd instead of /proc/self/fd).

----------
files: fd_dir.patch
keywords: patch
messages: 239920
nosy: ced
priority: normal
severity: normal
status: open
title: Wrong FD_DIR file name on OpenBSD
Added file: http://bugs.python.org/file38807/fd_dir.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23852>
_______________________________________

From report at bugs.python.org  Thu Apr  2 17:30:24 2015
From: report at bugs.python.org (STINNER Victor)
Date: Thu, 02 Apr 2015 15:30:24 +0000
Subject: [New-bugs-announce] [issue23853] PEP 475: handle EINTR in the ssl
	module
Message-ID: <1427988624.66.0.803600947043.issue23853@psf.upfronthosting.co.za>


New submission from STINNER Victor:

The _ssl module uses an helper function check_socket_and_wait_for_timeout() to poll until the socket is ready (got data or became writable). check_socket_and_wait_for_timeout() always uses the socket timeout. If select() or poll() is interrupted by a signal (fails with EINTR), check_socket_and_wait_for_timeout() is restarted with the same timeout, which doesn't respect the contract of the timeout: the operation must timeout if it takes more than timeout seconds.

The code must be modified to recompute the timeout, as done in the new sock_call() function of Modules/socketmodule.c (issue #23618). At least, the timeout must decreases when select()/poll() fails with EINTR.

Currently, the timeout is reset after each read/write/handshake operation. IMO the timeout must apply on the total duration of the ssl method, not be reset. But changing this may break backward compatibility :-/

Note: if the signal handler raises an exception, the ssl method fails with the exception. This issue is specific to signal handlers not raising an exception.

----------
messages: 239924
nosy: haypo
priority: normal
severity: normal
status: open
title: PEP 475: handle EINTR in the ssl module
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23853>
_______________________________________

From report at bugs.python.org  Thu Apr  2 19:50:06 2015
From: report at bugs.python.org (Ali Arar)
Date: Thu, 02 Apr 2015 17:50:06 +0000
Subject: [New-bugs-announce] [issue23854] qtconsole and all windows based
	python have issues	loading
Message-ID: <1427997006.27.0.45801285354.issue23854@psf.upfronthosting.co.za>


New submission from Ali Arar:

All of the sudden, all windows based python stopped working.  qtconsole and spyder.  Also, conda, pip, and easy_setup are issuing error messages.  Below is the feedback I am getting.  I uninstalled and reinstalled the package that was working perfectly fine.  Also, downloaded Anaconda-2.2.0-Windows-x86_64.exe.  

===================
ipython qtconsole:
-------------------
Error in sys.excepthook:
Traceback (most recent call last):
  File "D:\Anaconda\lib\site-packages\IPython\qt\console\qtconsoleapp.py", line 45, in gui_excepthook
    old_excepthook(exctype, value, tb)
TypeError: 'NoneType' object is not callable

Original exception was:
Traceback (most recent call last):
  File "D:\Anaconda\Scripts\ipython-script.py", line 5, in <module>
    sys.exit(start_ipython())
  File "D:\Anaconda\lib\site-packages\IPython\__init__.py", line 120, in start_ipython
    return launch_new_instance(argv=argv, **kwargs)
  File "D:\Anaconda\lib\site-packages\IPython\config\application.py", line 573, in launch_instance
    app.initialize(argv)
  File "<string>", line 2, in initialize
  File "D:\Anaconda\lib\site-packages\IPython\config\application.py", line 75, in catch_config_error
    return method(app, *args, **kwargs)
  File "D:\Anaconda\lib\site-packages\IPython\terminal\ipapp.py", line 321, in initialize
    super(TerminalIPythonApp, self).initialize(argv)
  File "<string>", line 2, in initialize
  File "D:\Anaconda\lib\site-packages\IPython\config\application.py", line 75, in catch_config_error
    return method(app, *args, **kwargs)
  File "D:\Anaconda\lib\site-packages\IPython\core\application.py", line 369, in initialize
    self.parse_command_line(argv)
  File "D:\Anaconda\lib\site-packages\IPython\terminal\ipapp.py", line 316, in parse_command_line
    return super(TerminalIPythonApp, self).parse_command_line(argv)
  File "<string>", line 2, in parse_command_line
  File "D:\Anaconda\lib\site-packages\IPython\config\application.py", line 75, in catch_config_error
    return method(app, *args, **kwargs)
  File "D:\Anaconda\lib\site-packages\IPython\config\application.py", line 471, in parse_command_line
    return self.initialize_subcommand(subc, subargv)
  File "<string>", line 2, in initialize_subcommand
  File "D:\Anaconda\lib\site-packages\IPython\config\application.py", line 75, in catch_config_error
    return method(app, *args, **kwargs)
  File "D:\Anaconda\lib\site-packages\IPython\config\application.py", line 402, in initialize_subcommand
    subapp = import_item(subapp)
  File "D:\Anaconda\lib\site-packages\IPython\utils\importstring.py", line 42, in import_item
    module = __import__(package, fromlist=[obj])
  File "D:\Anaconda\lib\site-packages\IPython\qt\console\qtconsoleapp.py", line 56, in <module>
    from IPython.qt.console.ipython_widget import IPythonWidget
  File "D:\Anaconda\lib\site-packages\IPython\qt\console\ipython_widget.py", line 23, in <module>
    from .frontend_widget import FrontendWidget
  File "D:\Anaconda\lib\site-packages\IPython\qt\console\frontend_widget.py", line 25, in <module>
    from .pygments_highlighter import PygmentsHighlighter
  File "D:\Anaconda\lib\site-packages\IPython\qt\console\pygments_highlighter.py", line 3, in <module>
    from pygments.formatters.html import HtmlFormatter
  File "D:\Anaconda\lib\site-packages\pygments\formatters\__init__.py", line 19, in <module>
    from pygments.plugin import find_plugin_formatters
  File "D:\Anaconda\lib\site-packages\pygments\plugin.py", line 39, in <module>
    import pkg_resources
  File "D:\Anaconda\lib\site-packages\setuptools-14.3-py2.7.egg\pkg_resources\__init__.py", line 1366, in <module>

  File "D:\Anaconda\lib\site-packages\setuptools-14.3-py2.7.egg\pkg_resources\__init__.py", line 1370, in MarkerEvaluation

AttributeError: 'module' object has no attribute 'python_version'
===============================================

D:\>pip
---------
Traceback (most recent call last):
  File "D:\Anaconda\Scripts\pip-script.py", line 3, in <module>
    from pip import main
  File "D:\Anaconda\lib\site-packages\pip\__init__.py", line 13, in <module>
    from pip.utils import get_installed_distributions, get_prog
  File "D:\Anaconda\lib\site-packages\pip\utils\__init__.py", line 22, in <module>
    from pip._vendor import pkg_resources, six
  File "D:\Anaconda\lib\site-packages\pip\_vendor\__init__.py", line 61, in load_module
    __import__(name)
  File "D:\Anaconda\lib\site-packages\pip\_vendor\pkg_resources\__init__.py", line 1316, in <module>
    class MarkerEvaluation(object):
  File "D:\Anaconda\lib\site-packages\pip\_vendor\pkg_resources\__init__.py", line 1320, in MarkerEvaluation
    'python_full_version': platform.python_version,
AttributeError: 'module' object has no attribute 'python_version'
========================================================

D:\>conda
----------
Traceback (most recent call last):
  File "D:\Anaconda\Scripts\conda-script.py", line 2, in <module>
    from conda.cli import main
  File "D:\Anaconda\lib\site-packages\conda\cli\__init__.py", line 8, in <module>
    from conda.cli.main import main
  File "D:\Anaconda\lib\site-packages\conda\cli\main.py", line 45, in <module>
    from conda.cli import common
  File "D:\Anaconda\lib\site-packages\conda\cli\common.py", line 11, in <module>
    import conda.config as config
  File "D:\Anaconda\lib\site-packages\conda\config.py", line 12, in <module>
    from platform import machine
ImportError: cannot import name machine

====================================================

D:\>spyder
----------
'module' object has no attribute 'python_version'
Traceback (most recent call last):
  File "D:\Anaconda\Scripts\spyder-script.py", line 1, in <module>
    from spyderlib import start_app
  File "D:\Anaconda\lib\site-packages\spyderlib\start_app.py", line 14, in <module>
    from spyderlib.config import CONF
  File "D:\Anaconda\lib\site-packages\spyderlib\config.py", line 22, in <module>
    from spyderlib.utils import iofuncs, codeanalysis
  File "D:\Anaconda\lib\site-packages\spyderlib\utils\iofuncs.py", line 26, in <module>
    import pandas as pd
  File "D:\Anaconda\lib\site-packages\pandas\__init__.py", line 7, in <module>
    from . import hashtable, tslib, lib
  File "pandas\tslib.pyx", line 46, in init pandas.tslib (pandas\tslib.c:79899)
  File "D:\Anaconda\lib\site-packages\pytz\__init__.py", line 29, in <module>
    from pkg_resources import resource_stream
  File "D:\Anaconda\lib\site-packages\setuptools-14.3-py2.7.egg\pkg_resources\__init__.py", line 1366, in <module>

  File "D:\Anaconda\lib\site-packages\setuptools-14.3-py2.7.egg\pkg_resources\__init__.py", line 1370, in MarkerEvaluation

AttributeError: 'module' object has no attribute 'python_version'
================================================================

----------
components: Windows
messages: 239935
nosy: a3arar, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: qtconsole and all windows based python have issues loading
type: behavior
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23854>
_______________________________________

From report at bugs.python.org  Fri Apr  3 00:11:43 2015
From: report at bugs.python.org (Bill Parker)
Date: Thu, 02 Apr 2015 22:11:43 +0000
Subject: [New-bugs-announce] [issue23855] Missing Sanity Check for malloc()
	in PC/_msi.c
Message-ID: <1428012703.71.0.811050257631.issue23855@psf.upfronthosting.co.za>


New submission from Bill Parker:

Hello All,

   In reviewing code in Python-3.4.3/PC/_msi.c, I found a call to malloc() at line 326 in function 'static PyObject* msierror(int status)' in which the call is made and assigned to variable 'res', but no check for NULL, indicating failure is made afterwards.  The patch below corrects this issue:

--- _msi.c.orig 2015-04-02 15:01:02.882326352 -0700
+++ _msi.c      2015-04-02 15:02:43.382099357 -0700
@@ -324,6 +324,10 @@
     code = MsiRecordGetInteger(err, 1); /* XXX code */
     if (MsiFormatRecord(0, err, res, &size) == ERROR_MORE_DATA) {
         res = malloc(size+1);
+       if (res == NULL) /* malloc() failed, out of memory... */
+           PyErr_SetString(MSIError, "out of memory");
+           return NULL;
+       }
         MsiFormatRecord(0, err, res, &size);
         res[size]='\0';
     }

----------
components: Windows
files: _msi.c.patch
keywords: patch
messages: 239948
nosy: dogbert2, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Missing Sanity Check for malloc() in PC/_msi.c
type: behavior
versions: Python 3.4
Added file: http://bugs.python.org/file38811/_msi.c.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23855>
_______________________________________

From report at bugs.python.org  Fri Apr  3 10:33:46 2015
From: report at bugs.python.org (Tim Golden)
Date: Fri, 03 Apr 2015 08:33:46 +0000
Subject: [New-bugs-announce] [issue23856] build.bat -e shouldn't also build
Message-ID: <1428050026.69.0.479392405161.issue23856@psf.upfronthosting.co.za>


New submission from Tim Golden:

PCBuild\build.bat takes an argument of -e to pull in external libraries. 

Either by accident or by design the main build will run in addition. However if you'd run pcbuild -e simply to pull in externals, you might not have specified extra build params, such as -d or -p x64.

The attached patch naively drops out after get_externals has been called; other possibilities might include carrying on if -e was not the only parameter.

----------
components: Windows
files: build-externals.patch
keywords: patch
messages: 239957
nosy: steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
stage: patch review
status: open
title: build.bat -e shouldn't also build
type: behavior
versions: Python 3.5
Added file: http://bugs.python.org/file38813/build-externals.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23856>
_______________________________________

From report at bugs.python.org  Fri Apr  3 12:38:06 2015
From: report at bugs.python.org (Robert Kuska)
Date: Fri, 03 Apr 2015 10:38:06 +0000
Subject: [New-bugs-announce] [issue23857] [RFE] Make certificate
	verification optionable
Message-ID: <1428057486.92.0.857087998573.issue23857@psf.upfronthosting.co.za>


New submission from Robert Kuska:

Proposed patch adds possibility to opt-out certificate verification. 
Disclaimer: it is just proof of concept as the config value is hard-coded.

How it works?
This patch depends on existence of config file which holds information about the protocol settings.

> $ cat cert-verification.conf
[https]  # each protocol affected by cert-verification got its own section
verify=platform_default

Possible values for verify are:
enable - to enable certificate verification
disable - to disable certificate verification
platform_default - to use default (platform-specific) settings

Why platform_default?
This choice is for users who don't care about the security settings so they put the decision into their platform (distro) from which they get python. In rpm we can set package to not replace user edited configs when rpm is updated, so if user change the default value of config the config will remain the same.

Python example:

 >>> import http.client
 >>> cn = http.client.HTTPSConnection('www.google.com')
 >>> cn._context.verify_mode
 0L  # CERT_NONE
 >>> # config changed to verify=enable, still same interpreter
 >>> cn2 = http.client.HTTPSConnection('www.google.com')
 >>> cn2._context.verify_mode
 2L  # CERT_REQUIRED

This is how currently works patch attached, but I guess it would make more sense make this behave consistent within the same interpreter even when config is changed and the change will be propagated in the next interpreter run/service restart.

Also the patch could be changed to instead of being protocol based to be module based, but this would need also patching the affected modules.

I open the RFE mainly to see if there is a will to implement optionable certificate verification in upstream as it is in downstream [citation needed].

I've added some people to nosy list based on https://docs.python.org/devguide/experts.html

----------
components: Library (Lib)
messages: 239965
nosy: alex, dstufft, haypo, janssen, ncoghlan, pitrou, rkuska
priority: normal
severity: normal
status: open
title: [RFE] Make certificate verification optionable
type: enhancement
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23857>
_______________________________________

From report at bugs.python.org  Fri Apr  3 16:36:42 2015
From: report at bugs.python.org (Simon Ye)
Date: Fri, 03 Apr 2015 14:36:42 +0000
Subject: [New-bugs-announce] [issue23858] Look for local sqlite3 by parsing
	-I/-L flags in linux	as well.
Message-ID: <1428071802.53.0.603746125685.issue23858@psf.upfronthosting.co.za>


New submission from Simon Ye:

Based off of https://github.com/Homebrew/linuxbrew/pull/330.

Currently building python on linux only looks for sqlite include paths in a hardcoded set of system paths, but if the user specifies -I/-L options during compilation, python setup.py looks there for sqlite3 as well, but only on OS X. This is problem for people who want to build python against a local version of sqlite3, and it is also inconsistent with the OS X behavior. Also the comments say that it should work on any unix-y OS!

----------
components: Build
files: sqlite_patch.diff
keywords: patch
messages: 239997
nosy: yesimon
priority: normal
severity: normal
status: open
title: Look for local sqlite3 by parsing -I/-L flags in linux as well.
versions: Python 3.6
Added file: http://bugs.python.org/file38819/sqlite_patch.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23858>
_______________________________________

From report at bugs.python.org  Fri Apr  3 17:13:17 2015
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 03 Apr 2015 15:13:17 +0000
Subject: [New-bugs-announce] [issue23859] asyncio: document behaviour of
	wait() cancellation
Message-ID: <1428073997.46.0.438809738394.issue23859@psf.upfronthosting.co.za>


New submission from STINNER Victor:

When wait(fs) is cancelled, futures of fs are not cancelled. This behaviour is not documented. It should be documented, and a warning should be added to the Cancellation section:
https://docs.python.org/dev/library/asyncio-dev.html#cancellation

See my message "Cancellation of asyncio.wait()":
https://mail.google.com/mail/u/0/#label/Tulip/14b34f0a5d31114f

----------
components: asyncio
messages: 240001
nosy: gvanrossum, haypo, yselivanov
priority: normal
severity: normal
status: open
title: asyncio: document behaviour of wait() cancellation
versions: Python 3.4, Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23859>
_______________________________________

From report at bugs.python.org  Fri Apr  3 19:44:00 2015
From: report at bugs.python.org (Bill Parker)
Date: Fri, 03 Apr 2015 17:44:00 +0000
Subject: [New-bugs-announce] [issue23860] Failure to check return value from
	lseek() in	Modules/mmapmodule.c
Message-ID: <1428083040.47.0.668729418368.issue23860@psf.upfronthosting.co.za>


New submission from Bill Parker:

Hello All,

   In reviewing code in directory Python-3.4.3/Modules, file 
'mmapmodule', I found a call to 'lseek()' without a check for
a return value of -1, indicating failure.  The patch file below
corrects this issue (diff -u format):

--- mmapmodule.c.orig	2015-04-02 19:05:30.380554538 -0700
+++ mmapmodule.c	2015-04-02 19:11:00.320488207 -0700
@@ -1335,7 +1335,11 @@
             return NULL;
         }
         /* Win9x appears to need us seeked to zero */
-        lseek(fileno, 0, SEEK_SET);
+	if (lseek(fileno, 0, SEEK_SET) == -1) { /* call to lseek() failed */
+	    PyErr_SetFromErrno(PyExc_OSError);
+	    return NULL;
+	}
+
     }
 
     m_obj = (mmap_object *)type->tp_alloc(type, 0);

I am attaching the patch file to this bug report...

----------
components: Interpreter Core
files: mmapmodule.c.patch
keywords: patch
messages: 240015
nosy: dogbert2
priority: normal
severity: normal
status: open
title: Failure to check return value from lseek() in Modules/mmapmodule.c
type: behavior
versions: Python 3.4
Added file: http://bugs.python.org/file38823/mmapmodule.c.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23860>
_______________________________________

From report at bugs.python.org  Fri Apr  3 20:55:50 2015
From: report at bugs.python.org (Steve Dower)
Date: Fri, 03 Apr 2015 18:55:50 +0000
Subject: [New-bugs-announce] [issue23861] Make stdprinter use
	DebugOutputString when no	stdout/stderr available
Message-ID: <1428087350.41.0.355922680582.issue23861@psf.upfronthosting.co.za>


New submission from Steve Dower:

Currently we initialize stdprinter while initializing and terminate if we can't access std streams.

We should always write to DebugOutputString on Windows (only does anything if a debugger is attached) and avoid terminating in this case.

----------
components: Windows
messages: 240025
nosy: steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Make stdprinter use DebugOutputString when no stdout/stderr available
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23861>
_______________________________________

From report at bugs.python.org  Fri Apr  3 21:04:53 2015
From: report at bugs.python.org (rengine)
Date: Fri, 03 Apr 2015 19:04:53 +0000
Subject: [New-bugs-announce] [issue23862] subprocess popen arguments using
	double quotes
Message-ID: <1428087893.92.0.437670101484.issue23862@psf.upfronthosting.co.za>


New submission from rengine:

Using Python 2.7.9

Noticed when I run a subprocess Popen with an argument containing double quotes, it will have a different affect depending on my operating system. 

In Linux, if I run ./runme.py, it will call runme.sh which will append someargs.txt correctly without escaped quotations

In Windows, if I run ./runme.py, it will call runme.bat which will append someargs.txt with escaped quotations

Also in Windows, if I run runme.bat with an argument containing quotations it will append someargs.txt correctly without escaped quotations so this problem seems to be stemming from Python.

===================================== runme.py

#!/usr/bin/python
import sys
import subprocess
import shlex

# works on Linux:
#command_line = "./runme.sh --include=\"check\""

# fails on Windows:
command_line = "runme.bat --include=\"check\""
#command_line = "runme.bat --include=\"check\""

arg = shlex.shlex(command_line)
arg.quotes = '"'
arg.whitespace_split = True
arg.commenters = ''
command_line_args = list(arg)
print command_line_args

command_line_process = subprocess.Popen(
    command_line_args,
    universal_newlines=True,
    stdout=subprocess.PIPE,
    stderr=subprocess.PIPE
)

line = ""
while True:
    line = command_line_process.stdout.readline()
    if line:
        print line
    else:
        break

===================================== runme.bat

echo %*
echo %* >> someargs.txt

===================================== runme.sh

#!/bin/bash
echo $@
echo $@ >> someargs.txt

----------
messages: 240026
nosy: rengine
priority: normal
severity: normal
status: open
title: subprocess popen arguments using double quotes
type: behavior
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23862>
_______________________________________

From report at bugs.python.org  Sat Apr  4 04:51:25 2015
From: report at bugs.python.org (Jeff McNeil)
Date: Sat, 04 Apr 2015 02:51:25 +0000
Subject: [New-bugs-announce] [issue23863] Fix EINTR Socket Module issues in
	2.7
Message-ID: <1428115885.34.0.897753768137.issue23863@psf.upfronthosting.co.za>


New submission from Jeff McNeil:

There are a collection of places in the socket module that do not correctly retry on EINTR.  Updated to wrap those calls in a retry loop.  However, when fixing connect calls, I noticed that when EINTR is retried on a socket with a timeout specified, the retry fails with EALREADY.. so I fixed that.

I was going to shy away from primitive calls on sockets as one expects these things when working at a lower level, however, due to the way socket timeouts were implemented, I handled it differently in internal_connect.  The create_connection calls probably ought to shield users from retry.

Python 2.7.6.

----------
files: socket_intr.py
messages: 240044
nosy: mcjeff
priority: normal
severity: normal
status: open
title: Fix EINTR Socket Module issues in 2.7
versions: Python 2.7
Added file: http://bugs.python.org/file38825/socket_intr.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23863>
_______________________________________

From report at bugs.python.org  Sat Apr  4 12:57:47 2015
From: report at bugs.python.org (Martijn Pieters)
Date: Sat, 04 Apr 2015 10:57:47 +0000
Subject: [New-bugs-announce] [issue23864] issubclass without registration
	only works for	"one-trick pony" collections ABCs.
Message-ID: <1428145067.47.0.0726533343331.issue23864@psf.upfronthosting.co.za>


New submission from Martijn Pieters:

The collections.abc documentation implies that *any* of the container ABCs can be used in an issubclass test against a class that implements all abstract methods:

> These ABCs allow us to ask classes or instances if they provide particular functionality [...]

In reality this only applies to the "One Trick Ponies" (term from PEP 3119, things like Container and Iterable, those classes with one or two methods). It fails for the compound container ABCs:

>>> from collections.abc import Sequence, Container, Sized
>>> class MySequence(object):
...     def __contains__(self, item): pass
...     def __len__(self): pass
...     def __iter__(self): pass
...     def __getitem__(self, index): pass
...     def __len__(self): pass
... 
>>> issubclass(MySequence, Container)
True
>>> issubclass(MySequence, Sized)
True
>>> issubclass(MySequence, Sequence)
False

That's because the One Trick Ponies implement a __subclasshook__ method that is locked to the specific class and returns NotImplemented for subclasses; for instance, the Iterable.__subclasshook__ implementation is:

    @classmethod
    def __subclasshook__(cls, C):
        if cls is Iterable:
            if any("__iter__" in B.__dict__ for B in C.__mro__):
                return True
        return NotImplemented

The compound container classes build on top of the One Trick Ponies, so the class test will fail, NotImplemented is returned and the normal ABC tests for base classes that have been explicitly registered continues, but this won't include unregistered complete implementations.

Either the compound classes need their own __subclasshook__ implementations, or the documentation needs to be updated to make it clear that without explicit registrations the issubclass() (and isinstance()) tests only apply to the One Trick Ponies.

----------
assignee: docs at python
components: Documentation, Library (Lib)
messages: 240060
nosy: docs at python, mjpieters
priority: normal
severity: normal
status: open
title: issubclass without registration only works for "one-trick pony" collections ABCs.

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23864>
_______________________________________

From report at bugs.python.org  Sat Apr  4 13:13:00 2015
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sat, 04 Apr 2015 11:13:00 +0000
Subject: [New-bugs-announce] [issue23865] Fix possible leaks in close methods
Message-ID: <1428145980.2.0.463866688715.issue23865@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka:

Proposed patch fixes two related issues in a number of modules.

1. close() methods sometimes release multiple resources. Every closing operation can fail, but it shouldn't prevent releasing other resources. See for example issue21802.

2. close() should be idempotent. I.e. calling close() second times shouldn't have any effect. Even if close() failed, repeated call of close() (usually in __exit__(), in __del__(), or in finally block) shouldn't raise an exception.

Many close() methods already satisfy these conditions, but not all.

----------
components: Library (Lib)
files: close.patch
keywords: patch
messages: 240063
nosy: serhiy.storchaka
priority: normal
severity: normal
stage: patch review
status: open
title: Fix possible leaks in close methods
type: resource usage
versions: Python 2.7, Python 3.4, Python 3.5
Added file: http://bugs.python.org/file38829/close.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23865>
_______________________________________

From report at bugs.python.org  Sat Apr  4 13:41:09 2015
From: report at bugs.python.org (Antoine Pitrou)
Date: Sat, 04 Apr 2015 11:41:09 +0000
Subject: [New-bugs-announce] [issue23866] array module broken
Message-ID: <1428147669.14.0.904457263314.issue23866@psf.upfronthosting.co.za>


New submission from Antoine Pitrou:

The array.array constructor has stopped working. I feel like this may be due to the latest Argument Clinic changes.

======================================================================
ERROR: test_create_from_bytes (test.test_array.ByteTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/antoine/cpython/default/Lib/test/test_array.py", line 1016, in test_create_from_bytes
    a = array.array('H', b"1234")
TypeError: a bytes-like object is required, not 'tuple'

----------
components: Library (Lib)
messages: 240067
nosy: pitrou, serhiy.storchaka
priority: release blocker
severity: normal
status: open
title: array module broken
type: behavior
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23866>
_______________________________________

From report at bugs.python.org  Sat Apr  4 16:24:36 2015
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sat, 04 Apr 2015 14:24:36 +0000
Subject: [New-bugs-announce] [issue23867] Argument Clinic: inline parsing
	code for 1-argument	functions
Message-ID: <1428157476.28.0.0922120616231.issue23867@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka:

Proposed patch makes Argument Clinic to inline parsing code for most popular formats in functions with single positional argument. This makes parsing faster.

----------
components: Argument Clinic
files: clinic_meth_o_inline.patch
keywords: patch
messages: 240074
nosy: larry, serhiy.storchaka
priority: normal
severity: normal
stage: patch review
status: open
title: Argument Clinic: inline parsing code for 1-argument functions
type: enhancement
versions: Python 3.5
Added file: http://bugs.python.org/file38830/clinic_meth_o_inline.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23867>
_______________________________________

From report at bugs.python.org  Sat Apr  4 17:00:34 2015
From: report at bugs.python.org (Hristo Venev)
Date: Sat, 04 Apr 2015 15:00:34 +0000
Subject: [New-bugs-announce] [issue23868] Uninitialized objects are tracked
	by the garbage	collector
Message-ID: <1428159634.21.0.640344593113.issue23868@psf.upfronthosting.co.za>


New submission from Hristo Venev:

An object starts being tracked by the GC after being allocated, but before being initialized. If during initialization the GC runs, this may lead to tp_traverse being called on an uninitialized object.

----------
components: Interpreter Core
messages: 240079
nosy: h.venev
priority: normal
severity: normal
status: open
title: Uninitialized objects are tracked by the garbage collector
versions: Python 3.2, Python 3.3, Python 3.4, Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23868>
_______________________________________

From report at bugs.python.org  Sat Apr  4 17:03:22 2015
From: report at bugs.python.org (Hristo Venev)
Date: Sat, 04 Apr 2015 15:03:22 +0000
Subject: [New-bugs-announce] [issue23869] Initialization is being done in
	PyType_GenericAlloc
Message-ID: <1428159802.19.0.247308187517.issue23869@psf.upfronthosting.co.za>


New submission from Hristo Venev:

In PyType_GenericAlloc, the initialization is being done. Namely, the PyObject part of the object is initialized and it is tracked by the garbage collector.

In the documentation it is stated that tp_alloc should do no initialization.

----------
messages: 240080
nosy: h.venev
priority: normal
severity: normal
status: open
title: Initialization is being done in PyType_GenericAlloc

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23869>
_______________________________________

From report at bugs.python.org  Sat Apr  4 19:19:36 2015
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sat, 04 Apr 2015 17:19:36 +0000
Subject: [New-bugs-announce] [issue23870] pprint collections classes
Message-ID: <1428167976.78.0.301380656754.issue23870@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka:

Proposed patch adds support of all collections classes (except namedtuple) in pprint. It uses undocumented unstable private API. After implementing issue7434 the code could be rewritten with using public stable well-designed API. One day in the womb of time.

But for now this patch allows to grope requirements to future API.

----------
components: Library (Lib)
files: pprint_collections.patch
keywords: patch
messages: 240084
nosy: rhettinger, serhiy.storchaka
priority: normal
severity: normal
stage: patch review
status: open
title: pprint collections classes
type: enhancement
versions: Python 3.5
Added file: http://bugs.python.org/file38833/pprint_collections.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23870>
_______________________________________

From report at bugs.python.org  Sun Apr  5 00:59:03 2015
From: report at bugs.python.org (Antony Lee)
Date: Sat, 04 Apr 2015 22:59:03 +0000
Subject: [New-bugs-announce] [issue23871] turning itertools.{repeat,
	count} into indexable iterables
Message-ID: <1428188343.49.1.91705834608e-05.issue23871@psf.upfronthosting.co.za>


New submission from Antony Lee:

itertools.repeat and itertools.count could be made into indexable iterables (rather than iterators), rather than iterators, like range is right now.

----------
components: Library (Lib)
messages: 240096
nosy: Antony.Lee
priority: normal
severity: normal
status: open
title: turning itertools.{repeat,count} into indexable iterables
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23871>
_______________________________________

From report at bugs.python.org  Sun Apr  5 01:47:05 2015
From: report at bugs.python.org (Hoolean)
Date: Sat, 04 Apr 2015 23:47:05 +0000
Subject: [New-bugs-announce] [issue23872] Typo in response in smtpd
Message-ID: <1428191225.22.0.658176098333.issue23872@psf.upfronthosting.co.za>


New submission from Hoolean:

A spelling mistake is present: in all other instances in the class, the string is ' [SP <mail-parameters>]', yet at this line it is [incorrectly] ' [SP <mail parameters]', missing a '-' and a '>' character.

I have attached a patch correcting this.

----------
components: Library (Lib)
files: correction.patch
keywords: patch
messages: 240097
nosy: Hoolean
priority: normal
severity: normal
status: open
title: Typo in response in smtpd
type: behavior
Added file: http://bugs.python.org/file38835/correction.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23872>
_______________________________________

From report at bugs.python.org  Sun Apr  5 02:57:43 2015
From: report at bugs.python.org (Hoolean)
Date: Sun, 05 Apr 2015 00:57:43 +0000
Subject: [New-bugs-announce] [issue23873] Removal of dead code in smtpd
Message-ID: <1428195463.82.0.559504880721.issue23873@psf.upfronthosting.co.za>


New submission from Hoolean:

Code was present that checked conditions that had previous been checked and had returned the function prematurely if the condition was true. As the condition has not changed before the check is made again, the condition will always be false and the code inside the if-statement will never be evaluated.

The attached patch removes unnecessary code.

----------
components: Library (Lib)
files: correction.patch
keywords: patch
messages: 240098
nosy: Hoolean
priority: normal
severity: normal
status: open
title: Removal of dead code in smtpd
type: performance
Added file: http://bugs.python.org/file38836/correction.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23873>
_______________________________________

From report at bugs.python.org  Sun Apr  5 04:08:27 2015
From: report at bugs.python.org (Jason R. Coombs)
Date: Sun, 05 Apr 2015 02:08:27 +0000
Subject: [New-bugs-announce] [issue23874] Encrypted MSI fails to install
	with code 2755
Message-ID: <1428199707.15.0.13472678589.issue23874@psf.upfronthosting.co.za>


New submission from Jason R. Coombs:

When installing Python (3.4.3 or 2.7.9) from an EFS encrypted installer file, the installation proceeds normally through the target selection and feature selection, but then immediately reports "/!\ The system cannot open the device or file specified," and aborts the installation with "The installer has encountered an unexpected error installing this package. This may indicate a problem with the package. The error code is 2755."

Other MSI products install fine when the MSI is encrypted (pandoc, MongoDB), so there appears to be something unique about the Python installer that's failing when the Trusted Installer cannot read the MSI itself. Decrypting the MSI before installing works around the issue.

Environment: Windows 8.1 64-bit with Python 64-bit

----------
components: Installation, Windows
messages: 240099
nosy: jason.coombs, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Encrypted MSI fails to install with code 2755
versions: Python 2.7, Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23874>
_______________________________________

From report at bugs.python.org  Mon Apr  6 14:01:52 2015
From: report at bugs.python.org (Serhiy Storchaka)
Date: Mon, 06 Apr 2015 12:01:52 +0000
Subject: [New-bugs-announce] [issue23875] Incorrect argument parsing in _ssl
Message-ID: <1428321712.11.0.166628574781.issue23875@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka:

Argument parsing code for functions _ssl.enum_certificates() and _ssl.enum_crls() look not correct. 

    if (!PyArg_ParseTupleAndKeywords(args, kwds, "s|s:enum_certificates",
                                     kwlist, &store_name)) {
        return NULL;
    }

The format contains codes for two string arguments. But only one address (&store_name) is passed. And kwlist contains only one member, "store_name".

These functions are provided only on Windows, so I can't check what happen if call them with two string arguments. May be crash or memory corruption.

----------
components: Extension Modules
messages: 240151
nosy: alex, christian.heimes, dstufft, giampaolo.rodola, janssen, pitrou, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Incorrect argument parsing in _ssl
type: behavior
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23875>
_______________________________________

From report at bugs.python.org  Mon Apr  6 14:41:15 2015
From: report at bugs.python.org (Jeffrey Armstrong)
Date: Mon, 06 Apr 2015 12:41:15 +0000
Subject: [New-bugs-announce] [issue23876] Fix mkdir() call for Watcom
	compilers on UNIX-like	platforms
Message-ID: <1428324075.5.0.672514551353.issue23876@psf.upfronthosting.co.za>


New submission from Jeffrey Armstrong:

Within Modules/posixmodule.c:4914 (in 3.5.0a3), the preprocessor checks for compiler macros as such:

#if ( defined(__WATCOMC__) || defined(PYCC_VACPP) ) && !defined(__QNX__)
        result = mkdir(path->narrow);
#else
        result = mkdir(path->narrow, mode);
#endif

The purpose of the code was to detect if we're compiling using Watcom, but not on QNX, or VisualAge as our compiler, where mkdir() wouldn't accept a mode.  However, Watcom supports Linux as well and properly implements the mode argument, causing the compilation to fail.

The proper check, rather than looking for "!defined(__QNX__)" would be "!defined(__UNIX__)," which would allow the code to properly compile using Watcom on either Linux or QNX:

#if ( defined(__WATCOMC__) || defined(PYCC_VACPP) ) && !defined(__UNIX__)
        result = mkdir(path->narrow);
#else
        result = mkdir(path->narrow, mode);
#endif


FYI, in Watcom, the __UNIX__ macro is defined on both Linux and QNX, so this change will not break code for people who are still using Watcom to build Python for QNX (which is probably nobody at all).

There are two other places where the __QNX__ macro should be replaced in Modules/posixmodule.c, and the attached patch fixes both.

----------
components: Interpreter Core
files: watcom_qnx_to_unix-3.5.0a3.patch
keywords: patch
messages: 240153
nosy: Jeffrey.Armstrong
priority: normal
severity: normal
status: open
title: Fix mkdir() call for Watcom compilers on UNIX-like platforms
type: compile error
versions: Python 3.5
Added file: http://bugs.python.org/file38843/watcom_qnx_to_unix-3.5.0a3.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23876>
_______________________________________

From report at bugs.python.org  Mon Apr  6 14:48:51 2015
From: report at bugs.python.org (Jeffrey Armstrong)
Date: Mon, 06 Apr 2015 12:48:51 +0000
Subject: [New-bugs-announce] [issue23877] Build fails when threads are
	disabled during configure	step
Message-ID: <1428324531.94.0.57917377486.issue23877@psf.upfronthosting.co.za>


New submission from Jeffrey Armstrong:

If threads are disabled, either via --disable-threads or using a compiler/standard library that doesn't support threads, the build will fail when linking the Python interpreter because the following is undefined:

PyGILState_GetThisThreadState

The error is caused by a change since 3.5.0a1 that uses a call to this function in Python/pylifecycle.c:1303 without first checking if the WITH_THREADS macro is defined.  If WITH_THREADS is undefined, the function PyGILState_GetThisThreadState is not built.

I've attached a simple patch correcting the issue, but it should be trivial to add a simple macro check around the call.

----------
components: Interpreter Core
files: pylifecycle-threads-3.5.0a3.patch
keywords: patch
messages: 240154
nosy: Jeffrey.Armstrong
priority: normal
severity: normal
status: open
title: Build fails when threads are disabled during configure step
type: compile error
versions: Python 3.5
Added file: http://bugs.python.org/file38844/pylifecycle-threads-3.5.0a3.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23877>
_______________________________________

From report at bugs.python.org  Mon Apr  6 19:00:27 2015
From: report at bugs.python.org (Bill Parker)
Date: Mon, 06 Apr 2015 17:00:27 +0000
Subject: [New-bugs-announce] [issue23878] Missing sanity checks for various
	C library function	calls...
Message-ID: <1428339627.31.0.628073381935.issue23878@psf.upfronthosting.co.za>


New submission from Bill Parker:

Hello All,

   In reviewing code for Python-3.4.3 in directory
'Modules/_ctypes/libffi/src/arm', file 'ffi.c', I found a pair
of calls to calloc() which do not test for a return value
of NULL, indicating failure.  The patch file below corrects
this issue:

--- ffi.c.orig  2015-04-04 15:43:19.662709073 -0700
+++ ffi.c       2015-04-04 15:51:27.142665269 -0700
@@ -629,12 +629,21 @@
 
     /* We have valid trampoline and config pages */
     table = calloc (1, sizeof(ffi_trampoline_table));
+    if (table == NULL) { /* oops, calloc() failed, now what??? */
+      fprintf(stderr, "vm calloc() failure: %d at %s:%d\n", kt, __FILE__, __LINE__);
+      return NULL; /* go home??? */
+    }
     table->free_count = FFI_TRAMPOLINE_COUNT;
     table->config_page = config_page;
     table->trampoline_page = trampoline_page;
 
     /* Create and initialize the free list */
     table->free_list_pool = calloc(FFI_TRAMPOLINE_COUNT, sizeof(ffi_trampoline_table_entry));
+    if (table->free_list_pool == NULL) { /* oops, calloc() failed, now what */
+      fprintf(stderr, "vm calloc() failure: %d at %s:%d\n", kt, __FILE__, __LINE__);
+      free(table);  /* free table (from previos calloc() call) */
+      return NULL;  /* go home??? *
+    }
 
     uint16_t i;
     for (i = 0; i < table->free_count; i++) {

In directory 'Modules', file 'getpath.c', I found a call to fseek()
which is not checked for a return value < 0, indicating failure.  The
patch file below corrects this issue:

--- getpath.c.orig      2015-04-04 16:07:25.540472702 -0700
+++ getpath.c   2015-04-04 16:09:30.988416490 -0700
@@ -265,7 +265,9 @@
     int result = 0; /* meaning not found */
     char buffer[MAXPATHLEN*2+1];  /* allow extra for key, '=', etc. */
 
-    fseek(env_file, 0, SEEK_SET);
+    if (fseek(env_file, 0, SEEK_SET) < 0)
+        return result;
+   
     while (!feof(env_file)) {
         char * p = fgets(buffer, MAXPATHLEN*2, env_file);
         wchar_t tmpbuffer[MAXPATHLEN*2+1];
		 		
I am attaching the patch file(s) to this bug report...

Bill Parker (wp02855 at gmail dot com)

----------
components: Interpreter Core
files: getpath.c.patch
keywords: patch
messages: 240160
nosy: dogbert2
priority: normal
severity: normal
status: open
title: Missing sanity checks for various C library function calls...
type: behavior
versions: Python 3.4
Added file: http://bugs.python.org/file38848/getpath.c.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23878>
_______________________________________

From report at bugs.python.org  Tue Apr  7 10:23:09 2015
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 07 Apr 2015 08:23:09 +0000
Subject: [New-bugs-announce] [issue23879] asyncio:
	SelectorEventLoop.sock_connect() must not retry	connect() on
	InterruptedError
Message-ID: <1428394989.42.0.151918238179.issue23879@psf.upfronthosting.co.za>


New submission from STINNER Victor:

According to the issue #23618, when connect() fails with EINTR, retrying connect() may only work on some platforms. To have a reliable behaviour on all platforms, we should wait until the socket becomes writable because the connection runs asynchronously in background. When the socket becomes writable, we have to gets its error code (getsockopt(SO_ERROR)) to check if the connection succeeded or failed.

Attached patch fixes SelectorEventLoop.sock_connect().

----------
components: asyncio
files: connect_eintr.patch
keywords: patch
messages: 240195
nosy: gvanrossum, haypo, yselivanov
priority: normal
severity: normal
status: open
title: asyncio: SelectorEventLoop.sock_connect() must not retry connect() on InterruptedError
versions: Python 3.4, Python 3.5
Added file: http://bugs.python.org/file38854/connect_eintr.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23879>
_______________________________________

From report at bugs.python.org  Tue Apr  7 10:41:04 2015
From: report at bugs.python.org (Serhiy Storchaka)
Date: Tue, 07 Apr 2015 08:41:04 +0000
Subject: [New-bugs-announce] [issue23880] Tkinter: getint and getdouble
	should support Tcl_Obj
Message-ID: <1428396064.01.0.00215215635093.issue23880@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka:

getint, getdouble and getboolean were thin wrappers around Tcl functions that converted string result of Tcl call to specified Python type. Since 2.3 _tkinter can return not only string, but int, float, etc and Tcl_Obj (if wantobject is True). getXXX methods was updated to work with respective automatically converted types (getint with int, etc), but they don't work with general Tcl_Obj, so can't be applied to the result of _tkinter call in general case. As a workaround you should use int(str(value)) or like.

Support of Tcl_Obj in getbool was added in issue15133. Proposed patch adds support of Tcl_Obj in getint and getdouble and int in getdouble. It also restores the use of getint and getdouble in Tkinter.

----------
components: Extension Modules, Tkinter
files: tkinter_getxxx_tclobj.patch
keywords: patch
messages: 240196
nosy: serhiy.storchaka
priority: normal
severity: normal
stage: patch review
status: open
title: Tkinter: getint and getdouble should support Tcl_Obj
type: enhancement
versions: Python 3.5
Added file: http://bugs.python.org/file38855/tkinter_getxxx_tclobj.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23880>
_______________________________________

From report at bugs.python.org  Tue Apr  7 12:43:02 2015
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 07 Apr 2015 10:43:02 +0000
Subject: [New-bugs-announce] [issue23881] test_urllib2net: issues
	with	ftp://gatekeeper.research.compaq.com/ and ftp://ftp.mirror.nl/
Message-ID: <1428403382.55.0.763005294099.issue23881@psf.upfronthosting.co.za>


New submission from STINNER Victor:

test_urllib2net uses multiple public FTP servers. Since yesterday, there are issues with 2 FTP servers:

(1) ftp://gatekeeper.research.compaq.com/pub/DEC/SRC/research-reports/00README-Legal-Rules-Regs
=> DNS server error

$ host gatekeeper.research.compaq.com 8.8.8.8
Host gatekeeper.research.compaq.com not found: 2(SERVFAIL)
$ host gatekeeper.research.compaq.com 212.27.40.240
Host gatekeeper.research.compaq.com not found: 2(SERVFAIL)

(2) ftp://ftp.mirror.nl/pub/gnu/

* The FTP connection succeeded
* Changing the directory to /pub/gnu succeeded
* but listing the content of /pub/gnu/ takes forever...

In Firefox, I get the error "425 Failed to establish connection".

----------
messages: 240198
nosy: haypo
priority: normal
severity: normal
status: open
title: test_urllib2net: issues with ftp://gatekeeper.research.compaq.com/ and ftp://ftp.mirror.nl/

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23881>
_______________________________________

From report at bugs.python.org  Tue Apr  7 13:36:36 2015
From: report at bugs.python.org (Florian Apolloner)
Date: Tue, 07 Apr 2015 11:36:36 +0000
Subject: [New-bugs-announce] [issue23882] unittest discovery and namespaced
	packages
Message-ID: <1428406596.02.0.0629964312505.issue23882@psf.upfronthosting.co.za>


New submission from Florian Apolloner:

Unittest discovery does not seem to work if the tests package is  a namespace package. The attached file should have all details to reproduce, as soon as I readd an __init__.py everything works fine. 

Test was done using python3.4.2 and 3.4.3

----------
files: console_log.txt
messages: 240205
nosy: Florian.Apolloner
priority: normal
severity: normal
status: open
title: unittest discovery and namespaced packages
versions: Python 3.4
Added file: http://bugs.python.org/file38857/console_log.txt

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23882>
_______________________________________

From report at bugs.python.org  Tue Apr  7 18:23:10 2015
From: report at bugs.python.org (Serhiy Storchaka)
Date: Tue, 07 Apr 2015 16:23:10 +0000
Subject: [New-bugs-announce] [issue23883] __all__ lists are incomplete
Message-ID: <1428423790.05.0.732881467443.issue23883@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka:

Here is a (perhaps incomplete) list of documented names absent in __all__ lists of respective modules. Perhaps some of them (but not all) are worth to be added to __all__ lists.

calendar.Calendar
calendar.HTMLCalendar
calendar.TextCalendar
cgi.test
configparser.Error
csv.unix_dialect
doctest.DocFileCase
doctest.DocTestCase
enum.EnumMeta
fileinput.fileno
ftplib.Error
ftplib.error_perm
ftplib.error_reply
gettext.bind_textdomain_codeset
gettext.lgettext
gettext.lngettext
http.client.HTTPMessage
http.cookies.Morsel
http.server.test
logging.shutdown
mailbox.Error
mailbox.ExternalClashError
mailbox.NoSuchMailboxError
mailbox.NotEmptyError
mimetypes.MimeTypes
optparse.check_choice
pickletools.OpcodeInfo
plistlib.InvalidFileException
pydoc.doc
smtpd.SMTPChannel
subprocess.SubprocessError
subprocess.TimeoutExpired
tarfile.CompressionError
tarfile.HeaderError
tarfile.ReadError
tarfile.open
threading.BrokenBarrierError
tkinter.ttk.Widget
tokenize.open
traceback.FrameSummary
traceback.StackSummary
traceback.TracebackException
traceback.walk_stack
traceback.walk_tb
wave.Wave_read
wave.Wave_write
xml.etree.ElementTree.XMLPullParser

----------
components: Library (Lib)
messages: 240217
nosy: r.david.murray, serhiy.storchaka, vadmium
priority: normal
severity: normal
status: open
title: __all__ lists are incomplete
type: enhancement
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23883>
_______________________________________

From report at bugs.python.org  Tue Apr  7 19:29:13 2015
From: report at bugs.python.org (Peter Marsh)
Date: Tue, 07 Apr 2015 17:29:13 +0000
Subject: [New-bugs-announce] [issue23884] New DateType for argparse (like
	FileType but for dates)
Message-ID: <1428427753.85.0.794213637884.issue23884@psf.upfronthosting.co.za>


New submission from Peter Marsh:

Hello,

Reasonably frequently I find myself needing to pass a date as a command line argument to a Python script I've written. Currently, argparse does not have a built support for dates - this adds a new class to argparse (much like the existing FileType) that parses arguments to datetime.date instances.

Example:

      >>> parser = argparse.ArgumentParser()
      >>> parser.add_argument('--start', type=argparse.DateType('%d/%m/%Y))
      >>> parser.add_argument('end', type=argparse.DateType())
      >>> parser.parse_args(['--start', '01/02/2015', '2015-01-03'])
      Namespace(end=datetime.date(2015, 1, 3), start=datetime.date(2015, 1, 2))


I think this would be a useful addition to the standard library, a quick Google shows that many people roll their own version of this anyway.

Support for datetime.datetime and perhaps even datetime.timedeltas might be good too, but date/times get a bit more complicated (timezones in general and varying support for the '%z' format string which is required to default to an ISO8601 date time).

Cheers,

Pete

----------
components: Library (Lib)
files: argparse_datetype.patch
keywords: patch
messages: 240220
nosy: petedmarsh
priority: normal
severity: normal
status: open
title: New DateType for argparse (like FileType but for dates)
type: enhancement
versions: Python 3.5
Added file: http://bugs.python.org/file38859/argparse_datetype.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23884>
_______________________________________

From report at bugs.python.org  Tue Apr  7 23:10:15 2015
From: report at bugs.python.org (=?utf-8?q?Marcin_Ko=C5=9Bcielnicki?=)
Date: Tue, 07 Apr 2015 21:10:15 +0000
Subject: [New-bugs-announce] [issue23885] urllib.quote horribly mishandles
	unicode as second	parameter
Message-ID: <1428441015.17.0.142718858432.issue23885@psf.upfronthosting.co.za>


New submission from Marcin Ko?cielnicki:

All hell breaks loose when unicode is passed as the second argument to urllib.quote in Python 2:

>>> import urllib
>>> urllib.quote('\xce\x91', u'')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/urllib.py", line 1292, in quote
    if not s.rstrip(safe):
UnicodeDecodeError: 'ascii' codec can't decode byte 0xce in position 0: ordinal not in range(128)

This on its own wouldn't be that bad - just another Python 2 unicode wonkiness.  However, coupled with caching done by the quote function (quoters are cached based on the second parameter, and u'' == ''), it means that a random preceding call to quote from an entirely different place in the application can break your code:

$ python2
Python 2.7.9 (default, Dec 11 2014, 04:42:00)
[GCC 4.9.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import urllib
>>> urllib.quote('\xce\x91', '')
'%CE%91'
>>>


$ python2
Python 2.7.9 (default, Dec 11 2014, 04:42:00)
[GCC 4.9.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import urllib
>>> urllib.quote('a', u'')
'a'
>>> urllib.quote('\xce\x91', '')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/urllib.py", line 1292, in quote
    if not s.rstrip(safe):
UnicodeDecodeError: 'ascii' codec can't decode byte 0xce in position 0: ordinal not in range(128)

Good luck debugging that.

So, one of two things needs to happen:

- a TypeError when unicode is passed as the second parameter, or
- a cast of the second parameter to str

----------
components: Library (Lib)
messages: 240230
nosy: koriakin
priority: normal
severity: normal
status: open
title: urllib.quote horribly mishandles unicode as second parameter
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23885>
_______________________________________

From report at bugs.python.org  Wed Apr  8 10:03:04 2015
From: report at bugs.python.org (Albert Zeyer)
Date: Wed, 08 Apr 2015 08:03:04 +0000
Subject: [New-bugs-announce] [issue23886] faulthandler_user should use
	_PyThreadState_Current
Message-ID: <1428480184.73.0.0889790949933.issue23886@psf.upfronthosting.co.za>


New submission from Albert Zeyer:

SIGUSR1/2 will get delivered to any running thread. The current thread of the signal doesn't give any useful information. Try to get the current Python thread which holds the GIL instead, or use NULL.

I have patched this for the external faulthandler module here:
https://github.com/haypo/faulthandler/pull/12
https://github.com/albertz/faulthandler/commit/dc92265

----------
components: Library (Lib)
messages: 240252
nosy: Albert.Zeyer
priority: normal
severity: normal
status: open
title: faulthandler_user should use _PyThreadState_Current
type: behavior
versions: Python 3.4, Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23886>
_______________________________________

From report at bugs.python.org  Wed Apr  8 13:44:13 2015
From: report at bugs.python.org (Facundo Batista)
Date: Wed, 08 Apr 2015 11:44:13 +0000
Subject: [New-bugs-announce] [issue23887] HTTPError doesn't have a good
	"repr" representation
Message-ID: <1428493453.93.0.931112960505.issue23887@psf.upfronthosting.co.za>


New submission from Facundo Batista:

I normally print(repr()) the exception I got, for debugging purposes. I use repr() because for builtin exceptions, str() will print only the message, and not the exception type.

But for HTTPError, the repr() of it is "HTTPError()", without further explanation...

----------
messages: 240262
nosy: facundobatista
priority: normal
severity: normal
status: open
title: HTTPError doesn't have a good "repr" representation
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23887>
_______________________________________

From report at bugs.python.org  Wed Apr  8 16:17:39 2015
From: report at bugs.python.org (ssh)
Date: Wed, 08 Apr 2015 14:17:39 +0000
Subject: [New-bugs-announce] [issue23888] Fixing fractional expiry time bug
	in cookiejar
Message-ID: <1428502659.33.0.324281998086.issue23888@psf.upfronthosting.co.za>


New submission from ssh:

If the FileCookieJar reads a cookie whose expiry time is a decimal fraction, it crashes.

Chrome extensions "cookies.txt" and "EdiThisCookie" export the expiry time as decimal fractions. This is accepted by wget and curl, but not by the FileCookieJar which ends up crashing.

I made a StackOverflow question checking if fractional decimal expiry times were even allowed (if it was a bug in the extensions), but didn't get a response: https://stackoverflow.com/questions/29502672/can-the-cookie-expires-field-be-a-decimal-value

At any rate, this patch should make the library more robust.

----------
components: Library (Lib)
files: mywork.patch
keywords: patch
messages: 240265
nosy: serhiy.storchaka, ssh
priority: normal
severity: normal
status: open
title: Fixing fractional expiry time bug in cookiejar
type: crash
versions: Python 3.5
Added file: http://bugs.python.org/file38864/mywork.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23888>
_______________________________________

From report at bugs.python.org  Wed Apr  8 19:37:50 2015
From: report at bugs.python.org (Yury Selivanov)
Date: Wed, 08 Apr 2015 17:37:50 +0000
Subject: [New-bugs-announce] [issue23889] Speedup inspect.Signature.bind
Message-ID: <1428514670.65.0.478019521392.issue23889@psf.upfronthosting.co.za>


New submission from Yury Selivanov:

Right now the implementation of Signature.bind is very complex, which leads to a subpar performance. The only way to significantly speed it up is to employ code generation and cache (the other way it to rewrite it in C, but that's something I'd like to avoid). I'll upload an initial implementation soon.

----------
assignee: yselivanov
components: Library (Lib)
messages: 240279
nosy: asvetlov, brett.cannon, larry, ncoghlan, yselivanov
priority: normal
severity: normal
status: open
title: Speedup inspect.Signature.bind
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23889>
_______________________________________

From report at bugs.python.org  Wed Apr  8 20:37:05 2015
From: report at bugs.python.org (Vjacheslav Fyodorov)
Date: Wed, 08 Apr 2015 18:37:05 +0000
Subject: [New-bugs-announce] [issue23890] assertRaises increases reference
	counter
Message-ID: <1428518225.53.0.665360255509.issue23890@psf.upfronthosting.co.za>


New submission from Vjacheslav Fyodorov:

Sometimes unittest's assertRaises increases reference counter of callable. This can break tests in tricky cases. Not seen in 2.X version. Demo file attached.

----------
components: Library (Lib)
files: test_assertRaises.py
messages: 240280
nosy: Vjacheslav.Fyodorov
priority: normal
severity: normal
status: open
title: assertRaises increases reference counter
type: behavior
versions: Python 3.4
Added file: http://bugs.python.org/file38866/test_assertRaises.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23890>
_______________________________________

From report at bugs.python.org  Wed Apr  8 21:30:10 2015
From: report at bugs.python.org (A.M. Kuchling)
Date: Wed, 08 Apr 2015 19:30:10 +0000
Subject: [New-bugs-announce] [issue23891] Tutorial doesn't mention either
	pip or virtualenv
Message-ID: <1428521410.44.0.984619918633.issue23891@psf.upfronthosting.co.za>


New submission from A.M. Kuchling:

(from discussion at the 2015 Python Language Summit)

Current versions of Python make it relatively easy to install third-party packages such as requests.  New users may not realize this, though, and continue using libraries such as urllib/urllib2 because they're in the stdlib.

The Python tutorial doesn't seem to mention either virtualenv or pip. It should, describing how to create a virtualenv, install packages, and manage them in basic ways (removing, 'pip freeze', requirements.txt.)

----------
assignee: akuchling
components: Documentation
messages: 240281
nosy: akuchling
priority: normal
severity: normal
status: open
title: Tutorial doesn't mention either pip or virtualenv
type: enhancement
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23891>
_______________________________________

From report at bugs.python.org  Wed Apr  8 23:41:44 2015
From: report at bugs.python.org (Brett Cannon)
Date: Wed, 08 Apr 2015 21:41:44 +0000
Subject: [New-bugs-announce] [issue23892] Introduce
	sys.implementation.opt_levels ?
Message-ID: <1428529304.36.0.272921433795.issue23892@psf.upfronthosting.co.za>


New submission from Brett Cannon:

Eric suggested in a code review for issue #23731 that maybe we should have the possible optimization levels listed somewhere.

----------
components: Library (Lib)
messages: 240286
nosy: brett.cannon, eric.snow
priority: low
severity: normal
stage: test needed
status: open
title: Introduce sys.implementation.opt_levels ?
type: enhancement
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23892>
_______________________________________

From report at bugs.python.org  Wed Apr  8 23:53:40 2015
From: report at bugs.python.org (Brett Cannon)
Date: Wed, 08 Apr 2015 21:53:40 +0000
Subject: [New-bugs-announce] [issue23893] Forward-port future_builtins
Message-ID: <1428530020.25.0.529453311381.issue23893@psf.upfronthosting.co.za>


New submission from Brett Cannon:

future_builtins exists in Python 2.7, but not Python 3 which is annoying for code that wants to rely on it in Python 2.7 but not 2to3 (who is the primary user of the module for rewrite rules).

----------
components: Library (Lib)
messages: 240287
nosy: brett.cannon
priority: normal
severity: normal
stage: test needed
status: open
title: Forward-port future_builtins
type: enhancement
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23893>
_______________________________________

From report at bugs.python.org  Thu Apr  9 15:18:15 2015
From: report at bugs.python.org (Eli Bendersky)
Date: Thu, 09 Apr 2015 13:18:15 +0000
Subject: [New-bugs-announce] [issue23894] lib2to3 doesn't recognize rb'...'
	as a raw byte string	in Python 3
Message-ID: <1428585495.21.0.103830054834.issue23894@psf.upfronthosting.co.za>


New submission from Eli Bendersky:

lib2to3 tokenizes br'abc' as a single STRING token, but rb'abc' as two separate tokens (NAME "rb" and STRING 'abc')

This is because pgen2/tokenize.py doesn't list rb'' as a viable prefix for a string, even though according to https://docs.python.org/3/reference/lexical_analysis.html it is

----------
keywords: easy
messages: 240322
nosy: benjamin.peterson, eli.bendersky
priority: normal
severity: normal
status: open
title: lib2to3 doesn't recognize rb'...' as a raw byte string in Python 3
versions: Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23894>
_______________________________________

From report at bugs.python.org  Thu Apr  9 15:43:05 2015
From: report at bugs.python.org (Andrew Stormont)
Date: Thu, 09 Apr 2015 13:43:05 +0000
Subject: [New-bugs-announce] [issue23895] PATCH: python socket module fails
	to build on Solaris	when -zignore is in LDFLAGS
Message-ID: <1428586985.69.0.885305793504.issue23895@psf.upfronthosting.co.za>


New submission from Andrew Stormont:

The socket module fails to build when -zignore is in LDFLAGS.  This option changes the linker behaviour so it will only resolve against the libraries linked in explicitly instead of doing resolution recursively against their dependencies too.

----------
components: Build
files: solaris-socket-zignore.diff
keywords: patch
messages: 240325
nosy: andy_js
priority: normal
severity: normal
status: open
title: PATCH: python socket module fails to build on Solaris when -zignore is in LDFLAGS
type: compile error
versions: Python 2.7
Added file: http://bugs.python.org/file38877/solaris-socket-zignore.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23895>
_______________________________________

From report at bugs.python.org  Thu Apr  9 15:49:42 2015
From: report at bugs.python.org (Eli Bendersky)
Date: Thu, 09 Apr 2015 13:49:42 +0000
Subject: [New-bugs-announce] [issue23896] lib2to3 doesn't provide a grammar
	where exec is a	function
Message-ID: <1428587382.51.0.855590572872.issue23896@psf.upfronthosting.co.za>


New submission from Eli Bendersky:

lib2to3 helpfully provides pygram.python_grammar_no_print_statement for parsing Python 3 ('print' has the semantics of an identifier, not a keyword)

However, the same courtesy is not extended to 'exec', which also turns from a statement to an identifier in Python 3.

I'd propose adding something like python_grammar_no_print_and_exec_statement 

The name's a handful, but it's explicit and I can't think of anything else, given that we don't want to change the lib2to3 API at this time to rename these grammars using some other convention.

----------
components: Library (Lib)
messages: 240326
nosy: eli.bendersky
priority: normal
severity: normal
stage: needs patch
status: open
title: lib2to3 doesn't provide a grammar where exec is a function
type: behavior
versions: Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23896>
_______________________________________

From report at bugs.python.org  Thu Apr  9 16:15:35 2015
From: report at bugs.python.org (Nick Coghlan)
Date: Thu, 09 Apr 2015 14:15:35 +0000
Subject: [New-bugs-announce] [issue23897] Update Python 3 extension module
	porting guide
Message-ID: <1428588935.72.0.479238222018.issue23897@psf.upfronthosting.co.za>


New submission from Nick Coghlan:

The extension module porting guide at https://docs.python.org/dev/howto/cporting.html should be updated with Linux distro porting experience.

Barry's notes: https://wiki.python.org/moin/PortingToPy3k/BilingualQuickRef#Python_extension_modules

Petr's compatibility helper tools: https://py3c.readthedocs.org

----------
assignee: barry
messages: 240328
nosy: barry, bkabrda, doko, encukou, kushal.das, ncoghlan
priority: normal
severity: normal
status: open
title: Update Python 3 extension module porting guide

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23897>
_______________________________________

From report at bugs.python.org  Thu Apr  9 16:34:46 2015
From: report at bugs.python.org (mike bayer)
Date: Thu, 09 Apr 2015 14:34:46 +0000
Subject: [New-bugs-announce] [issue23898] inspect() changes in Python3.4 are
	not compatible with	objects that implement special __bool__, __eq__
Message-ID: <1428590086.18.0.0932223784087.issue23898@psf.upfronthosting.co.za>


New submission from mike bayer:

this bug appeared in Python 3.4.    The inspect.classify_class_attrs compares the identity objects of unknown type using the `==` operator unnecessarily and also evaluates objects of unknown type assuming they return `True` for a straight boolean evaluation.  This breaks among other things the ability to use the help() function with SQLAlchemy mapped objects.

Demo:

class MySpecialObject(object):

    def __eq__(self, other):
        return MySpecialObject()

    def __bool__(self):
        raise NotImplementedError(
            "This object does not specify a boolean value")


class MyClass(object):
    some_thing = MySpecialObject()

import inspect

print(inspect.classify_class_attrs(MyClass))

# ultimate goal:  help(MyClass)

A patch here would be to compare unknown objects for identity using the `is` operator as well as using `is not None` when asserting that an object of unknown type is non-None.   This patch resolves:

--- inspect_orig.py	2015-04-09 10:28:46.000000000 -0400
+++ inspect.py	2015-04-09 10:29:37.000000000 -0400
@@ -380,7 +380,7 @@
                     # first look in the classes
                     for srch_cls in class_bases:
                         srch_obj = getattr(srch_cls, name, None)
-                        if srch_obj == get_obj:
+                        if srch_obj is get_obj:
                             last_cls = srch_cls
                     # then check the metaclasses
                     for srch_cls in metamro:
@@ -388,7 +388,7 @@
                             srch_obj = srch_cls.__getattr__(cls, name)
                         except AttributeError:
                             continue
-                        if srch_obj == get_obj:
+                        if srch_obj is get_obj:
                             last_cls = srch_cls
                     if last_cls is not None:
                         homecls = last_cls
@@ -402,7 +402,7 @@
             # unable to locate the attribute anywhere, most likely due to
             # buggy custom __dir__; discard and move on
             continue
-        obj = get_obj or dict_obj
+        obj = get_obj if get_obj is not None else dict_obj
         # Classify the object or its descriptor.
         if isinstance(dict_obj, staticmethod):
             kind = "static method"

----------
components: Library (Lib)
messages: 240331
nosy: zzzeek
priority: normal
severity: normal
status: open
title: inspect() changes in Python3.4 are not compatible with objects that implement special __bool__, __eq__
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23898>
_______________________________________

From report at bugs.python.org  Thu Apr  9 17:36:23 2015
From: report at bugs.python.org (Jason R. Coombs)
Date: Thu, 09 Apr 2015 15:36:23 +0000
Subject: [New-bugs-announce] [issue23899] HTTP regression in distutils
	uploads to chishop
Message-ID: <1428593783.07.0.14236214587.issue23899@psf.upfronthosting.co.za>


New submission from Jason R. Coombs:

Beginning with Python 2.7.9 and 3.4.3, distutils uploads to chishop (https://pypi.python.org/pypi/chishop) now fail with a 501 NOT IMPLEMENTED. This error looks very similar to the error that Twine triggered in https://github.com/pypa/twine/issues/27.

I acknowledge that chishop is old and probably abandoned, but I also expect that a minor update (such as 3.4.2 to 3.4.3) should not break. I understand that 2.7.9 may have special consideration in this regard, so I'll focus on 3.4.3.

I expect these changes in protocol may also affect other HTTP services, though it may be isolated to distutils uploads.

Donald, can you speak to changes in Python 3.4.3 that might have behavior in common with Twine that would have affected distutils uploads?

----------
components: Distutils
keywords: 3.4regression
messages: 240341
nosy: dstufft, eric.araujo, jason.coombs
priority: normal
severity: normal
status: open
title: HTTP regression in distutils uploads to chishop
versions: Python 2.7, Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23899>
_______________________________________

From report at bugs.python.org  Thu Apr  9 18:42:37 2015
From: report at bugs.python.org (Nick Coghlan)
Date: Thu, 09 Apr 2015 16:42:37 +0000
Subject: [New-bugs-announce] [issue23900] Add a default docstring to Enum
	subclasses
Message-ID: <1428597757.76.0.767487050149.issue23900@psf.upfronthosting.co.za>


New submission from Nick Coghlan:

Issue #15582 added docstring inheritance to the inspect module. This means that Enum subclasses without their own docstring now inherit the generic docstring from the base class definition:

>>> import enum, inspect
>>> class MyEnum(enum.Enum):
...   a = 1
... 
>>> inspect.getdoc(MyEnum)
'Generic enumeration.\n\nDerive from this class to define new enumerations.'

Perhaps the metaclass could automatically derive a more suitable docstring if the subclass doesn't set one of its own?

----------
messages: 240348
nosy: barry, eli.bendersky, ethan.furman, ncoghlan, serhiy.storchaka
priority: normal
severity: normal
stage: needs patch
status: open
title: Add a default docstring to Enum subclasses
type: enhancement
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23900>
_______________________________________

From report at bugs.python.org  Thu Apr  9 19:22:37 2015
From: report at bugs.python.org (Paul Moore)
Date: Thu, 09 Apr 2015 17:22:37 +0000
Subject: [New-bugs-announce] [issue23901] Force console stdout to use UTF8
	on Windows
Message-ID: <1428600157.62.0.0673126456323.issue23901@psf.upfronthosting.co.za>


New submission from Paul Moore:

Console code page issues are a consistent source of problems on Windows. It would be nice, given that the Windows console has Unicode support, if Python could write the full range of Unicode to the console by default.

The MSVC runtime appears to have a flag that can be set via _setmode(), _O_U8TEXT, which "enables Unicode mode" (see https://msdn.microsoft.com/en-us/library/tw4k6df8%28v=vs.100%29.aspx?f=255&MSPPError=-2147217396, in particular the second example). It seems as if Python could set U8TEXT mode on sys.stdout on startup (assuming it's a console) and set the encoding to UTF8, and then Unicode output would "just work".

I don't have code that implements this yet, but if I can get my head round the IO stack and the Python startup code, I'll give it a go.

Steve - any comments on whether this might work? I've never seen any real-world code using U8TEXT which makes me wonder if it's reliable (doing msvcrt.setmode(sys.stdout.fileno(), 0x40000) in Python 3.4 causes Python to crash, which is worrying, but it works in 3.5...).

----------
assignee: paul.moore
components: Windows
messages: 240354
nosy: paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Force console stdout to use UTF8 on Windows
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23901>
_______________________________________

From report at bugs.python.org  Thu Apr  9 22:18:28 2015
From: report at bugs.python.org (Travis Everett)
Date: Thu, 09 Apr 2015 20:18:28 +0000
Subject: [New-bugs-announce] [issue23902] let exception react to being
	raised or the setting of	magic properties (like __cause__)
	within Python
Message-ID: <1428610708.13.0.662863454956.issue23902@psf.upfronthosting.co.za>


New submission from Travis Everett:

I've been working on a testing tool which raises its own exceptions from those thrown by code under test. The tool's exceptions do some analysis to categorize and add additional information to the underlying exceptions, and they need access to the __cause__ property in order to build this information.

Unfortunately, because the __cause__ property isn't available on the exception objects at init time, some number of workarounds must be employed which make the exception harder to use properly and code handling it less intuitive. While the workarounds are fine at the moment, it would be ideal if the exceptions could be notified instead of burdening the site of each use with workarounds.

It seems sufficient to call a magic method on the exception immediately after these the traceback/cause/context parameters have been set while it is being raised, allowing the exception to perform any essential work. A bells-and-whistles implementation might be a magic method called at raise time with all of these properties as arguments (and the responsibility to deal with them)--but I assume there are reasons exception objects don't already have this level of control.

----------
messages: 240377
nosy: abathur
priority: normal
severity: normal
status: open
title: let exception react to being raised or the setting of magic properties (like __cause__) within Python
type: enhancement
versions: Python 3.2, Python 3.3, Python 3.4, Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23902>
_______________________________________

From report at bugs.python.org  Fri Apr 10 05:49:21 2015
From: report at bugs.python.org (Zachary Ware)
Date: Fri, 10 Apr 2015 03:49:21 +0000
Subject: [New-bugs-announce] [issue23903] Generate PC/python3.def by
	scraping headers
Message-ID: <1428637761.13.0.348463350169.issue23903@psf.upfronthosting.co.za>


New submission from Zachary Ware:

The attached patch adds a script that can scrape header files to pick out PyAPI_DATA and PyAPI_FUNCs to generate PC/python3.def.  It's not complete and is not integrated into the build at all yet.  I'm planning to work on this further at the PyCon sprints.

Running this and comparing output to the current checked-in python3.def shows that we're not keeping it up-to-date well at all.

----------
assignee: zach.ware
components: Build, Windows
files: add_python3defgen.diff
keywords: patch
messages: 240409
nosy: steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
stage: needs patch
status: open
title: Generate PC/python3.def by scraping headers
type: enhancement
versions: Python 3.5
Added file: http://bugs.python.org/file38886/add_python3defgen.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23903>
_______________________________________

From report at bugs.python.org  Fri Apr 10 13:11:21 2015
From: report at bugs.python.org (Bruno Cauet)
Date: Fri, 10 Apr 2015 11:11:21 +0000
Subject: [New-bugs-announce] [issue23904] pathlib.PurePath does not accept
	bytes components
Message-ID: <1428664281.99.0.802237757059.issue23904@psf.upfronthosting.co.za>


New submission from Bruno Cauet:

At https://docs.python.org/3/library/pathlib.html#pure-paths one can read

> Each element of pathsegments can be either a string or bytes object representing a path segment; it can also be another path object:

which is a lie:

>>> pathlib.PurePath(b"/foo")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/bru/code/cpython/Lib/pathlib.py", line 609, in __new__
    return cls._from_parts(args)
  File "/home/bru/code/cpython/Lib/pathlib.py", line 638, in _from_parts
    drv, root, parts = self._parse_args(args)
  File "/home/bru/code/cpython/Lib/pathlib.py", line 630, in _parse_args
    % type(a))
TypeError: argument should be a path or str object, not <class 'bytes'>

So either
(1) the doc is wrong
(2) PathLib path management fails: it should decode bytes parts with os.fsdecode()
I doubt I tagged both components. I'll be happy to provide a fix once you decide what is the right solution.

I take this opportunity to share an itch: filesystem encoding on Unix cannot be reliably determined. sys.getfilesystemencoding() is only a preference and there is no guarantee that an arbitrary file will respect it. This is extensively discussed in the following thread: https://mail.python.org/pipermail/python-dev/2014-August/135873.html
What is the right way to deal with those? If I use "surrogateescape" (see PEP383) how can I display the fake-unicode path to the user? `print()` does seems to use strict encoding. Should I encode it with "surrogateescape" or "ignore" myself beforehand?

----------
assignee: docs at python
components: Documentation, Library (Lib)
messages: 240414
nosy: bru, docs at python
priority: normal
severity: normal
status: open
title: pathlib.PurePath does not accept bytes components
versions: Python 3.4, Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23904>
_______________________________________

From report at bugs.python.org  Fri Apr 10 17:18:51 2015
From: report at bugs.python.org (Ezequiel Gonzalez)
Date: Fri, 10 Apr 2015 15:18:51 +0000
Subject: [New-bugs-announce] [issue23905] Python here maing CPU g to 100%
Message-ID: <1428679131.49.0.72023198194.issue23905@psf.upfronthosting.co.za>


New submission from Ezequiel Gonzalez:

So I wanted to start on Python!  Downloaded from this site.  
long with it came poweliks.  What gives?  It toasted my laptop to the point i had to replace the HD.

----------
messages: 240424
nosy: zekescript
priority: normal
severity: normal
status: open
title: Python here maing CPU g to 100%
type: resource usage
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23905>
_______________________________________

From report at bugs.python.org  Fri Apr 10 17:26:33 2015
From: report at bugs.python.org (Guillermo Narvaja)
Date: Fri, 10 Apr 2015 15:26:33 +0000
Subject: [New-bugs-announce] [issue23906] poplib maxline behaviour may be
	wrong
Message-ID: <1428679593.1.0.415911204922.issue23906@psf.upfronthosting.co.za>


New submission from Guillermo Narvaja:

After #16041 was fixed, Python started to validate that lines coming from the POP server should be under 2048 bytes. 

This breaks the mail retrieval from at least dovecot servers, as this mail server does not breaks responses in 512 o 2048 sized lines.

On dovecot's side, they said there is a misunderstood of the RFC on the Python side, that the RFC 1939 "is talking about POP3 responses themselves - not about the actual email message body". You can see here the related mail thread:

http://dovecot.org/pipermail/dovecot/2015-April/100475.html

I'm not sure Who is right, but I think it's a problem (at least it was for me).

----------
components: Library (Lib)
messages: 240425
nosy: berker.peksag, christian.heimes, doko, gnarvaja
priority: normal
severity: normal
status: open
title: poplib maxline behaviour may be wrong
type: behavior
versions: Python 2.7, Python 3.2

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23906>
_______________________________________

From report at bugs.python.org  Fri Apr 10 18:27:12 2015
From: report at bugs.python.org (Ezequiel Gonzalez)
Date: Fri, 10 Apr 2015 16:27:12 +0000
Subject: [New-bugs-announce] [issue23907] Python
Message-ID: <4652150F-F3B4-471A-B3E6-17356544D635@me.com>


New submission from Ezequiel Gonzalez:

Ezequiel Gonzalez added the comment:

No.  Clean PC.  Nothing on it.  Eradicated Windows.  I've got several MS bills that thy attempted to root out Poweliks but where unable.  Not to be an azz but you could pay one of them off and see what you think.  Apple side is unaffected.  I threw 8 antivirus/ malware at it and it survived damaged but came back.  Used Apple side to crush Poweliks.

Sent from my iPhone

Sent from my iPhone

----------
messages: 240434
nosy: zekescript
priority: normal
severity: normal
status: open
title: Python

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23907>
_______________________________________

From report at bugs.python.org  Fri Apr 10 18:43:52 2015
From: report at bugs.python.org (Serhiy Storchaka)
Date: Fri, 10 Apr 2015 16:43:52 +0000
Subject: [New-bugs-announce] [issue23908] Check path arguments of os
	functions for null character
Message-ID: <1428684232.77.0.530667773352.issue23908@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka:

Proposed patch adds checks for null character in unicode path arguments of os functions on Windows. Null character is already tested on Unix, in bytes paths on Windows, and in unicode argument of _io.FileIO.

Removed private function _PyUnicode_HasNULChars(), because it is used only in two places and inlined code is simpler and more efficient.

The patch doesn't contain tests because I can't test them. But they should be simple, just pass a path with null character to os function.

----------
components: Extension Modules
files: path_converter_null_char.patch
keywords: patch
messages: 240435
nosy: haypo, serhiy.storchaka
priority: normal
severity: normal
stage: patch review
status: open
title: Check path arguments of os functions for null character
type: behavior
versions: Python 3.4, Python 3.5
Added file: http://bugs.python.org/file38890/path_converter_null_char.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23908>
_______________________________________

From report at bugs.python.org  Fri Apr 10 20:17:36 2015
From: report at bugs.python.org (Mark Mikofski)
Date: Fri, 10 Apr 2015 18:17:36 +0000
Subject: [New-bugs-announce] [issue23909] highlight query string does not
	work on docs.python.org/2
Message-ID: <1428689856.01.0.985094593971.issue23909@psf.upfronthosting.co.za>


New submission from Mark Mikofski:

expected: using Sphinx search in docs, search keyword is usually highlighted on resulting pages.

observerd: query string ?highligh=<string> does nothing on http://docs.python.org/2 although it actually does work for 2.6 and all 3.x docs. Doesn't appear to be a function of Sphinx version as 3.x docs and 2.7.x docs all use Sphinx-1.2.3

examples:

works: https://docs.python.org/3/tutorial/appetite.html?highlight=indent

does'nt: https://docs.python.org/2/tutorial/appetite.html?highlight=indent

platform: works?
chrome: no
firefox: no

----------
assignee: docs at python
components: Documentation
messages: 240443
nosy: bwanamarko, docs at python
priority: normal
severity: normal
status: open
title: highlight query string does not work on docs.python.org/2
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23909>
_______________________________________

From report at bugs.python.org  Fri Apr 10 22:10:37 2015
From: report at bugs.python.org (Joe Jevnik)
Date: Fri, 10 Apr 2015 20:10:37 +0000
Subject: [New-bugs-announce] [issue23910] C implementation of namedtuple
	(WIP)
Message-ID: <1428696637.37.0.716274154339.issue23910@psf.upfronthosting.co.za>


New submission from Joe Jevnik:

I am working on implementing nameduple in C; I am almost there; however, on the path of moving to full compatibility, I ran into a refcount issue somewhere. Hopefully someone can help me work this out.

To describe the issue, When I run the collections tests I most frequently segfault in a non namedtuple test; however, sometimes it runs (and passes) however that probably means I am using an unowned obect somewhere. I am new to the CPython API so I would love to go through this with someone to learn more about how it all works.

I am currently at PyCon and would absolutly love to speak to someone in person. I will be at CPython sprints for at least one day.

----------
components: Extension Modules
files: namedtuple.patch
keywords: patch
messages: 240447
nosy: llllllllll
priority: normal
severity: normal
status: open
title: C implementation  of namedtuple (WIP)
type: enhancement
versions: Python 3.5
Added file: http://bugs.python.org/file38892/namedtuple.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23910>
_______________________________________

From report at bugs.python.org  Sat Apr 11 03:29:38 2015
From: report at bugs.python.org (Eric Snow)
Date: Sat, 11 Apr 2015 01:29:38 +0000
Subject: [New-bugs-announce] [issue23911] Move path-based bootstrap code to
	a separate frozen file.
Message-ID: <1428715778.23.0.540970510492.issue23911@psf.upfronthosting.co.za>


New submission from Eric Snow:

The bootstrap code has a clear division between the core import functionality and the path-based import machinery.  The attached patch makes that division explicit by moving the latter into its own module.  The module is also frozen, necessarily.  In addition to clearly distinguishing the two parts, this division will help with some later work that I'd like to do later with an encapsulated import system abstraction.

The patch uses the name "pathy" throughout, which I'll change to something more descriptive later.  I'll also add the news entry then.

----------
assignee: eric.snow
components: Interpreter Core
files: path-based-importlib.diff
keywords: patch
messages: 240457
nosy: brett.cannon, eric.snow, ncoghlan
priority: normal
severity: normal
stage: patch review
status: open
title: Move path-based bootstrap code to a separate frozen file.
type: enhancement
versions: Python 3.5
Added file: http://bugs.python.org/file38893/path-based-importlib.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23911>
_______________________________________

From report at bugs.python.org  Sat Apr 11 04:15:46 2015
From: report at bugs.python.org (James Edwards)
Date: Sat, 11 Apr 2015 02:15:46 +0000
Subject: [New-bugs-announce] [issue23912] Inconsistent whitespace/formatting
	in	docs/reference/datamodel/Special Method Lookup
Message-ID: <1428718546.88.0.646269142708.issue23912@psf.upfronthosting.co.za>


New submission from James Edwards:

There's inconsistent leading whitespace between the two classes in the 4th code snippet of the "Special Method Lookup" section.

https://docs.python.org/3/reference/datamodel.html#special-method-lookup

The (very substantial :) included patch makes both classes use consistent leading whitespace, that is PEP-8 conformant.

(This issue also exists in the Python 2 documentation[1], but since other things have changed -- e.g. print statement -> function, metaclass declaration -- the same patch won't apply cleanly.)

[1] https://docs.python.org/2/reference/datamodel.html#special-method-lookup-for-new-style-classes

----------
assignee: docs at python
components: Documentation
files: datamodel.rst.diff
keywords: patch
messages: 240458
nosy: docs at python, jedwards
priority: normal
severity: normal
status: open
title: Inconsistent whitespace/formatting in docs/reference/datamodel/Special Method Lookup
versions: Python 3.6
Added file: http://bugs.python.org/file38894/datamodel.rst.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23912>
_______________________________________

From report at bugs.python.org  Sat Apr 11 10:03:59 2015
From: report at bugs.python.org (Boyd Blackwell)
Date: Sat, 11 Apr 2015 08:03:59 +0000
Subject: [New-bugs-announce] [issue23913] error in some byte sizes of docs
	in array module
Message-ID: <1428739439.99.0.273745755962.issue23913@psf.upfronthosting.co.za>


New submission from Boyd Blackwell:

See 8.6. array ? Efficient arrays of numeric values?

I think these two table entries should list 4 instead of 2, at least for 64 python.
The error is currently in 2.710rc0, but also in previous versions.
also in 3.4.3, presumably some previous versions
it might also be argued that the column heading should not but minimum size but simply Element Size (bytes)


'i'	signed int	int	2
'I'	unsigned int	long	2

code:
import array

a = array.array('I')
a.fromstring('\x01\x02\x03\x04')
print(a)
#array('I', [67305985L])  # one element as expected (4 bytes, 4 bytes/elt)

a = array.array('H')
a.fromstring('\x01\x02\x03\x04')
print(a)
#array('H', [513, 1027])  # two elements as expected (4 bytes, 2 bytes/elt)

----------
assignee: docs at python
components: Documentation
files: arraydocbug.py
messages: 240466
nosy: bdb112, docs at python
priority: normal
severity: normal
status: open
title: error in some byte sizes of docs in array module
versions: Python 2.7
Added file: http://bugs.python.org/file38897/arraydocbug.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23913>
_______________________________________

From report at bugs.python.org  Sat Apr 11 14:52:42 2015
From: report at bugs.python.org (Alex Gaynor)
Date: Sat, 11 Apr 2015 12:52:42 +0000
Subject: [New-bugs-announce] [issue23914] pickle fails with SystemError
Message-ID: <1428756762.72.0.925495036586.issue23914@psf.upfronthosting.co.za>


New submission from Alex Gaynor:

>>> pickle.loads(b'(o.\x7f.')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
SystemError: Objects/tupleobject.c:71: bad argument to internal function

(Or the equivalent using cPickle on Python 2)

Found using http://lcamtuf.coredump.cx/afl/

----------
components: Extension Modules
messages: 240475
nosy: alex
priority: normal
severity: normal
status: open
title: pickle fails with SystemError
type: crash
versions: Python 2.7, Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23914>
_______________________________________

From report at bugs.python.org  Sat Apr 11 18:29:20 2015
From: report at bugs.python.org (Travis A. Everett)
Date: Sat, 11 Apr 2015 16:29:20 +0000
Subject: [New-bugs-announce] [issue23915] traceback set with
	BaseException.with_traceback()	overwritten on raise
Message-ID: <1428769760.16.0.251757749986.issue23915@psf.upfronthosting.co.za>


New submission from Travis A. Everett:

When BaseException.with_traceback(tb) is used, the __traceback__ property is properly set, but the property gets overwritten when the exception is raised. 

The attached file demonstrates the issue by raising exception a, which doesn't use with_traceback, and exception b, which uses with_traceback(a.__traceback__). It also demonstrates that the exception object can't observe this change. Executing the attached file produces output like:

a.__traceback__ before raise: [None]
a.__traceback__ after raise : [<traceback object at 0x7f95c5a21708>]
b.__traceback__ before raise: [<traceback object at 0x7f95c5a21708>]
b.__traceback__ after raise : [<traceback object at 0x7f95c5a21748>]

----------
files: exctest.py
messages: 240483
nosy: abathur
priority: normal
severity: normal
status: open
title: traceback set with BaseException.with_traceback() overwritten on raise
type: behavior
versions: Python 3.2, Python 3.3, Python 3.4, Python 3.5, Python 3.6
Added file: http://bugs.python.org/file38898/exctest.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23915>
_______________________________________

From report at bugs.python.org  Sat Apr 11 22:07:34 2015
From: report at bugs.python.org (David Roundy)
Date: Sat, 11 Apr 2015 20:07:34 +0000
Subject: [New-bugs-announce] [issue23916] module importing performance
	regression
Message-ID: <1428782854.88.0.072820618341.issue23916@psf.upfronthosting.co.za>


New submission from David Roundy:

I have observed a performance regression in module importing.  In python 3.4.2, importing a module from the current directory (where the script is located) causes the entire directory to be read.  When there are many files in this directory, this can cause the script to run very slowly.

In python 2.7.9, this behavior is not present.

It would be preferable (in my opinion) to revert the change that causes python to read the entire user directory.

----------
messages: 240491
nosy: daveroundy
priority: normal
severity: normal
status: open
title: module importing performance regression
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23916>
_______________________________________

From report at bugs.python.org  Sun Apr 12 00:52:09 2015
From: report at bugs.python.org (Matthias Klose)
Date: Sat, 11 Apr 2015 22:52:09 +0000
Subject: [New-bugs-announce] [issue23917] please fall back to sequential
	compilation when	concurrent doesn't exist
Message-ID: <1428792729.56.0.740382381475.issue23917@psf.upfronthosting.co.za>


New submission from Matthias Klose:

issue #16104 introduces parallel byte compilation, however the method is now overly strict when workers > 1 and no concurrent support available.  Please just fall back to sequential byte compilation in this case.

----------
components: Library (Lib)
messages: 240507
nosy: Claudiu.Popa, brett.cannon, doko
priority: normal
severity: normal
status: open
title: please fall back to sequential compilation when concurrent doesn't exist
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23917>
_______________________________________

From report at bugs.python.org  Sun Apr 12 01:36:35 2015
From: report at bugs.python.org (Matthias Klose)
Date: Sat, 11 Apr 2015 23:36:35 +0000
Subject: [New-bugs-announce] [issue23918] symbols namespace pollution
Message-ID: <1428795395.74.0.22674392758.issue23918@psf.upfronthosting.co.za>


New submission from Matthias Klose:

3.5 introduces the symbols:

DirEntryType
ScandirIteratorType

introduced by the fix for issue #22524.

These should either be local symbols, or prefixed with _Py or Py.

----------
components: Extension Modules
messages: 240512
nosy: doko, haypo
priority: normal
severity: normal
status: open
title: symbols namespace pollution
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23918>
_______________________________________

From report at bugs.python.org  Sun Apr 12 04:34:27 2015
From: report at bugs.python.org (Zachary Ware)
Date: Sun, 12 Apr 2015 02:34:27 +0000
Subject: [New-bugs-announce] [issue23919] test_os fails several C-level
	assertions
Message-ID: <1428806067.72.0.786420001732.issue23919@psf.upfronthosting.co.za>


New submission from Zachary Ware:

Uninteresting bits of the output elided:

C:\Data\code\CPython\hg.python.org\default>python -m test -vn test_os
Running Debug|Win32 interpreter...
== CPython 3.5.0a3+ (default:0b3027a2abbc, Apr 11 2015, 21:11:57) [MSC v.1900 32
 bit (Intel)]
==   Windows-post2012Server-6.3.9600 little-endian
==   hash algorithm: siphash24 32bit
==   C:\Data\code\CPython\hg.python.org\default\build\test_python_4312
Testing with flags: sys.flags(debug=0, inspect=0, interactive=0, optimize=0, don
t_write_bytecode=0, no_user_site=0, no_site=0, ignore_environment=0, verbose=0,
bytes_warning=0, quiet=0, hash_randomization=1, isolated=0)
[1/1] test_os
...
test_bad_fd (test.test_os.DeviceEncodingTests) ... minkernel\crts\ucrt\src\appcrt\lowio\osfinfo.cpp(257) : Assertion failed: fh >= 0 && (unsigned)fh < (unsigned)_nhandle
ok
...
test_closerange (test.test_os.FileTests) ... minkernel\crts\ucrt\src\appcrt\lowio\osfinfo.cpp(258) : Assertion failed: _osfile(fh) & FOPEN
minkernel\crts\ucrt\src\appcrt\lowio\osfinfo.cpp(258) : Assertion failed: _osfile(fh) & FOPEN
ok
...
test_15261 (test.test_os.StatAttributeTests) ... minkernel\crts\ucrt\src\appcrt\lowio\osfinfo.cpp(258) : Assertion failed: _osfile(fh) & FOPEN
ok
...
test_stty_match (test.test_os.TermsizeTests)
Check if stty returns the same results ... minkernel\crts\ucrt\src\appcrt\lowio\osfinfo.cpp(257) : Assertion failed: fh >= 0 && (unsigned)fh < (unsigned)_nhandle
skipped 'stty invocation failed'
...
test_closerange (test.test_os.TestInvalidFD) ... minkernel\crts\ucrt\src\appcrt\lowio\osfinfo.cpp(258) : Assertion failed: _osfile(fh) & FOPEN
minkernel\crts\ucrt\src\appcrt\lowio\osfinfo.cpp(258) : Assertion failed: _osfile(fh) & FOPEN
minkernel\crts\ucrt\src\appcrt\lowio\osfinfo.cpp(258) : Assertion failed: _osfile(fh) & FOPEN
minkernel\crts\ucrt\src\appcrt\lowio\osfinfo.cpp(258) : Assertion failed: _osfile(fh) & FOPEN
minkernel\crts\ucrt\src\appcrt\lowio\osfinfo.cpp(258) : Assertion failed: _osfile(fh) & FOPEN
minkernel\crts\ucrt\src\appcrt\lowio\osfinfo.cpp(258) : Assertion failed: _osfile(fh) & FOPEN
minkernel\crts\ucrt\src\appcrt\lowio\osfinfo.cpp(258) : Assertion failed: _osfile(fh) & FOPEN
minkernel\crts\ucrt\src\appcrt\lowio\osfinfo.cpp(258) : Assertion failed: _osfile(fh) & FOPEN
minkernel\crts\ucrt\src\appcrt\lowio\osfinfo.cpp(258) : Assertion failed: _osfile(fh) & FOPEN
minkernel\crts\ucrt\src\appcrt\lowio\osfinfo.cpp(258) : Assertion failed: _osfile(fh) & FOPEN
minkernel\crts\ucrt\src\appcrt\lowio\osfinfo.cpp(258) : Assertion failed: _osfile(fh) & FOPEN
minkernel\crts\ucrt\src\appcrt\lowio\osfinfo.cpp(258) : Assertion failed: _osfile(fh) & FOPEN
minkernel\crts\ucrt\src\appcrt\lowio\osfinfo.cpp(258) : Assertion failed: _osfile(fh) & FOPEN
minkernel\crts\ucrt\src\appcrt\lowio\osfinfo.cpp(258) : Assertion failed: _osfile(fh) & FOPEN
minkernel\crts\ucrt\src\appcrt\lowio\osfinfo.cpp(258) : Assertion failed: _osfile(fh) & FOPEN
minkernel\crts\ucrt\src\appcrt\lowio\osfinfo.cpp(258) : Assertion failed: _osfile(fh) & FOPEN
minkernel\crts\ucrt\src\appcrt\lowio\osfinfo.cpp(258) : Assertion failed: _osfile(fh) & FOPEN
minkernel\crts\ucrt\src\appcrt\lowio\osfinfo.cpp(258) : Assertion failed: _osfile(fh) & FOPEN
ok
test_dup (test.test_os.TestInvalidFD) ... minkernel\crts\ucrt\src\appcrt\lowio\osfinfo.cpp(258) : Assertion failed: _osfile(fh) & FOPEN
ok
test_dup2 (test.test_os.TestInvalidFD) ... minkernel\crts\ucrt\src\appcrt\lowio\osfinfo.cpp(258) : Assertion failed: _osfile(fh) & FOPEN
ok
...
test_fdopen (test.test_os.TestInvalidFD) ... minkernel\crts\ucrt\src\appcrt\lowio\osfinfo.cpp(258) : Assertion failed: _osfile(fh) & FOPEN
ok
...
test_fstat (test.test_os.TestInvalidFD) ... minkernel\crts\ucrt\src\appcrt\lowio\osfinfo.cpp(258) : Assertion failed: _osfile(fh) & FOPEN
ok
...
test_fsync (test.test_os.TestInvalidFD) ... minkernel\crts\ucrt\src\appcrt\lowio\osfinfo.cpp(258) : Assertion failed: _osfile(fh) & FOPEN
ok
...
test_inheritable (test.test_os.TestInvalidFD) ... minkernel\crts\ucrt\src\appcrt\lowio\osfinfo.cpp(258) : Assertion failed: _osfile(fh) & FOPEN
minkernel\crts\ucrt\src\appcrt\lowio\osfinfo.cpp(258) : Assertion failed: _osfile(fh) & FOPEN
ok
test_isatty (test.test_os.TestInvalidFD) ... minkernel\crts\ucrt\src\appcrt\lowio\osfinfo.cpp(258) : Assertion failed: _osfile(fh) & FOPEN
ok
test_lseek (test.test_os.TestInvalidFD) ... minkernel\crts\ucrt\src\appcrt\lowio\osfinfo.cpp(258) : Assertion failed: _osfile(fh) & FOPEN
ok
test_read (test.test_os.TestInvalidFD) ... minkernel\crts\ucrt\src\appcrt\lowio\osfinfo.cpp(258) : Assertion failed: _osfile(fh) & FOPEN
ok
...
test_write (test.test_os.TestInvalidFD) ... minkernel\crts\ucrt\src\appcrt\lowio\osfinfo.cpp(258) : Assertion failed: _osfile(fh) & FOPEN
ok
...

----------------------------------------------------------------------
Ran 189 tests in 7.424s

OK (skipped=75)
1 test OK.

----------
components: Windows
messages: 240522
nosy: steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
stage: needs patch
status: open
title: test_os fails several C-level assertions
type: behavior
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23919>
_______________________________________

From report at bugs.python.org  Sun Apr 12 06:24:21 2015
From: report at bugs.python.org (Larry Hastings)
Date: Sun, 12 Apr 2015 04:24:21 +0000
Subject: [New-bugs-announce] [issue23920] Should Clinic have "nullable" or
	types=NoneType?
Message-ID: <1428812661.52.0.406302391077.issue23920@psf.upfronthosting.co.za>


New submission from Larry Hastings:

I was staring at Clinic tonight and a thought came to me.  We could express "nullable=True" as "types='NoneType'".  For example, the converter for 'Z' could change from

    Py_UNICODE(nullable=True)

to

    Py_UNICODE(types="str NoneType")

There Should Be Only One Way To Do It.  Should we get rid of "nullable=True" and use types=NoneType instead?

FWIW it's pretty arbitrary either way; the arguments to the converter are just telling the converter how to generate the code.  I don't think it would make any difference in the converter's code--either way is fine.

----------
assignee: larry
components: Argument Clinic
messages: 240534
nosy: larry, serhiy.storchaka, zach.ware
priority: normal
severity: normal
stage: test needed
status: open
title: Should Clinic have "nullable" or types=NoneType?
type: enhancement
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23920>
_______________________________________

From report at bugs.python.org  Sun Apr 12 06:58:59 2015
From: report at bugs.python.org (James Edwards)
Date: Sun, 12 Apr 2015 04:58:59 +0000
Subject: [New-bugs-announce] [issue23921] Standardize documentation
	whitespace, formatting
Message-ID: <1428814739.03.0.523944029114.issue23921@psf.upfronthosting.co.za>


New submission from James Edwards:

I realize this is a huge patch, I'd be happy to split it to multiple little 
patches (one per file, one per documentation directory, etc.) to make things 
easier.  Just let me know.

The patch attempts to do a few things (with exceptions, as noted below):

    1. Standardize leading whitespace in block bodies (4 spaces per level)
    2. Enforce >= 2 spaces between code and inline comment
    3. In interactive interpreter snippets, ensures lines meant to be entered
        are preceded by '>>> ' or '... '
    4. Inline whitespace standardization (following a liberal / lenient 
        (not strict) PEP-8 testing)

Scanning the documentation, extracting the code snippets, and testing them for
PEP-8 compliance was done with a script, but all adjustments were done by hand.

This means that there remain some code that fails the lenient PEP-8 checks that
I used.  For example, code at [1] ("How to write obfuscated one liners") 
obviously threw errors, but because of the context of the snippet and the use it 
serves in the documentation, it was left it unchanged.

Similarly, there are (intentionally) poorly formatted snippets in [2] ("Lexical
Analysis") that were also left unchanged.

Since these changes were applied by hand, I was able to ignore situations where
things that would normally raise warnings.  I erred on the side of leaving the
documentation examples unchanged, and strived to make only innocuous changes.

I made no attempt to change the functionality or semantics of any of the 
snippets.  The only changes I made were "harmless" formatting.  None of the
changes will affect the function or output of the snippets.

What the changes do, however, is give a consistency to the documentation that
will allow readers to become more comfortable with the structure of the language
and improve readability.

[1] https://docs.python.org/3/faq/programming.html#is-it-possible-to-write-obfuscated-one-liners-in-python
[2] https://docs.python.org/3/reference/lexical_analysis.html#indentation
[3] http://pep8.readthedocs.org/en/latest/intro.html

In addition to the checks that are ignored by default by the pep8[3] module for 
not being unanimously accepted:

    E121    # continuation line under-indented for hanging indent
    E123    # closing bracket does not match indentation of opening bracket?s line
    E126    # continuation line over-indented for hanging indent
    E133    # closing bracket is missing indentation
    E226    # missing whitespace around arithmetic operator
    E241    # multiple spaces after ?,? 
    E242    # tab after ?,?
    E704    # multiple statements on one line (def)

The following checks were "globally" ignored in my script, though many others 
were conditionally ignored by the script or by myself.

    ignore.append('W292')       # no newline at end of file
    ignore.append('E302')       # expected 2 blank lines, found 0
    ignore.append('E401')       # multiple imports on one line
    ignore.append('W391')       # blank line at end of file
    ignore.append('E231')       # missing whitespace after ','
    ignore.append('E114')       # indentation is not multiple of four (comment)
    ignore.append('E116')       # unexpected indentation (comment)
    
While the patch diffstat is:

    67 files changed, 450 insertions(+), 412 deletions(-)

Ignoring all whitespace changes, the diffstat is only:

    10 files changed, 118 insertions(+), 114 deletions(-)

The majority of these remaining changes fix the inconsistency of interactive interpreter snippets, where, within the same snippet, some lines have '>>> ' while others are missing it, or the '... ' continuation prompt.

Let me know if you need anything from me (such as splitting this patch up) to
help get it merged.  Thanks.

----------
assignee: docs at python
components: Documentation
files: docsdiff.diff
keywords: patch
messages: 240540
nosy: docs at python, jedwards
priority: normal
severity: normal
status: open
title: Standardize documentation whitespace, formatting
type: enhancement
Added file: http://bugs.python.org/file38903/docsdiff.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23921>
_______________________________________

From report at bugs.python.org  Sun Apr 12 20:00:15 2015
From: report at bugs.python.org (Al Sweigart)
Date: Sun, 12 Apr 2015 18:00:15 +0000
Subject: [New-bugs-announce] [issue23922] turtle.py and turtledemo use the
	default tkinter icon
Message-ID: <1428861614.96.0.721771203423.issue23922@psf.upfronthosting.co.za>


New submission from Al Sweigart:

The turtle.py and turtledemo scripts launch windows that have the default tkinter icon. Instead, they should make use of the IDLE icon (which are already in the idlelib/Icons folder)

----------
components: IDLE
messages: 240564
nosy: Al.Sweigart
priority: normal
severity: normal
status: open
title: turtle.py and turtledemo use the default tkinter icon
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23922>
_______________________________________

From report at bugs.python.org  Sun Apr 12 23:54:03 2015
From: report at bugs.python.org (Antoon Houben)
Date: Sun, 12 Apr 2015 21:54:03 +0000
Subject: [New-bugs-announce] [issue23923] Integer operations (// or %) on
	negative numbers product	wrong results.
Message-ID: <1428875643.44.0.160297907605.issue23923@psf.upfronthosting.co.za>


New submission from Antoon Houben:

-17 // 3 should produce -5 but is currently producing -6
-17 % 3 should produce 2 but is currently producing 1

----------
messages: 240569
nosy: Antoon
priority: normal
severity: normal
status: open
title: Integer operations (// or %) on negative numbers product wrong results.
type: behavior
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23923>
_______________________________________

From report at bugs.python.org  Mon Apr 13 02:18:19 2015
From: report at bugs.python.org (Carol Willing)
Date: Mon, 13 Apr 2015 00:18:19 +0000
Subject: [New-bugs-announce] [issue23924] Add OS X to Dev Guide Quickstart
	build step
Message-ID: <1428884299.95.0.569526514807.issue23924@psf.upfronthosting.co.za>


New submission from Carol Willing:

Since many devs use OS X, it would be helpful to add a mention of OS X in Step 2 (Build). It would also be helpful to add that it is okay to ignore build errors/warnings as long as CPython reports building correctly.

----------
components: Devguide
messages: 240575
nosy: ezio.melotti, willingc
priority: normal
severity: normal
status: open
title: Add OS X to Dev Guide Quickstart build step

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23924>
_______________________________________

From report at bugs.python.org  Mon Apr 13 03:18:13 2015
From: report at bugs.python.org (Jamiel Almeida)
Date: Mon, 13 Apr 2015 01:18:13 +0000
Subject: [New-bugs-announce] [issue23925] test_cmd_line failing on
	PYTHONSTARTUP
Message-ID: <1428887893.96.0.607136539753.issue23925@psf.upfronthosting.co.za>


New submission from Jamiel Almeida:

Running the test test_cmd_line with my current PYTHONSTARTUP set to the attached file breaks the test.

$ ./python.exe -m test test_cmd_line

Changing line 280 on Lib/test/test_cmd_line.py to include either -I or -E makes the error be different but still breaks the test.

adding a line with `del env['PYTHONSTARTUP']` before the subprocess works around the issue.

This leads me to believe that -E and -I aren't fully ignoring the environment variable.

Running with:
$ PYTHONSTARTUP= ./python.exe -m test test_cmd_line

Also works around the issue.

Ran the following to build python and run tests

$ hg clone https://hg.python.org/cpython
$ cd cpython
$ ./configure --with-pydebug && make -j4
$ ./python -m test -j4

Currently on revision: 2a18f6b85da2 2015-04-12 | [rdmurray] #23464: remove JoinableQueue that was deprecated in 3.4.4.

----------
components: Interpreter Core
files: pythonrc
messages: 240581
nosy: slashfoo
priority: normal
severity: normal
status: open
title: test_cmd_line failing on PYTHONSTARTUP
versions: Python 3.5
Added file: http://bugs.python.org/file38911/pythonrc

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23925>
_______________________________________

From report at bugs.python.org  Mon Apr 13 04:44:00 2015
From: report at bugs.python.org (Larry Hastings)
Date: Mon, 13 Apr 2015 02:44:00 +0000
Subject: [New-bugs-announce] [issue23926] skipitem() in getargs.c still
	supports 'w' and 'w#', and shouldn't
Message-ID: <1428893040.89.0.319300847517.issue23926@psf.upfronthosting.co.za>


New submission from Larry Hastings:

tl;dr: skipitem() in Python 3 still supports "w" and "w#", which were abandoned.  They should be removed.

--

If: 
* you have an extension that parses its arguments with
  PyArg_ParseTupleAndKeywords (or its cousins),
* you have optional positional parameters, and
* you run out of positional arguments

CPython will call skipitem() to skip over the remaining positional parameters so it can process keyword arguments.  It does this by knowing all the format units, and iterating over them and throwing away the various varargs pointers until it hits the keyword arguments part of the format string.

PyArg_ParseTuple() etc. in Python 2 supported "w" and "w#" for parsing "objects with the read-write buffer interface".  These were removed in Python 3.

skipitem() in Python 3 still supports both skipping "w" and "w#".  But in fact the only legal format unit starting with a 'w' in Python 3 is 'w*'.  So a function with a 'w*' as an optional parameter that got skipped would be misinterpreted; skipitem would see the 'w', not recognize the following '*' and leave it there, then the next function that read a character from the format string (either skipitem() or convertsimple()) would see the '*' and throw an "impossible <bad format char>" exception.

----------
assignee: larry
messages: 240587
nosy: larry
priority: low
severity: normal
stage: test needed
status: open
title: skipitem() in getargs.c still supports 'w' and 'w#', and shouldn't
type: crash
versions: Python 3.4, Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23926>
_______________________________________

From report at bugs.python.org  Mon Apr 13 04:44:28 2015
From: report at bugs.python.org (Larry Hastings)
Date: Mon, 13 Apr 2015 02:44:28 +0000
Subject: [New-bugs-announce] [issue23927] getargs.c skipitem() doesn't skip
	'w*'
Message-ID: <1428893068.41.0.966263121411.issue23927@psf.upfronthosting.co.za>


New submission from Larry Hastings:

tl;dr: skipitem() in Python/getsargs.c doesn't know about the 'w*' format unit, which could lead to a crash.  It should know about it.

--

If: 
* you have an extension that parses its arguments with
  PyArg_ParseTupleAndKeywords (or its cousins),
* you have optional positional parameters, and
* you run out of positional arguments

CPython will call skipitem() to skip over the remaining positional parameters so it can process keyword arguments.  It does this by knowing all the format units, and iterating over them and throwing away the various varargs pointers until it hits the keyword arguments part of the format string.

Since nobody ever explained about the "w*" format unit to it, that means when it sees 'w*' it will misinterpret it.

In both 2.7 and 3.x, it'll interpret it as 'w', throw away one "char *" item from varargs, then the next thing that tries to parse a format unit (skipitem or convertsimple) will see the '*' and throw an 'impossible<bad format char>' exception.

(It's another bug that skipitem() supports 'w' and 'w#' in Python 3, but I filed that separately as #23926 .)

----------
assignee: larry
components: Interpreter Core
messages: 240588
nosy: larry
priority: low
severity: normal
stage: test needed
status: open
title: getargs.c skipitem() doesn't skip 'w*'
type: crash
versions: Python 2.7, Python 3.4, Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23927>
_______________________________________

From report at bugs.python.org  Mon Apr 13 09:17:28 2015
From: report at bugs.python.org (Jeffrey Walton)
Date: Mon, 13 Apr 2015 07:17:28 +0000
Subject: [New-bugs-announce] [issue23928] SSL wiki page, host name matching,
	CN and SAN
Message-ID: <1428909448.45.0.483164672396.issue23928@psf.upfronthosting.co.za>


New submission from Jeffrey Walton:

The Python wiki page on SSL states (https://wiki.python.org/moin/SSL):

    To validate that a certificate matches requested site,
    you need to check commonName field in the subject of
    the certificate.

I don't think its quite correct.

Both the IETF and the CA/B Forums deprecated the use of a hostname or IP address in the commonName (CN). All hostnames and IP addresses must be listed in the subjectAlternateName (SAN), and that's where to look for them.

Though deprecated, placing a name in the CN is not forbidden. In fact, RFC 6125 states the CN should be used as a "last resort" in Section 6.4.4:

    Therefore, if and only if the presented identifiers
    do not include a DNS-ID, SRV-ID, URI-ID, or any
    application-specific identifier types supported by
    the client, then the client MAY as a last resort check
    for a string whose form matches that of a fully
    qualified DNS domain name in a Common Name field of
    the subject field (i.e., a CN-ID). 

Following the advice on the wiki might lead to a Type II error, where an otherwise good certificate is rejected. Its not as bad as accepting a bad certificate, though (by omitting the hostname matching checks).

The IETF deprecated the practice of placing a name in the CN in RFC 6125, Section 6.4.4. The CA/Browser Forum deprecated a DNS name in the CN in Baseline Requirements (BR) Section 9.2.2 Subject Common Name Field.

----------
assignee: docs at python
components: Documentation
messages: 240590
nosy: Jeffrey.Walton, docs at python
priority: normal
severity: normal
status: open
title: SSL wiki page, host name matching, CN and SAN
type: enhancement

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23928>
_______________________________________

From report at bugs.python.org  Mon Apr 13 09:30:11 2015
From: report at bugs.python.org (Philippe)
Date: Mon, 13 Apr 2015 07:30:11 +0000
Subject: [New-bugs-announce] [issue23929] Minor typo (way vs. away) in
	os.renames docstring
Message-ID: <1428910211.95.0.877861303749.issue23929@psf.upfronthosting.co.za>


New submission from Philippe:

There is a minor typo in the docstring of os.renames:

See https://hg.python.org/cpython/file/37905786b34b/Lib/os.py#l275

"After the rename, directories corresponding to rightmost path segments of the old name will be pruned way until [..]"

This should be using away instead of way:

"After the rename, directories corresponding to rightmost path segments of the old name will be pruned away until [..]"

----------
assignee: docs at python
components: Documentation
messages: 240595
nosy: docs at python, pombreda
priority: normal
severity: normal
status: open
title: Minor typo (way vs. away) in os.renames docstring
type: enhancement
versions: Python 2.7, Python 3.2, Python 3.3, Python 3.4, Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23929>
_______________________________________

From report at bugs.python.org  Mon Apr 13 15:17:33 2015
From: report at bugs.python.org (Piotr)
Date: Mon, 13 Apr 2015 13:17:33 +0000
Subject: [New-bugs-announce] [issue23930] SimpleCookie doesn't parse
	comma-only separated cookies	correctly
Message-ID: <1428931053.68.0.491418357539.issue23930@psf.upfronthosting.co.za>


New submission from Piotr:

Skype WISPr and iPassConnect (and maybe other bots) return cookies as a comma separated list. It's not a comma + space (which works).

C = cookies.SimpleCookie()
C.load('a=b,z=zz')
>>> C['a']
<Morsel: a='b,z=zz'>

I wonder what would those bots do if there was a comma in a cookie value.

----------
components: Library (Lib)
messages: 240603
nosy: riklaunim
priority: normal
severity: normal
status: open
title: SimpleCookie doesn't  parse comma-only separated cookies correctly
type: behavior
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23930>
_______________________________________

From report at bugs.python.org  Mon Apr 13 15:39:49 2015
From: report at bugs.python.org (Carol Willing)
Date: Mon, 13 Apr 2015 13:39:49 +0000
Subject: [New-bugs-announce] [issue23931] Update DevGuide link in Quickstart
	Step 1
Message-ID: <1428932389.16.0.201041171689.issue23931@psf.upfronthosting.co.za>


New submission from Carol Willing:

Change Quickstart step 1 link to redirect to Section 1.1 Getting Started. Currently, it redirects to 1.1.2 Getting the Source Code. For developers coming from git, step 1.1.1 Version Control Setup would be helpful.

Feedback from PyCon 2015 Sprints suggest this change.

----------
components: Devguide
messages: 240605
nosy: ezio.melotti, willingc
priority: normal
severity: normal
status: open
title: Update DevGuide link in Quickstart Step 1
type: behavior

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23931>
_______________________________________

From report at bugs.python.org  Mon Apr 13 15:39:54 2015
From: report at bugs.python.org (Zachary Ware)
Date: Mon, 13 Apr 2015 13:39:54 +0000
Subject: [New-bugs-announce] [issue23932] Tutorial section on function
	annotations is out of date	re: PEP 484
Message-ID: <1428932394.25.0.74013439586.issue23932@psf.upfronthosting.co.za>


New submission from Zachary Ware:

(Brought up by 'Tronster' on docs@)

https://docs.python.org/3/tutorial/controlflow.html#function-annotations

In light of the PEP 484 and its clear intentions for function annotations, the tutorial section on function annotations is out of date.  It should be updated to be clear that function annotations are intended for type hints, and that other uses are no longer encouraged.  The example should also use some built-in types in a manner consistent with PEP 484.

----------
assignee: docs at python
components: Documentation
messages: 240606
nosy: docs at python, gvanrossum, tim.golden, zach.ware
priority: normal
severity: normal
stage: needs patch
status: open
title: Tutorial section on function annotations is out of date re: PEP 484
versions: Python 3.4, Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23932>
_______________________________________

From report at bugs.python.org  Mon Apr 13 16:25:51 2015
From: report at bugs.python.org (Anderson)
Date: Mon, 13 Apr 2015 14:25:51 +0000
Subject: [New-bugs-announce] [issue23933] Struct module should acept arrays
Message-ID: <1428935151.96.0.568768704224.issue23933@psf.upfronthosting.co.za>


New submission from Anderson:

Correct me if I'm wrong, the struct module does not work with array of ints, floats etc (just work with char in the form of strings). I think it should since this are valid elements in C structs. 

More specifically, consider I have this C struct

struct{
 int array[4];
};

I'm forced to do something like this:
  struct.pack('iiii', v1,v2,v3,v4)  #'4i' is just the same as 'iiii'

I would like to do something like this:
  struct.pack('i[4]', [v1,v2,v3,v4])

Of course this is useful if I want to pack with zeros:
  struct.pack('i[4]', [0]*4)

----------
messages: 240610
nosy: gamaanderson
priority: normal
severity: normal
status: open
title: Struct module should acept arrays
type: enhancement
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23933>
_______________________________________

From report at bugs.python.org  Mon Apr 13 18:38:13 2015
From: report at bugs.python.org (Nick Coghlan)
Date: Mon, 13 Apr 2015 16:38:13 +0000
Subject: [New-bugs-announce] [issue23934] inspect.signature reporting "()"
	for all builtin &	extension types
Message-ID: <1428943093.46.0.94678170323.issue23934@psf.upfronthosting.co.za>


New submission from Nick Coghlan:

inspect.signature isn't currently handling builtin & extension types correctly - these show up as having neither __new__ *nor* __init__ as pure Python callables, so the inspect.signature logic falls through into a currently unhandled case.

_testcapi isn't currently exporting an extension type as docstring introspection fodder, only callables, so test_inspect didn't pick up the problem. The problem can be seen with builtin types like str:

>>> import inspect
>>> inspect.signature(str)
<inspect.Signature object at 0x7fb81d44e518>
>>> print(inspect.signature(str))
()

Expected behaviour would be to throw a ValueError as with builtin callables without a signature:

>>> import _testcapi
>>> import inspect
>>> inspect.signature(_testcapi.docstring_no_signature)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/ncoghlan/devel/py3k/Lib/inspect.py", line 2830, in signature
    return Signature.from_callable(obj)
  File "/home/ncoghlan/devel/py3k/Lib/inspect.py", line 2586, in from_callable
    return _signature_from_callable(obj, sigcls=cls)
  File "/home/ncoghlan/devel/py3k/Lib/inspect.py", line 2064, in _signature_from_callable
    skip_bound_arg=skip_bound_arg)
  File "/home/ncoghlan/devel/py3k/Lib/inspect.py", line 1984, in _signature_from_builtin
    raise ValueError("no signature found for builtin {!r}".format(func))
ValueError: no signature found for builtin <built-in function docstring_no_signature>

----------
messages: 240649
nosy: james, larry, ncoghlan, yselivanov
priority: normal
severity: normal
stage: test needed
status: open
title: inspect.signature reporting "()" for all builtin & extension types
type: behavior
versions: Python 3.4, Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23934>
_______________________________________

From report at bugs.python.org  Mon Apr 13 18:53:54 2015
From: report at bugs.python.org (Larry Hastings)
Date: Mon, 13 Apr 2015 16:53:54 +0000
Subject: [New-bugs-announce] [issue23935] Clean up Clinic's type expressions
	of buffers
Message-ID: <1428944034.0.0.309169492838.issue23935@psf.upfronthosting.co.za>


New submission from Larry Hastings:

Clinic was previously pretty confused about what things accepted "bytes", "bytearray", "buffer", "robuffer", and "rwbuffer".  This is because the documentation itself was somewhat confusing.

The documentation was recently cleaned up on these in trunk (including one final fix from me this morning).  This patch cleans up Clinic's understanding of how the types map to the format units.

My notes on how the format units map to types is copy & pasted below.

--


the documentation for format units is a little confused, and it is therefore confusing

inside:


getbuffer (buffer)
y* s* z*
  uses PyObject_GetBuffer(arg, view, PyBUF_SIMPLE)
  requires PyBuffer_IsContiguous(view, 'C')
  doesn't check specifically for bytes objects
  handles bytes objects
  accepts bytesarray
 

convertbuffer (robuffer)
y y# s# z#
  handles "read-only buffer object",
  all it really does is ensure it doesn't have a pb_releasebuffer,
  then call getbuffer() above

  so it doesn't accept bytesarray, *just because*


doesn't accept bytes
s z es es#

requires writeable buffer (rwbuffer)
w*

actually checks / specifically handles bytes objects
S

actually checks / specifically handles bytearray objects
Y

actually checks / specifically handles bytes and bytearray objects
c et et#

----------
assignee: larry
components: Argument Clinic
messages: 240660
nosy: larry, serhiy.storchaka, zach.ware
priority: normal
severity: normal
stage: patch review
status: open
title: Clean up Clinic's type expressions of buffers
type: behavior
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23935>
_______________________________________

From report at bugs.python.org  Mon Apr 13 19:22:55 2015
From: report at bugs.python.org (=?utf-8?q?Ra=C3=BAl_Cumplido?=)
Date: Mon, 13 Apr 2015 17:22:55 +0000
Subject: [New-bugs-announce] [issue23936] Wrong references to deprecated
	find_module instead of	find_spec
Message-ID: <1428945775.22.0.0538000929194.issue23936@psf.upfronthosting.co.za>


New submission from Ra?l Cumplido:

While taking a look on the import mechanisms I've seen in the documentation that find_module has been deprecated for find_spec, but on different parts of the documentation there are still references to find_module, as in the definition of sys.meta_path (https://docs.python.org/3/library/sys.html#sys.meta_path).

Shouldn't it be (example on this case) a list of finder objects that have their find_spec() methods called, instead of find_module method?

I've been taking a look on _bootstrap.py and I can see we call find_spec:

for finder in sys.meta_path:
    with _ImportLockContext():
        try:
            find_spec = finder.find_spec

If you agree with me that this is wrong I'll submit a patch to fix it.

----------
assignee: docs at python
components: Documentation
messages: 240671
nosy: brett.cannon, docs at python, eric.snow, raulcd
priority: normal
severity: normal
status: open
title: Wrong references to deprecated find_module instead of find_spec
versions: Python 3.4, Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23936>
_______________________________________

From report at bugs.python.org  Mon Apr 13 22:05:32 2015
From: report at bugs.python.org (Aidan Lowe)
Date: Mon, 13 Apr 2015 20:05:32 +0000
Subject: [New-bugs-announce] [issue23937] IDLE start maximized
Message-ID: <1428955532.11.0.740629698052.issue23937@psf.upfronthosting.co.za>


New submission from Aidan Lowe:

I feel like IDLE should start maximized as opposed to starting... 'unmaximized' I guess. Maybe instead of the current option to start the window at 'blank' by 'blank' size, it should be a check button that defaults to maximized and can be turned off to start it at the default height and width.

----------
components: IDLE
messages: 240725
nosy: zektron42
priority: normal
severity: normal
status: open
title: IDLE start maximized
type: enhancement
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23937>
_______________________________________

From report at bugs.python.org  Mon Apr 13 22:08:34 2015
From: report at bugs.python.org (Alex Walters)
Date: Mon, 13 Apr 2015 20:08:34 +0000
Subject: [New-bugs-announce] [issue23938] Deprecation of windows xp support
	missing from whats new
Message-ID: <1428955714.97.0.53855207407.issue23938@psf.upfronthosting.co.za>


New submission from Alex Walters:

the Whats New in Python 3.5 lacks an indication that cpython is dropping support for that platform due to Microsoft dropping continuing support for it (see PEP 11).

----------
files: docs_unsupported_xp_35.patch
keywords: patch
messages: 240727
nosy: tritium
priority: normal
severity: normal
status: open
title: Deprecation of windows xp support missing from whats new
versions: Python 3.5
Added file: http://bugs.python.org/file38951/docs_unsupported_xp_35.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23938>
_______________________________________

From report at bugs.python.org  Mon Apr 13 22:39:11 2015
From: report at bugs.python.org (Alex Lord)
Date: Mon, 13 Apr 2015 20:39:11 +0000
Subject: [New-bugs-announce] [issue23939] test_get_platform_osx failure on
	Python 3.5.0a0 osx 10.6
Message-ID: <1428957551.09.0.505644010744.issue23939@psf.upfronthosting.co.za>


New submission from Alex Lord:

On a fresh clone of cpython 3.5.0a0 if you run

$ ./configure --with-pydebug && make -j2
$ ./python.exe -m test.test__osx_support -j3

on osx 10.10.2 (14C109) these two test failures are reported.

    ======================================================================
    FAIL: test_get_platform_osx (__main__.Test_OSXSupport)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "/Users/alexlord/mercurial/cpython/Lib/test/test__osx_support.py", line 272, in test_get_platform_osx
        self.assertEqual(('macosx', '10.6', 'fat'), result)
    AssertionError: Tuples differ: ('macosx', '10.6', 'fat') != ('macosx', '10.6', ' ')

    First differing element 2:
    fat


    - ('macosx', '10.6', 'fat')
    ?                     ^^^

    + ('macosx', '10.6', ' ')
    ?                     ^


    ----------------------------------------------------------------------
    Ran 14 tests in 0.354s


Doing a little more digging I found that this if statement is the one failing.

    if ((macrelease + '.') >= '10.4.' and$
                 '-arch' in cflags.strip()):

Specifically this line 

    (macrelease + '.') >= '10.4'

I used distutils.version.StrictVersion to solve this comparison error.

which is failing because

'10.10' >= '10.4' # This fails because the character 4 is greater than 1.

----------
files: test_get_platform_osx.patch
keywords: patch
messages: 240741
nosy: Alex.Lord
priority: normal
severity: normal
status: open
title: test_get_platform_osx failure on Python 3.5.0a0 osx 10.6
versions: Python 3.5
Added file: http://bugs.python.org/file38962/test_get_platform_osx.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23939>
_______________________________________

From report at bugs.python.org  Mon Apr 13 23:19:25 2015
From: report at bugs.python.org (Alex Lord)
Date: Mon, 13 Apr 2015 21:19:25 +0000
Subject: [New-bugs-announce] [issue23940] test__supports_universal_builds
	failure on Python	3.5.0a0 osx 10.6
Message-ID: <1428959965.47.0.850349280401.issue23940@psf.upfronthosting.co.za>


New submission from Alex Lord:

On a fresh clone of cpython 3.5.0a0 if you run

$ ./configure --with-pydebug && make -j2
$ ./python.exe -m test.test__osx_support -j3

on osx 10.10.2 (14C109) these two test failures are reported.


    ======================================================================
    FAIL: test__supports_universal_builds (__main__.Test_OSXSupport)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "/Users/alexlord/mercurial/cpython/Lib/test/test__osx_support.py", line 113, in test__supports_universal_builds
        _osx_support._supports_universal_builds())
    AssertionError: False != True

This turned out to be a problem with the line 12 in test_osx_support.py

        self.assertEqual(platform.mac_ver()[0].split('.') >= ['10', '4'],$
                             _osx_support._supports_universal_builds())$

Specifically the section

    platform.mac_ver()[0].split('.') >= ['10', '4']

Which reduced to

    ['10', '10', '2'] >= ['10', '4] which evaluated to False.

To fix this I imported distutils.version.StrictVersion and added these three lines.

+        mac_version = StrictVersion(platform.mac_ver()[0])
+        test_if_greater_version = StrictVersion('10.4')
+        self.assertEqual(mac_version >= test_if_greater_version,

----------
files: test__supports_universal_builds.patch
keywords: patch
messages: 240758
nosy: Alex.Lord
priority: normal
severity: normal
status: open
title: test__supports_universal_builds failure on Python 3.5.0a0 osx 10.6
Added file: http://bugs.python.org/file38968/test__supports_universal_builds.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23940>
_______________________________________

From report at bugs.python.org  Mon Apr 13 23:32:24 2015
From: report at bugs.python.org (Ernest)
Date: Mon, 13 Apr 2015 21:32:24 +0000
Subject: [New-bugs-announce] [issue23941] Google Login not working
Message-ID: <1428960744.3.0.396656923698.issue23941@psf.upfronthosting.co.za>


New submission from Ernest:

What the hell.  I will create this because there doesn't seem to be a issue being tracked related to it with the keyword "Google" and "Login" in it.

----------
messages: 240763
nosy: ozialien
priority: normal
severity: normal
status: open
title: Google Login not working
type: behavior

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23941>
_______________________________________

From report at bugs.python.org  Tue Apr 14 01:02:17 2015
From: report at bugs.python.org (Maciej Szulik)
Date: Mon, 13 Apr 2015 23:02:17 +0000
Subject: [New-bugs-announce] [issue23942] Explain naming of the patch files
	in the bug tracker
Message-ID: <1428966137.48.0.790822523429.issue23942@psf.upfronthosting.co.za>


New submission from Maciej Szulik:

During the lifecycle of a bug you do:
1. submit a patch
2. get a review with some comments
3. re-submit the patch

Talking to R. David Murray I got info there's no convention on naming those, I think it will be valuable to have some information in the devguide about that. 

Apparently you can either (when just fixing the patch) to stick with exactly the same name or add increasing suffix, eg. mypatch-1, mypatch-2, etc.).  Otherwise you can change the name, but the review system will be able to pick up the right patch, so you should not worry about that.

----------
components: Devguide
messages: 240800
nosy: ezio.melotti, maciej.szulik, willingc
priority: normal
severity: normal
status: open
title: Explain naming of the patch files in the bug tracker

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23942>
_______________________________________

From report at bugs.python.org  Tue Apr 14 03:39:36 2015
From: report at bugs.python.org (Piotr Kasprzyk)
Date: Tue, 14 Apr 2015 01:39:36 +0000
Subject: [New-bugs-announce] [issue23943] Misspellings in a few files
Message-ID: <1428975576.21.0.663921788116.issue23943@psf.upfronthosting.co.za>


New submission from Piotr Kasprzyk:

Correct misspellings in various files.

----------
assignee: docs at python
components: Documentation
files: misspellings.patch
keywords: patch
messages: 240825
nosy: docs at python, kwadrat
priority: normal
severity: normal
status: open
title: Misspellings in a few files
type: enhancement
versions: Python 3.5
Added file: http://bugs.python.org/file38977/misspellings.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23943>
_______________________________________

From report at bugs.python.org  Tue Apr 14 04:28:04 2015
From: report at bugs.python.org (Zachary Ware)
Date: Tue, 14 Apr 2015 02:28:04 +0000
Subject: [New-bugs-announce] [issue23944] Argument Clinic: wrap impl's
	declaration if it's too long
Message-ID: <1428978484.39.0.888495674318.issue23944@psf.upfronthosting.co.za>


New submission from Zachary Ware:

For example:

_winapi_CreateFile_impl(PyModuleDef *module, LPCTSTR file_name, DWORD desired_access, DWORD share_mode, LPSECURITY_ATTRIBUTES security_attributes, DWORD creation_disposition, DWORD flags_and_attributes, HANDLE template_file)

----------
components: Argument Clinic
messages: 240831
nosy: larry, zach.ware
priority: low
severity: normal
stage: needs patch
status: open
title: Argument Clinic: wrap impl's declaration if it's too long
type: enhancement
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23944>
_______________________________________

From report at bugs.python.org  Tue Apr 14 11:51:09 2015
From: report at bugs.python.org (andig2)
Date: Tue, 14 Apr 2015 09:51:09 +0000
Subject: [New-bugs-announce] [issue23945] webbrowser.open opens twice on
	Windows if BROWSER is set
Message-ID: <1429005069.92.0.435113491074.issue23945@psf.upfronthosting.co.za>


New submission from andig2:

Setup: IE default browser, FF37 preferred browser and configured in BROWSER env variable

Test:

import webbrowser
url = "http://localhost"
webbrowser.open(url + '?XDEBUG_SESSION_START=sublime.xdebug')

Behaviour:

1st run: FF opens -> ok
2nd run: FF opens new tab -> ok, in addition, IE opens -> WRONG

----------
components: Library (Lib)
messages: 240863
nosy: andig2
priority: normal
severity: normal
status: open
title: webbrowser.open opens twice on Windows if BROWSER is set
type: behavior
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23945>
_______________________________________

From report at bugs.python.org  Tue Apr 14 13:56:59 2015
From: report at bugs.python.org (Cristi Fati)
Date: Tue, 14 Apr 2015 11:56:59 +0000
Subject: [New-bugs-announce] [issue23946] Invalid timestamps reported by
	os.stat() when Windows	FILETIME structures are mistakenly reset.
Message-ID: <1429012619.96.0.846306201826.issue23946@psf.upfronthosting.co.za>


New submission from Cristi Fati:

In WinPE environment (at least this is where I found it, but there might be cases where this issue could be spotted in normal Windows), some folders might have some of the FILETIME structures reset (because of RAMDrive?). When the conversion between WIN style (100 * nsecs from  1601/01/01) to UX style (secs from 1970/01/01) is taking place (epoch is being substracted), it could yield invalid filetime values (that will be reported by os.stat).

The attached patch simply goes around this initializing the UX timestamps to 0.

It was found in Python-2.7.3, the patch is for 2.7.8, but I think it can be reproduced with all the newer versions.

----------
components: Interpreter Core, Windows
files: Python-2.7.8-invalid_filetimes.patch
keywords: patch
messages: 240875
nosy: CristiFati, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Invalid timestamps reported by os.stat() when Windows FILETIME structures are mistakenly reset.
type: behavior
versions: Python 2.7
Added file: http://bugs.python.org/file38985/Python-2.7.8-invalid_filetimes.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23946>
_______________________________________

From report at bugs.python.org  Tue Apr 14 16:45:19 2015
From: report at bugs.python.org (Steve Dower)
Date: Tue, 14 Apr 2015 14:45:19 +0000
Subject: [New-bugs-announce] [issue23947] Add mechanism to import stdlib
	package bypassing user	packages
Message-ID: <1429022719.25.0.0685276960077.issue23947@psf.upfronthosting.co.za>


New submission from Steve Dower:

There are some situations where the stdlib imports modules that could be absent, notably "import readline" in site.py. This import is expected to fail in some situations, but because sys.path is fully configured it can be importing arbitrary code.

To limit these imports to only installed packages, we could add a fake _stdlib module with __path__ set to a restricted set (approximately/exactly(?) what -I uses) and an importlib helper to import it and alias it in sys.modules.

Open question about what to do when a user has already imported their own module and it isn't the stdlib one. We discussed displaying a warning in this case.

If the import helper is private we should be able to backport to 2.7/3.4 easily enough.

----------
components: Library (Lib)
messages: 240896
nosy: brett.cannon, christian.heimes, eric.snow, ncoghlan, steve.dower
priority: normal
severity: normal
status: open
title: Add mechanism to import stdlib package bypassing user packages
versions: Python 2.7, Python 3.4, Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23947>
_______________________________________

From report at bugs.python.org  Tue Apr 14 17:24:21 2015
From: report at bugs.python.org (John Ehresman)
Date: Tue, 14 Apr 2015 15:24:21 +0000
Subject: [New-bugs-announce] [issue23948] Deprecate os.kill() on Windows
Message-ID: <1429025061.32.0.415018485708.issue23948@psf.upfronthosting.co.za>


New submission from John Ehresman:

os.kill() on Windows cannot act like it does on non-windows platforms because of differences in the underlying platforms.  I think only kill() with a signal number of 9 (terminate process unconditionally) and a signal number of 0 (test to see if process exists) could be implemented.  It isn't possibly to send the INT signal or a HUP signal to another process -- or at least it isn't possible without the use of questionable api's and code.

Windows specific functions would be added for terminateprocess() and generateconsolectrlevent().  os.kill() on non-windows platforms would be left alone.

----------
components: Windows
messages: 240905
nosy: jpe, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Deprecate os.kill() on Windows
type: behavior
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23948>
_______________________________________

From report at bugs.python.org  Tue Apr 14 18:28:57 2015
From: report at bugs.python.org (Ulaga Nathan Mahadevan)
Date: Tue, 14 Apr 2015 16:28:57 +0000
Subject: [New-bugs-announce] [issue23949] Number of elements display in
	error message is wrong	while unpacking in traceback
Message-ID: <1429028937.85.0.30895290076.issue23949@psf.upfronthosting.co.za>


New submission from Ulaga Nathan Mahadevan:

data = ['David',50,91.1,(2012,12,21)]
print ("Data = ",data)
name,shares,price,date,value = data

print("Name = ",name, ", no of shares = ",shares,", unit price = ",price,", date of purchase = ",date)

After running the script
Data =  ['David', 50, 91.1, (2012, 12, 21)]
    name,shares,price,date,value = data
ValueError: need more than 4 values to unpack

There are only 4 values to unback and "not more than 4". The given number of elements are 4 and so required number is 4. The error message is not clear or confusing.

----------
messages: 240935
nosy: ulaganathanm123 at gmail.com
priority: normal
severity: normal
status: open
title: Number of elements display in error message is wrong while unpacking in traceback
type: enhancement
versions: Python 3.2

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23949>
_______________________________________

From report at bugs.python.org  Tue Apr 14 18:48:15 2015
From: report at bugs.python.org (Rob King)
Date: Tue, 14 Apr 2015 16:48:15 +0000
Subject: [New-bugs-announce] [issue23950] Odd behavior with "file" and
	"filename" attributes in	cgi.FieldStorage
Message-ID: <1429030095.23.0.220534977623.issue23950@psf.upfronthosting.co.za>


New submission from Rob King:

Hello, everyone.

I've noticed an issue that could be just a documentation inaccuracy or a genuine, minor bug in cgi.FieldStorage.

The documentation for the module states:

"You can test for an uploaded file by testing either the filename attribute or the file attribute."

However, I've noticed that on a form submission that includes both a file field and a normal form field, *all* of the members of the form have a non-None "file" attribute, while only the actual uploaded file has a non-None "filename" attribute.

This would appear to be caused by the cgi module's use of MiniFieldStorage for in some situations, and FieldStorage in others. The FieldStorage class (if I'm reading the code correctly) would appear to always have a non-None "file" attribute but may have a None "filename" attribute.

This makes the documentation for the module unclear, as testing for a non-None "file" attribute will always return True in cases of FieldStorage, even if the particular field is not actually a file.

I would suggest modifying the documentation to say something like:

"You can test for an uploaded file by testing the filename attribute."

Please let me know if I'm missing something, that's always also a possibility. :)

----------
messages: 240942
nosy: deadpixi
priority: normal
severity: normal
status: open
title: Odd behavior with "file" and "filename" attributes in cgi.FieldStorage
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23950>
_______________________________________

From report at bugs.python.org  Tue Apr 14 18:48:45 2015
From: report at bugs.python.org (Carol Willing)
Date: Tue, 14 Apr 2015 16:48:45 +0000
Subject: [New-bugs-announce] [issue23951] Update devguide style to use a
	similar theme as Docs
Message-ID: <1429030125.69.0.476464501234.issue23951@psf.upfronthosting.co.za>


New submission from Carol Willing:

It would be helpful to update the conf.py file and dependencies to use a modified 'pydoctheme' for the devguide style. The benefits would be improved readability, a consistent look with the stdlib docs, and collapsible sidebars.

Attached is a screenshot of a quick hack of the conf.py file and the rendered devguide index.

----------
components: Devguide
files: Screen Shot 2015-04-14 at 12.42.48 PM.png
messages: 240943
nosy: ezio.melotti, willingc
priority: normal
severity: normal
status: open
title: Update devguide style to use a similar theme as Docs
type: enhancement
Added file: http://bugs.python.org/file38995/Screen Shot 2015-04-14 at 12.42.48 PM.png

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23951>
_______________________________________

From report at bugs.python.org  Tue Apr 14 19:58:05 2015
From: report at bugs.python.org (Rob King)
Date: Tue, 14 Apr 2015 17:58:05 +0000
Subject: [New-bugs-announce] [issue23952] Document the 'maxlen' member of
	the cgi module
Message-ID: <1429034285.17.0.254053228622.issue23952@psf.upfronthosting.co.za>


New submission from Rob King:

The cgi module has a global variable, 'maxlen', that specifies the maximum length of a POST request. By default, this limit is 0, meaning an unlimited POST request size.

Having an unlimited default opens up CGI scripts to resource-exhaustion attacks. Setting the maxlen variable to a nonzero integer solves this problem, but this fix is not in the official documentation - neither the reference manual nor the module's docstring.

I would recommend augmenting the module's docstring with the following statement:

"The maxlen variable can be set to an integer indicating the maximum size of a POST request. POST requests larger than this size will result in a ValueError being raised during parsing. The default value of this variable is 0, meaning the request size is unlimited."

----------
messages: 240965
nosy: deadpixi
priority: normal
severity: normal
status: open
title: Document the 'maxlen' member of the cgi module
versions: Python 3.4, Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23952>
_______________________________________

From report at bugs.python.org  Tue Apr 14 20:17:35 2015
From: report at bugs.python.org (Larry Hastings)
Date: Tue, 14 Apr 2015 18:17:35 +0000
Subject: [New-bugs-announce] [issue23953] test_mmap uses cruel and unusual
	amounts of disk space
Message-ID: <1429035455.58.0.977803706841.issue23953@psf.upfronthosting.co.za>


New submission from Larry Hastings:

My laptop is running 64-bit Linux (14.10).  It has 4.6GB of free disk space.  Naturally that's not enough to run test_mmap.

When I run the test suite, test_mmap consumes all available disk space, then fails.  (Hopefully freeing all its temporary files!)  If I used "-j" to run more than one test at a time, this usually means *other* tests fail too, because I'm running multiple tests in parallel and there are plenty of other tests that require, y'know... any disk space whatsoever.

The documentation for the test suite ("./python -m test -h") says that "-u largefile" allows tests that use more than 2GB.  Surely test_mmap's delicious 800PB tests should be marked largefile-enabled-only?

I'd like to see this fix backported to 3.4 too.  And if 2.7 shows this behavior, maybe there too.

----------
components: Tests
messages: 240974
nosy: larry
priority: normal
severity: normal
stage: needs patch
status: open
title: test_mmap uses cruel and unusual amounts of disk space
type: resource usage
versions: Python 3.4, Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23953>
_______________________________________

From report at bugs.python.org  Tue Apr 14 20:52:26 2015
From: report at bugs.python.org (Al Sweigart)
Date: Tue, 14 Apr 2015 18:52:26 +0000
Subject: [New-bugs-announce] [issue23954] Pressing enter/return or clicking
	IDLE's autocomplete	does not select the option
Message-ID: <1429037546.84.0.736850932542.issue23954@psf.upfronthosting.co.za>


New submission from Al Sweigart:

Currently, when you press Ctrl+Space or select Edit > Show Completions, a dropdown menu appears and lets you select an autocomplete option.

If you press Tab, the highlighted option in the dropdown is selected and your typing autocompletes. However, pressing enter (which is a natural way to select an option) does not work. Instead of newline is entered, which is an annoyance.

Also, clicking on the item with the mouse does not select an item from the dropdown either.

----------
components: IDLE
messages: 240986
nosy: Al.Sweigart
priority: normal
severity: normal
status: open
title: Pressing enter/return or clicking IDLE's autocomplete does not select the option
type: behavior
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23954>
_______________________________________

From report at bugs.python.org  Tue Apr 14 21:16:30 2015
From: report at bugs.python.org (Steve Dower)
Date: Tue, 14 Apr 2015 19:16:30 +0000
Subject: [New-bugs-announce] [issue23955] Add python.ini file for
	embedded/applocal installs
Message-ID: <1429038990.96.0.956334607163.issue23955@psf.upfronthosting.co.za>


New submission from Steve Dower:

When Python starts running we (getpathp.c) looks in the registry for its library directory. To avoid this, you can set PYTHONPATH and PYTHONHOME environment variables, but this isn't always easy if you've copied Python into your app's directory.

We should look for a python35.ini alongside python35.dll when initializing and extract values from it. Possibly this could just be an [Environment] section with key-value pairs to set before loading.

This would ensure that app-local Python installations never look for a global install in the registry. (As an aside, for the 3.5 beta we'll be shipping an app-local "installer", which will make this situation easier to end up in, but it also has value for xcopy installs.)

----------
assignee: steve.dower
components: Interpreter Core, Windows
messages: 240989
nosy: ncoghlan, richard, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Add python.ini file for embedded/applocal installs
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23955>
_______________________________________

From report at bugs.python.org  Tue Apr 14 22:05:58 2015
From: report at bugs.python.org (Jacinda Shelly)
Date: Tue, 14 Apr 2015 20:05:58 +0000
Subject: [New-bugs-announce] [issue23956] Compatibility misspelled in
	Lib/imp.py
Message-ID: <1429041958.63.0.857805541545.issue23956@psf.upfronthosting.co.za>


New submission from Jacinda Shelly:

Minor misspelling in Lib/imp.py. Doing this as a training exercise for my first patch.

----------
assignee: docs at python
components: Documentation
files: misspelling.patch
keywords: patch
messages: 241005
nosy: docs at python, jacinda
priority: normal
severity: normal
status: open
title: Compatibility misspelled in Lib/imp.py
type: enhancement
versions: Python 3.5
Added file: http://bugs.python.org/file39014/misspelling.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23956>
_______________________________________

From report at bugs.python.org  Tue Apr 14 22:06:13 2015
From: report at bugs.python.org (Andrew Pinkham)
Date: Tue, 14 Apr 2015 20:06:13 +0000
Subject: [New-bugs-announce] [issue23957] Additional misspelled in
	documentation
Message-ID: <1429041973.13.0.203426260536.issue23957@psf.upfronthosting.co.za>


New submission from Andrew Pinkham:

There is a minor typo under PyMem_SetupDebugHooks in the C-API section in 3.4 and 3.5 . "Additional" is misspelled as "Additionnal" on the page.

https://docs.python.org/3.5/c-api/memory.html
https://docs.python.org/3.4/c-api/memory.html

The problem does not appear to occur earlier.

----------
assignee: docs at python
components: Documentation
files: additionnal_typo_fix.patch
keywords: patch
messages: 241006
nosy: docs at python, jambonrose
priority: normal
severity: normal
status: open
title: Additional misspelled in documentation
type: enhancement
versions: Python 3.4, Python 3.5
Added file: http://bugs.python.org/file39015/additionnal_typo_fix.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23957>
_______________________________________

From report at bugs.python.org  Tue Apr 14 22:24:08 2015
From: report at bugs.python.org (Steve R. Hastings)
Date: Tue, 14 Apr 2015 20:24:08 +0000
Subject: [New-bugs-announce] [issue23958] compile warnings in libffi
Message-ID: <1429043048.03.0.90246390985.issue23958@psf.upfronthosting.co.za>


New submission from Steve R. Hastings:

The normal build should not have any warnings.  The source for libffi included some comparisons between signed int and unsigned int, which caused warnings.

The patch changes the signed int variables to unsigned int.

----------
components: Build
files: libffi_warning_patch.txt
messages: 241011
nosy: steveha
priority: normal
severity: normal
status: open
title: compile warnings in libffi
type: behavior
versions: Python 3.5
Added file: http://bugs.python.org/file39017/libffi_warning_patch.txt

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23958>
_______________________________________

From report at bugs.python.org  Tue Apr 14 22:47:44 2015
From: report at bugs.python.org (Maciej Szulik)
Date: Tue, 14 Apr 2015 20:47:44 +0000
Subject: [New-bugs-announce] [issue23959] Update imaplib to support RFC3501
Message-ID: <1429044464.49.0.731812923809.issue23959@psf.upfronthosting.co.za>


New submission from Maciej Szulik:

Current imaplib implementation follows obsoleted RFC2060. It should be updated to follow RFC3501.

----------
components: email
messages: 241022
nosy: barry, maciej.szulik, r.david.murray
priority: normal
severity: normal
status: open
title: Update imaplib to support RFC3501
type: enhancement

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23959>
_______________________________________

From report at bugs.python.org  Wed Apr 15 01:12:48 2015
From: report at bugs.python.org (Ofer Schwarz)
Date: Tue, 14 Apr 2015 23:12:48 +0000
Subject: [New-bugs-announce] [issue23960] PyErr_SetImportError doesn't clean
	up on some errors
Message-ID: <1429053168.87.0.990858167954.issue23960@psf.upfronthosting.co.za>


New submission from Ofer Schwarz:

When creating kwargs to construct the ImportError, if PyDict_SetItemString fails the function returns without decref'ing the already-created locals.

----------
components: Interpreter Core
files: importerrorcleanup.patch
keywords: patch
messages: 241052
nosy: blackfawn, gregory.p.smith
priority: normal
severity: normal
status: open
title: PyErr_SetImportError doesn't clean up on some errors
type: behavior
versions: Python 3.5
Added file: http://bugs.python.org/file39023/importerrorcleanup.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23960>
_______________________________________

From report at bugs.python.org  Wed Apr 15 03:22:15 2015
From: report at bugs.python.org (Al Sweigart)
Date: Wed, 15 Apr 2015 01:22:15 +0000
Subject: [New-bugs-announce] [issue23961] IDLE autocomplete window does not
	automatically close	when selection is made
Message-ID: <1429060935.51.0.684469355013.issue23961@psf.upfronthosting.co.za>


New submission from Al Sweigart:

All autocomplete features (for example, Eclipse & Android Studio) cause the autocomplete window to disappear when a selection is made. When tab is pressed in IDLE, the window does not disappear.

IDLE should be changed so that the window goes away when tab is pressed to complete the window after Ctrl+Space or Edit > Show Completions openedit. Currently, the window stays open until another keyboard key is pressed (e.g. space, or an opening parenthesis)

----------
components: IDLE
messages: 241068
nosy: Al.Sweigart
priority: normal
severity: normal
status: open
title: IDLE autocomplete window does not automatically close when selection is made
type: behavior
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23961>
_______________________________________

From report at bugs.python.org  Wed Apr 15 05:33:52 2015
From: report at bugs.python.org (Ryder Lewis)
Date: Wed, 15 Apr 2015 03:33:52 +0000
Subject: [New-bugs-announce] [issue23962] Incorrect TimeoutError referenced
	in concurrent.futures	documentation
Message-ID: <1429068832.51.0.845639899608.issue23962@psf.upfronthosting.co.za>


New submission from Ryder Lewis:

The documentation at https://docs.python.org/3/library/concurrent.futures.html has several functions that case raise a TimeoutError. The hyperlink generated for TimeoutError links to the built-in exception https://docs.python.org/3/library/exceptions.html#TimeoutError. However, the exception raised is a concurrent.futures._base.TimeoutError exception instead, which is undocumented.

----------
assignee: docs at python
components: Documentation
messages: 241074
nosy: docs at python, ryder.lewis
priority: normal
severity: normal
status: open
title: Incorrect TimeoutError referenced in concurrent.futures documentation
versions: Python 3.4, Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23962>
_______________________________________

From report at bugs.python.org  Wed Apr 15 14:20:30 2015
From: report at bugs.python.org (Anselm Kruis)
Date: Wed, 15 Apr 2015 12:20:30 +0000
Subject: [New-bugs-announce] [issue23963] Windows build error using original
	openssl source
Message-ID: <1429100430.86.0.568856715726.issue23963@psf.upfronthosting.co.za>


New submission from Anselm Kruis:

I just had an issue rebuilding 2.7.9 with openssl-1.0.1m. Because this openssl version is not yet available from svn.python.org, I used the original source from https://www.openssl.org/source/openssl-1.0.1m.tar.gz.
Of course Perl is installed.

Problem

The build fails, because the created makefile ms/nt.mak contains rules for excluded ciphers ("RC5", "MDC2", "IDEA"). The documentation in PCbuild/readme.txt anticipates a similar compile error: "You may have to manually remove $(OBJ_D)\i_*.obj from ms\nt.mak if the build process complains about missing files or forbidden IDEA."

Obviously manual patching of ms\nt.mak is inconvenient. A clean solution: invoke the openssl Configure script with appropriate arguments:

perl Configure no-rc5 no-mdc2 no-idea ...

The attached patch changes PCbuild/build_ssl.py to add the no-xxx args  to the Configure command line.

Python 3.x already contains a similar fix (changeset bcf93e3766e8).

----------
components: Build
files: 2.7-build_ssl.patch
keywords: patch
messages: 241096
nosy: anselm.kruis, zach.ware
priority: normal
severity: normal
status: open
title: Windows build error using original openssl source
type: compile error
versions: Python 2.7
Added file: http://bugs.python.org/file39034/2.7-build_ssl.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23963>
_______________________________________

From report at bugs.python.org  Wed Apr 15 15:32:33 2015
From: report at bugs.python.org (Al Sweigart)
Date: Wed, 15 Apr 2015 13:32:33 +0000
Subject: [New-bugs-announce] [issue23964] Update README documentation for
	IDLE tests.
Message-ID: <1429104753.6.0.763987492268.issue23964@psf.upfronthosting.co.za>


New submission from Al Sweigart:

It took me a while to read through the idle_test documentation to figure out how to run the test. Similar to the Profiler documentation, I've added "Instant User's Manual" section at the top of the file, as well as edited the text and formatting a bit.

----------
components: IDLE
files: idle_test_readme.patch
keywords: patch
messages: 241099
nosy: Al.Sweigart
priority: normal
severity: normal
status: open
title: Update README documentation for IDLE tests.
versions: Python 3.5
Added file: http://bugs.python.org/file39035/idle_test_readme.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23964>
_______________________________________

From report at bugs.python.org  Wed Apr 15 16:44:02 2015
From: report at bugs.python.org (Kushal Das)
Date: Wed, 15 Apr 2015 14:44:02 +0000
Subject: [New-bugs-announce] [issue23965] test_ssl failure on Fedora 22
Message-ID: <1429109042.15.0.0394513301498.issue23965@psf.upfronthosting.co.za>


New submission from Kushal Das:

OpenSSL version openssl-1.0.1k-6.fc22.x86_64
>>> ssl.OPENSSL_VERSION
'OpenSSL 1.0.1k-fips 8 Jan 2015'


======================================================================
ERROR: test_protocol_sslv23 (test.test_ssl.ThreadedTests)
Connecting to an SSLv23 server with various client options
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/kdas/code/python/cpython/Lib/test/test_ssl.py", line 119, in f
    return func(*args, **kwargs)
  File "/home/kdas/code/python/cpython/Lib/test/test_ssl.py", line 2454, in test_protocol_sslv23
    try_protocol_combo(ssl.PROTOCOL_SSLv23, ssl.PROTOCOL_SSLv3, 'SSLv3')
  File "/home/kdas/code/python/cpython/Lib/test/test_ssl.py", line 2203, in try_protocol_combo
    chatty=False, connectionchatty=False)
  File "/home/kdas/code/python/cpython/Lib/test/test_ssl.py", line 2130, in server_params_test
    s.connect((HOST, server.port))
  File "/home/kdas/code/python/cpython/Lib/ssl.py", line 1014, in connect
    self._real_connect(addr, False)
  File "/home/kdas/code/python/cpython/Lib/ssl.py", line 1005, in _real_connect
    self.do_handshake()
  File "/home/kdas/code/python/cpython/Lib/ssl.py", line 983, in do_handshake
    self._sslobj.do_handshake()
  File "/home/kdas/code/python/cpython/Lib/ssl.py", line 628, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:634)

======================================================================
ERROR: test_protocol_sslv3 (test.test_ssl.ThreadedTests)
Connecting to an SSLv3 server with various client options
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/kdas/code/python/cpython/Lib/test/test_ssl.py", line 119, in f
    return func(*args, **kwargs)
  File "/home/kdas/code/python/cpython/Lib/test/test_ssl.py", line 2497, in test_protocol_sslv3
    client_options=ssl.OP_NO_SSLv2)
  File "/home/kdas/code/python/cpython/Lib/test/test_ssl.py", line 2203, in try_protocol_combo
    chatty=False, connectionchatty=False)
  File "/home/kdas/code/python/cpython/Lib/test/test_ssl.py", line 2130, in server_params_test
    s.connect((HOST, server.port))
  File "/home/kdas/code/python/cpython/Lib/ssl.py", line 1014, in connect
    self._real_connect(addr, False)
  File "/home/kdas/code/python/cpython/Lib/ssl.py", line 1005, in _real_connect
    self.do_handshake()
  File "/home/kdas/code/python/cpython/Lib/ssl.py", line 983, in do_handshake
    self._sslobj.do_handshake()
  File "/home/kdas/code/python/cpython/Lib/ssl.py", line 628, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: UNSUPPORTED_PROTOCOL] unsupported protocol (_ssl.c:634)

======================================================================
FAIL: test_options (test.test_ssl.ContextTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/kdas/code/python/cpython/Lib/test/test_ssl.py", line 119, in f
    return func(*args, **kwargs)
  File "/home/kdas/code/python/cpython/Lib/test/test_ssl.py", line 789, in test_options
    ctx.options)
AssertionError: 2164261887 != 2197816319

----------------------------------------------------------------------

----------
components: Library (Lib)
messages: 241109
nosy: christian.heimes, kushal.das
priority: normal
severity: normal
status: open
title: test_ssl failure on Fedora 22
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23965>
_______________________________________

From report at bugs.python.org  Wed Apr 15 18:36:51 2015
From: report at bugs.python.org (Nick Coghlan)
Date: Wed, 15 Apr 2015 16:36:51 +0000
Subject: [New-bugs-announce] [issue23966] More clearly expose/explain native
	and cross-build	target information
Message-ID: <1429115811.5.0.862888158961.issue23966@psf.upfronthosting.co.za>


New submission from Nick Coghlan:

Discussion of issue 22980 made it clear to me that part of the problem with the cross-build support is that the difference between the build time information exposed in sysconfig and the live system information exposed in the platform module isn't clear.

Debian's multiarch triplets (https://wiki.debian.org/Multiarch/Tuples) describe this information well, but it currently isn't readily exposed to Python code. sysconfig.get_config_var("MULTIARCH") only works on Debian & derivatives, and even issue 22980 only exposes the information in sysconfig.get_config_var("SOABI") on a subset of platforms.

It would be good to extend the triples to cover Windows, and also to expose the *current* system triple at runtime in the platform module (including appropriate normalisation)

----------
messages: 241129
nosy: barry, bkabrda, doko, ncoghlan, rkuska
priority: normal
severity: normal
status: open
title: More clearly expose/explain native and cross-build target information
type: enhancement
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23966>
_______________________________________

From report at bugs.python.org  Wed Apr 15 20:52:37 2015
From: report at bugs.python.org (Larry Hastings)
Date: Wed, 15 Apr 2015 18:52:37 +0000
Subject: [New-bugs-announce] [issue23967] Make inspect.signature expression
	evaluation more	powerful
Message-ID: <1429123957.6.0.780975142449.issue23967@psf.upfronthosting.co.za>


New submission from Larry Hastings:

Peter's working on converting socket to use Argument Clinic.  He had a default that really should look like this:

    min(SOME_SOCKET_MODULE_CONSTANT, 128)

"min" wasn't something we'd needed before.  I thought about it and realized we could do a much better job of simulating the evaluation context of a shared module.

Initially I thought, all I needed was to bolster the environment we used for eval() to add the builtins.  (Which I've done.)  But this wasn't sufficient because we deliberately used ast.literal_eval(), which doesn't support function calls by design for superior security.  Or subscripting, or attribute access.  We already worked around those I think.

But how concerned are we about security?  What is the attack vector here?  If the user is able to construct an object that has a villainous __text_signature__ on it... surely they could already do as they like?

So here's a first draft at modifying the __text_signature__ evaluation environment so it can handle much more sophisticated expressions.  It can use anything from builtins, or anything in sys.modules, or anything in the current module; it can call functions, and subscript, and access attributes, and everything.

To make this work I had to write an ast printer that produces evaluatable Python code.  Note that it's not complete, I know it's not complete, it's missing loads of operators.  Assume that if this is a good idea I will add all the missing operators.

Nick was worried that *in the future* we might expose a "turn this string into a signature" function.  That might make an easier attack vector.  So he asked that the "trusted=" keyword flag be added, and the full-on eval only happen if the string is trusted.

----------
assignee: larry
components: Library (Lib)
files: larry.improved.signature.expressions.1.txt
messages: 241140
nosy: larry, ncoghlan, pdmccormick, serhiy.storchaka, yselivanov, zach.ware
priority: normal
severity: normal
stage: patch review
status: open
title: Make inspect.signature expression evaluation more powerful
type: enhancement
versions: Python 3.5
Added file: http://bugs.python.org/file39047/larry.improved.signature.expressions.1.txt

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23967>
_______________________________________

From report at bugs.python.org  Wed Apr 15 21:04:07 2015
From: report at bugs.python.org (Matthias Klose)
Date: Wed, 15 Apr 2015 19:04:07 +0000
Subject: [New-bugs-announce] [issue23968] rename the platform directory from
	plat-$(MACHDEP) to	plat-$(PLATFORM_TRIPLET)
Message-ID: <1429124647.47.0.946423694262.issue23968@psf.upfronthosting.co.za>


New submission from Matthias Klose:

Currently there is only one platform directory for all linux architectures, there are two reasons to have a separate directory for each architecture:

 - the contents of the files in this directory differ
   (most prominent case are the RTLD_* constants on various
   architectures).

 - The _sysconfigdata.py module is installed in the platdir.
   A cross build of the extensions in the standard library
   is done by running the python interpreter for the build
   machine with the platdir for the host machine.  This
   currently works for cross builds targeting a different
   os, but not cross building from one linux architecture
   to another.

This would then remove Lib/plat-linux and introduce some Lib/plat-$(PLATFORM_TRIPLET) directories.

----------
components: Installation
files: platdir.diff
keywords: patch
messages: 241143
nosy: doko, ncoghlan
priority: normal
severity: normal
status: open
title: rename the platform directory from plat-$(MACHDEP) to plat-$(PLATFORM_TRIPLET)
versions: Python 3.5
Added file: http://bugs.python.org/file39049/platdir.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23968>
_______________________________________

From report at bugs.python.org  Wed Apr 15 21:12:13 2015
From: report at bugs.python.org (Matthias Klose)
Date: Wed, 15 Apr 2015 19:12:13 +0000
Subject: [New-bugs-announce] [issue23969] please set a SOABI for MacOSX
Message-ID: <1429125133.46.0.136772675069.issue23969@psf.upfronthosting.co.za>


New submission from Matthias Klose:

split out from

http://bugs.python.org/issue22980#msg232065

please consider setting the SOABI for MacOSX. Afaics MacOSX is now the only major platform not setting these, at least including the SOABI. The idea is that it would be possible to ship "fat" wheels, containing extensions for every OS.  Not sure what the PLATFORM_TRIPLET would be, maybe just something like macosx, or darwin, if the MacOSX extension itself is a fat build.

----------
messages: 241146
nosy: doko, ned.deily, ronaldoussoren
priority: normal
severity: normal
status: open
title: please set a SOABI for MacOSX

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23969>
_______________________________________

From report at bugs.python.org  Thu Apr 16 00:22:53 2015
From: report at bugs.python.org (Steve Dower)
Date: Wed, 15 Apr 2015 22:22:53 +0000
Subject: [New-bugs-announce] [issue23970] Update distutils.msvccompiler for
	VC14
Message-ID: <1429136573.43.0.772566564267.issue23970@psf.upfronthosting.co.za>


New submission from Steve Dower:

Because of the compiler change, we need to rework the detection of MSVC for Python 3.5. I took the opportunity to clean up the entire module and remove msvc9compiler, and updated the tests.

----------
assignee: steve.dower
components: Windows
files: msvccompiler_1.patch
keywords: patch
messages: 241173
nosy: steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
stage: patch review
status: open
title: Update distutils.msvccompiler for VC14
type: behavior
versions: Python 3.5
Added file: http://bugs.python.org/file39058/msvccompiler_1.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23970>
_______________________________________

From report at bugs.python.org  Thu Apr 16 01:35:44 2015
From: report at bugs.python.org (Larry Hastings)
Date: Wed, 15 Apr 2015 23:35:44 +0000
Subject: [New-bugs-announce] [issue23971] dict(list) and dict.fromkeys()
	doesn't account for 2/3	fill ratio
Message-ID: <1429140944.27.0.450320519696.issue23971@psf.upfronthosting.co.za>


New submission from Larry Hastings:

Matt Mackall brought this over to me today.

If list "l" has one million pairs and you do
  dict(l)
it creates the dict, then resizes it to a million elements, then starts adding name/value pairs.  But since it's sized to a million elements, it gets beyond its "fill ratio" comfort level of 666k and resizes.  And when you're up to a million elements, the resize is quite slow.  It should have internally resized the dict correctly from the beginning such that no resize was necessary.

The same is true for dict.fromkeys--passed in a list of 1m elements, it uses an initial size of 1m, not 1.5m.

Attached is a sample patch to change the behavior of both these operations so no resizes are needed during the insertions.  I haven't checked that it actually fixes the behavior, I just made the change and ran the regression test.  (I'm oversubscribed here at the sprints, and am kind of rushing this diff out just to get the conversation rolling.)

Benjamin: what do you think?  Would it be appropriate to make a change like this in 2.7?

Python 3 (or at least 3.5 trunk) is fancy here.  It starts with the minimum size of a dict, then iterates until the new size is > 1.5*the  number of elements.  This means the dicts it creates are of the same size as they would be if we'd started with a minsize dict and inserted one element at a time.  This might help minimize wear and tear on the small block allocator for smaller dicts.

BTW, the internal function _PyDict_NewPresize should really have taken the fill ratio into account too.  But, even though it's an internal-only function, it's probably too late to change its behavior.

----------
components: Interpreter Core
messages: 241179
nosy: benjamin.peterson, larry, marmoute, mpm
priority: normal
severity: normal
stage: patch review
status: open
title: dict(list) and dict.fromkeys() doesn't account for 2/3 fill ratio
type: behavior
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23971>
_______________________________________

From report at bugs.python.org  Thu Apr 16 13:52:15 2015
From: report at bugs.python.org (Boris FELD)
Date: Thu, 16 Apr 2015 11:52:15 +0000
Subject: [New-bugs-announce] [issue23972] Asyncio reuseport
Message-ID: <1429185135.77.0.485030290106.issue23972@psf.upfronthosting.co.za>


New submission from Boris FELD:

I'm trying to create some UDP sockets for doing multicast communication. I have the working code in synchronous way and try to port it to asyncio.

One last issue is blocking for me, I'm on Mac OS X and for multicast UDP to work, the SO_REUSEPORT must be set on socket before bind. The problem is that I don't have access on socket, it's created inside asyncio method _create_connection_transport.

I've seen that SO_REUSEADDR is already set (https://github.com/gvanrossum/tulip-try3/blob/7b2d8abfce1d7ef18ef516f9b1b7032172630375/asyncio/base_events.py#L720), so maybe we could also set SO_REUSEPORT only on platforms where it's available. `if hasattr(socket, 'SO_REUSEPORT')` should works.

Or we could add an optional arguments with could be used to set some socket options, it could be more flexible that set SO_REUSEPORT.

I could provide a patch for the best solution selected.

----------
components: asyncio
messages: 241213
nosy: Boris.FELD, gvanrossum, haypo, yselivanov
priority: normal
severity: normal
status: open
title: Asyncio reuseport
type: behavior
versions: Python 3.4, Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23972>
_______________________________________

From report at bugs.python.org  Thu Apr 16 20:32:03 2015
From: report at bugs.python.org (Guido van Rossum)
Date: Thu, 16 Apr 2015 18:32:03 +0000
Subject: [New-bugs-announce] [issue23973] PEP 484 implementation
Message-ID: <1429209123.93.0.369207893318.issue23973@psf.upfronthosting.co.za>


New submission from Guido van Rossum:

I'm creating a release blocker placeholder issue for PEP 484. I think we're going to miss alpha 4 but I am expecting to make beta 1 (and Mark Shannon, the BDFL-Delegate has promised to work within this schedule). Of course it's possible that Mark ends up rejecting the PEP, but I'm betting that we'll be able to hammer out a compromise, so I'm creating this bug to remind myself and other devs of this deadline.

Also, the only thing to be included is typing.py (and its tests and hopefully some docs, even if the first rev of the docs just points to the PEP).

----------
assignee: gvanrossum
messages: 241256
nosy: gvanrossum
priority: release blocker
severity: normal
status: open
title: PEP 484 implementation
type: enhancement
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23973>
_______________________________________

From report at bugs.python.org  Thu Apr 16 21:09:59 2015
From: report at bugs.python.org (Christopher Gurnee)
Date: Thu, 16 Apr 2015 19:09:59 +0000
Subject: [New-bugs-announce] [issue23974] random.randrange() biased output
Message-ID: <1429211399.3.0.0363933708398.issue23974@psf.upfronthosting.co.za>


New submission from Christopher Gurnee:

Due to an optimization in random.randrange() only in Python 2, as the stop-start range approaches 2^53 the output becomes noticeably biased. This bug also affects random.SystemRandom.

For example, counting the number of even ints in a set of 10^6 random ints each in the range [0, 2^52 * 2/3) produces normal results; about half are even:

>>> sum(randrange(2**52 * 2//3) % 2 for i in xrange(1000000)) / 1000000.0
0.499932

Change the range to [0, 2^53 * 2/3), and you get a degenerate case where evens are two times more likely to occur than odds:

>>> sum(randrange(2**53 * 2//3) % 2 for i in xrange(1000000)) / 1000000.0
0.333339

The issue occurs in three places inside randrange(), here's one:

if istart >= _maxwidth:
    return self._randbelow(istart)
return _int(self.random() * istart)

_maxwidth is the max size of a double where every digit to the left of the decimal point can still be represented w/o loss of precision (2^53, where a double has 53 mantissa bits). With istart >= _maxwidth, _randbelow() behaves correctly. With istart < _maxwidth, the rounding error in random() * istart begins to cause problems as istart approaches 2^53.

Changing _maxwidth to be significantly less should (practically speaking anyways) fix this, although I'm open to suggestion on just how conservatively it should be set.

----------
components: Library (Lib)
messages: 241261
nosy: gurnec
priority: normal
severity: normal
status: open
title: random.randrange() biased output
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23974>
_______________________________________

From report at bugs.python.org  Thu Apr 16 22:06:57 2015
From: report at bugs.python.org (Wolfgang Maier)
Date: Thu, 16 Apr 2015 20:06:57 +0000
Subject: [New-bugs-announce] [issue23975] numbers.Rational implements
	__float__ incorrectly
Message-ID: <1429214817.83.0.816258978912.issue23975@psf.upfronthosting.co.za>


New submission from Wolfgang Maier:

numbers.Rational defines __float__ like this:

def __float__(self):
    """float(self) = self.numerator / self.denominator

    It's important that this conversion use the integer's "true"
    division rather than casting one side to float before dividing
    so that ratios of huge integers convert without overflowing.

    """
    return self.numerator / self.denominator


This assumes that division of two Integral numbers returns a float, which the numbers ABC does not enforce in any way.
IMO, the only logical assumption is that division of any two Integral or Rational numbers gives a Real, for which the ABC guarantees a __float__ method in turn.
So I think Rational.__float__ should

return float(self.numerator / self.denominator)

----------
components: Library (Lib)
messages: 241270
nosy: wolma
priority: normal
severity: normal
status: open
title: numbers.Rational implements __float__ incorrectly
type: behavior
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23975>
_______________________________________

From report at bugs.python.org  Thu Apr 16 22:48:40 2015
From: report at bugs.python.org (D. Alphus)
Date: Thu, 16 Apr 2015 20:48:40 +0000
Subject: [New-bugs-announce] [issue23976] ZipFile.writestr implies
	non-regular files
Message-ID: <1429217320.26.0.802976363026.issue23976@psf.upfronthosting.co.za>


New submission from D. Alphus:

When ZipFile.writestr is called with arcname as its first parameter, it creates a ZipInfo object that has 0o0600 << 16 as its external_attr. Files created with write generally have 0o0100600 (S_IFREG is set indicating a regular file). Some archive managers (like Ark) will not populate the extracted file unless it contains this flag.

----------
components: Library (Lib)
messages: 241274
nosy: dalphus
priority: normal
severity: normal
status: open
title: ZipFile.writestr implies non-regular files
type: behavior
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23976>
_______________________________________

From report at bugs.python.org  Fri Apr 17 00:18:37 2015
From: report at bugs.python.org (Al Sweigart)
Date: Thu, 16 Apr 2015 22:18:37 +0000
Subject: [New-bugs-announce] [issue23977] Enhancing IDLE's test_delegator.py
	unit test
Message-ID: <1429222717.34.0.436460627497.issue23977@psf.upfronthosting.co.za>


New submission from Al Sweigart:

As part of trying to understand the IDLE codebase, I'm taking a look at IDLE's unit tests.

-Added more specific messages when test_delegator.py's assertions fail.
-Added detail to the comments.
-Added an explicit test of the resetcache() method (which before was only tested as far as it was called when setdelegate() was called).

I've rerun the unit test on the existing source code and it passes.

----------
components: Tests
files: test_delegator.patch
keywords: patch
messages: 241284
nosy: Al.Sweigart
priority: normal
severity: normal
status: open
title: Enhancing IDLE's test_delegator.py unit test
type: enhancement
versions: Python 3.5
Added file: http://bugs.python.org/file39078/test_delegator.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23977>
_______________________________________

From report at bugs.python.org  Fri Apr 17 00:49:38 2015
From: report at bugs.python.org (jmorgensen)
Date: Thu, 16 Apr 2015 22:49:38 +0000
Subject: [New-bugs-announce] [issue23978] ttk.Style.element_create using
	incorrect tk.call syntax
Message-ID: <1429224578.03.0.989768410298.issue23978@psf.upfronthosting.co.za>


New submission from jmorgensen:

Style.element_create passes all spec elements as a single string into tk.call rather than breaking up arguments into "words." However, it passes the options properly as a list.

This causes python to crash with a stacktrace like the one below.

> def element_create(self, elementname, etype, *args, **kw):
> """Create a new element in the current theme of given etype."""
>     spec, opts = _format_elemcreate(etype, False, *args, **kw)
>     self.tk.call(self._name, "element", "create", elementname, etype,
>         spec, *opts)

And in _format_elemcreate:
>     spec = "%s %s" % (iname, imagespec)
>     ...
>     spec = "%s %s %s" % (class_name, part_id, statemap)

Reproduction (on win):

> >>> import ttk
> >>> Style().element_create('custom.test', 'vsapi', 'SPIN', 2)
>   File "<stdin>", line 1, in <module>
>   File "C:\Miniconda\lib\lib-tk\ttk.py", line 466, in element_create
>     spec, *opts)
> _tkinter.TclError: missing required arguments 'class' and/or 'partId'

and, similarly:
> >>> import ttk
> >>> ttk.Style().element_create('custom.test', 'vsapi', 'SPIN', 2, width=5)
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
>   File "C:\Miniconda\lib\lib-tk\ttk.py", line 466, in element_create
>     spec, *opts)
> _tkinter.TclError: expected integer but got "-width"

----------
components: Tkinter
messages: 241286
nosy: jmorgensen
priority: normal
severity: normal
status: open
title: ttk.Style.element_create using incorrect tk.call syntax
type: crash
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23978>
_______________________________________

From report at bugs.python.org  Fri Apr 17 01:14:09 2015
From: report at bugs.python.org (Luis)
Date: Thu, 16 Apr 2015 23:14:09 +0000
Subject: [New-bugs-announce] [issue23979] Multiprocessing Pool.map pickles
	arguments passed to	workers
Message-ID: <1429226049.08.0.139422751174.issue23979@psf.upfronthosting.co.za>


New submission from Luis:

Hi,

I've seen an odd behavior for multiprocessing Pool in Linux/MacOS:

-----------------------------
import multiprocessing as mp
from sys import getsizeof
import numpy as np


def f_test(x):
    print('process has received argument %s' % x )
    r = x[:100] # return will put in a queue for Pool, for objects > 4GB pickle complains
    return r

if __name__ == '__main__':
    # 2**28 runs ok, 2**29 or bigger breaks pickle
    big_param = np.random.random(2**29)

    # Process+big_parameter OK:
    proc = mp.Process(target=f_test, args=(big_param,))
    res = proc.start()
    proc.join()
    print('size of process result', getsizeof(res))

    # Pool+big_parameter BREAKS:
    pool = mp.Pool(1)
    res = pool.map(f_test, (big_param,))
    print('size of Pool result', getsizeof(res))

-----------------------------
$ python bug_mp.py
process has received argument [ 0.65282086  0.34977429  0.64148342 ...,  0.79902495  0.31427761
  0.02678803]
size of process result 16
Traceback (most recent call last):
  File "bug_mp.py", line 26, in <module>
    res = pool.map(f_test, (big_param,))
  File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/multiprocessing/pool.py", line 260, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/multiprocessing/pool.py", line 599, in get
    raise self._value
  File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/multiprocessing/pool.py", line 383, in _handle_tasks
    put(task)
  File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/multiprocessing/connection.py", line 206, in send
    self._send_bytes(ForkingPickler.dumps(obj))
  File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/multiprocessing/reduction.py", line 50, in dumps
    cls(buf, protocol).dump(obj)
OverflowError: cannot serialize a bytes object larger than 4 GiB

-----------------------------
There's another flavor of error seen in similar scenario:
...
struct.error: 'i' format requires -2147483648 <= number <= 2147483647

-----------------------------
Tested in:
Python 3.4.2 |Anaconda 2.1.0 (64-bit)| (default, Oct 21 2014, 17:16:37)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux
And in:
Python 3.4.3 (default, Apr  9 2015, 16:03:56)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.51)] on darwin

-----------------------------

Pool.map creates a "task Queue" to handle workers, and I think that but by doing this we are forcing any arguments passed to the workers to be pickled.
Process works OK, since no queue is created, it just forks.

My expectation would be that since we are in POSIX and forking, we shouldn't have to worry about arguments being pickled, and if this is expected behavior, it should be warned/documented (hope I've not missed this in the docs).

For small sized arguments, pickling-unpicking may not be an issue, but for big ones then, it is (I am aware of the Array and MemShare options).

Anybody has seen something similar, is perhaps this a hard requirement to Pool.map or I'm completely missing the point altogether?

----------
messages: 241289
nosy: kieleth
priority: normal
severity: normal
status: open
title: Multiprocessing Pool.map pickles arguments passed to workers
type: behavior
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23979>
_______________________________________

From report at bugs.python.org  Fri Apr 17 01:38:46 2015
From: report at bugs.python.org (Larry Hastings)
Date: Thu, 16 Apr 2015 23:38:46 +0000
Subject: [New-bugs-announce] [issue23980] Documentation for format units
	starting with 'e' is	inconsistent
Message-ID: <1429227526.01.0.733275706487.issue23980@psf.upfronthosting.co.za>


New submission from Larry Hastings:

Documentation is here:

    https://docs.python.org/3/c-api/arg.html#arg-parsing


The first line of documentation for each format unit follows this convention:
    formatunit (pythontype) [arguments, to, pyarg_parsetuple]

These represent the format unit itself, followed by the Python type it consumes in parentheses, followed by the C types it outputs in square brackets.  Thus
    i (int) [int]
means the format unit is 'i', it consumes a Python 'int', and it produces a C 'int'.  Similarly,
    s (str) [const char *]
means the format unit is 's', it consumes a Python 'str', and it produces a C 'const char *'.

When you call PyArg_ParseTuple (AndKeywords), you pass in a pointer to the thing you expect.  If it gives you an int, you pass in &my_int. So the type of the expression you pass in for 'i' is actually "int *".  And the type you pass in for 's' is actually "char **".

The format units that deal with encodings are a bit weirder.  You actually pass in a const char * string first, followed by the buffer you want to write data too.  Technically the types of the values you pass in for "es" are "const char *, char **".  But the documentation for es says
    es (str) [const char *encoding, char **buffer]

This led me to believe that I actually had to pass in a "char ***" for buffer!  Which is wrong and doing so makes your programs explode-y.


The documentation should

* explain this first-line convention precisely, and

* use the types consistently.

My suspicion is that the things in brackets have to be the precise C type, e.g. "int *" for i, "char **" for s, "const char *, char **" for es.

----------
assignee: larry
components: Documentation
messages: 241292
nosy: benjamin.peterson, georg.brandl, larry
priority: normal
severity: normal
stage: needs patch
status: open
title: Documentation for format units starting with 'e' is inconsistent
type: enhancement
versions: Python 2.7, Python 3.4, Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23980>
_______________________________________

From report at bugs.python.org  Fri Apr 17 02:00:35 2015
From: report at bugs.python.org (Christie)
Date: Fri, 17 Apr 2015 00:00:35 +0000
Subject: [New-bugs-announce] [issue23981] Update test_unicodedata.py to use
	script_helpers
Message-ID: <1429228835.0.0.167852596687.issue23981@psf.upfronthosting.co.za>


New submission from Christie:

As described in Issue9517, many test modules do not make use of the helpers in script_helpers.py to invoke the python interpreter in a subprocess. Issue9517 will be broken down into several smaller issues so we can address smaller change sets.

This issue is to update test_unicodedata.py to use script_helpers.py.

----------
components: Tests
messages: 241293
nosy: bobcatfish
priority: normal
severity: normal
status: open
title: Update test_unicodedata.py to use script_helpers
type: enhancement
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23981>
_______________________________________

From report at bugs.python.org  Fri Apr 17 02:51:39 2015
From: report at bugs.python.org (Martin Falatic)
Date: Fri, 17 Apr 2015 00:51:39 +0000
Subject: [New-bugs-announce] [issue23982] Tkinter in Python 3.4 for Windows
	incorrectly renders	certain colors using non-TclTk RGB values
Message-ID: <1429231899.59.0.405315540238.issue23982@psf.upfronthosting.co.za>


New submission from Martin Falatic:

In Python 2.7.9 for Windows, colors displayed match their RGB values as defined in TclTk:
http://www.tcl.tk/man/tcl8.5/TkCmd/colors.htm (8.6 is identical)

In Python 3.4.3 for Windows, the following colors differ noticeably from their TclTk counterparts: grey/gray, green, purple, and maroon.

Instead of the spec TclTk RGB values, these particular colors are rendered using the HTML RGB values.

This only happens in Python for Windows - OSX doesn't have this problem as it appears to use the same TclTk package for both 2.7 and 3.4 and correctly renders the colors per TclTk specs in each.

Tkinter ought to render named colors using the spec TclTk RGB values regardless of platform.

----------
components: Tkinter
files: colortest.py
messages: 241300
nosy: MartyMacGyver
priority: normal
severity: normal
status: open
title: Tkinter in Python 3.4 for Windows incorrectly renders certain colors using non-TclTk RGB values
type: behavior
versions: Python 3.4
Added file: http://bugs.python.org/file39081/colortest.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23982>
_______________________________________

From report at bugs.python.org  Fri Apr 17 03:17:28 2015
From: report at bugs.python.org (Berker Peksag)
Date: Fri, 17 Apr 2015 01:17:28 +0000
Subject: [New-bugs-announce] [issue23983] Update example in the pty
	documentation
Message-ID: <1429233448.23.0.72580455628.issue23983@psf.upfronthosting.co.za>


New submission from Berker Peksag:

I was reading pty docs after watching a PyCon talk and the example in https://docs.python.org/3/library/pty.html#example looked a bit outdated to me. Here is a patch to update it.

Changes:

* Fixed a ResourceWarning warning
* Used argparse instead of getopt

----------
assignee: docs at python
components: Documentation
files: pty-example.diff
keywords: patch
messages: 241301
nosy: berker.peksag, docs at python
priority: normal
severity: normal
stage: patch review
status: open
title: Update example in the pty documentation
type: enhancement
versions: Python 3.4, Python 3.5
Added file: http://bugs.python.org/file39082/pty-example.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23983>
_______________________________________

From report at bugs.python.org  Fri Apr 17 06:42:08 2015
From: report at bugs.python.org (bjonnh)
Date: Fri, 17 Apr 2015 04:42:08 +0000
Subject: [New-bugs-announce] [issue23984] Documentation error: Descriptors
Message-ID: <1429245728.01.0.810263114275.issue23984@psf.upfronthosting.co.za>


New submission from bjonnh:

in
https://docs.python.org/3.5/howto/descriptor.html#static-methods-and-class-methods

(same problem in all python 3.x documentations)

There is this example where the return of f function is printed and there is already a print in the function:
"""
>>> class E(object):
     def f(x):
          print(x)
     f = staticmethod(f)

>>> print(E.f(3))
3
>>> print(E().f(3))
3
"""

It should probably be:

"""
def f(x):
    return(x)
"""

or 

"""
>> E.f(3)
3
>> E().f(3)
"""

----------
assignee: docs at python
components: Documentation
messages: 241312
nosy: bjonnh, docs at python
priority: normal
severity: normal
status: open
title: Documentation error: Descriptors
type: enhancement
versions: Python 3.2, Python 3.3, Python 3.4, Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23984>
_______________________________________

From report at bugs.python.org  Fri Apr 17 08:37:48 2015
From: report at bugs.python.org (Johan Dahlberg)
Date: Fri, 17 Apr 2015 06:37:48 +0000
Subject: [New-bugs-announce] [issue23985] Crash when deleting slices from
	duplicated bytearray
Message-ID: <1429252668.91.0.45841644858.issue23985@psf.upfronthosting.co.za>


New submission from Johan Dahlberg:

Python 3.4.3 crashes after some time when running the attached program under Windows 7.

The program appends a fixed bytes "string" to two independent bytearray buffers.
If slices are removed from the beginnging of the two buffers and the two buffers are print:ed, the program will crash at some random occation.

----------
components: Windows
files: bytearray_bug.py
messages: 241316
nosy: johan, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Crash when deleting slices from duplicated bytearray
type: crash
versions: Python 3.4
Added file: http://bugs.python.org/file39086/bytearray_bug.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23985>
_______________________________________

From report at bugs.python.org  Fri Apr 17 09:22:58 2015
From: report at bugs.python.org (wim glenn)
Date: Fri, 17 Apr 2015 07:22:58 +0000
Subject: [New-bugs-announce] [issue23986] Inaccuracy about "in" keyword for
	list and tuple
Message-ID: <1429255378.08.0.140067438597.issue23986@psf.upfronthosting.co.za>


New submission from wim glenn:

The comparisons section of the python 2 docs says:

--- 

https://docs.python.org/2/reference/expressions.html#comparisons

For the list and tuple types, x in y is true if and only if there exists an index i such that x == y[i] is true.

---

But it's not strictly speaking correct.  Simplest counter-example:

x = float('nan')
y = [x]

The python 3 docs instead mention correct equivalent which is 

any(x is e or x == e for e in y)

----------
assignee: docs at python
components: Documentation
messages: 241317
nosy: docs at python, wim.glenn
priority: normal
severity: normal
status: open
title: Inaccuracy about "in" keyword for list and tuple
type: enhancement
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23986>
_______________________________________

From report at bugs.python.org  Fri Apr 17 10:09:25 2015
From: report at bugs.python.org (Ethan Furman)
Date: Fri, 17 Apr 2015 08:09:25 +0000
Subject: [New-bugs-announce] [issue23987] docs about containers membership
	testing wrong for	broken objects
Message-ID: <1429258165.63.0.284734084008.issue23987@psf.upfronthosting.co.za>


New submission from Ethan Furman:

https://docs.python.org/3/reference/expressions.html#comparisons:
----------------------------------------------------------------
The operators 'in' and 'not in' test for membership. 'x in s' evaluates to true if x is a member of s, and false otherwise. 'x not in s' returns the negation of 'x in s'. All built-in sequences and set types support this as well as dictionary, for which 'in' tests whether the dictionary has a given key. For container types such as list, tuple, set, frozenset, dict, or collections.deque, the expression 'x in y' is equivalent to 'any(x is e or x == e for e in y)'.

StackOverflow question for context: http://stackoverflow.com/q/29692140/208880

Summary:  if a user creates a broken object such that __hash__ returns a random number with every invocation, then that object will get lost in a dict or set; but the above statement about 'equivalent to' claims that such an object will still be found.

On the other hand, https://docs.python.org/3/glossary.html#term-hashable says that a constant return value is required for an object to be hashable (of course, Python can't tell if future calls to __hash__ will return the same value).

Perhaps a link to the #term-hashable would be appropriate?

----------
messages: 241320
nosy: eric.araujo, ethan.furman, ezio.melotti, georg.brandl
priority: normal
severity: normal
status: open
title: docs about containers membership testing wrong for broken objects
versions: Python 3.4, Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23987>
_______________________________________

From report at bugs.python.org  Fri Apr 17 16:40:27 2015
From: report at bugs.python.org (Albert Zeyer)
Date: Fri, 17 Apr 2015 14:40:27 +0000
Subject: [New-bugs-announce] [issue23988] keyworded argument count is wrong
Message-ID: <1429281627.58.0.914440284971.issue23988@psf.upfronthosting.co.za>


New submission from Albert Zeyer:

Code:

class C(object):
	def __init__(self, a, b=2, c=3):
		pass

class D(C):
	def __init__(self, d, **kwargs):
		super(D, self).__init__(**kwargs)

class E(D):
	def __init__(self, **kwargs):
		super(E, self).__init__(**kwargs)

E(d=42, b=0, c=0)

You get the funny message:

TypeError: __init__() takes at least 2 arguments (3 given)

----------
components: Interpreter Core
messages: 241335
nosy: Albert.Zeyer
priority: normal
severity: normal
status: open
title: keyworded argument count is wrong
type: behavior
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23988>
_______________________________________

From report at bugs.python.org  Fri Apr 17 20:33:42 2015
From: report at bugs.python.org (VanL)
Date: Fri, 17 Apr 2015 18:33:42 +0000
Subject: [New-bugs-announce] [issue23989] Add recommendation to use requests
	to the documentation, per summit
Message-ID: <1429295622.31.0.515179904566.issue23989@psf.upfronthosting.co.za>


New submission from VanL:

Attached is a basic patch to add a recommendation to use requests to the http.client documentation for 3.5. It is implemented as a seealso at the top of the file: 

See Also: The Requests package is recommended for a higher-level http client interface.

If people are happy with the wording, I'll also add the same message to urllib.request and do the same with previous versions (2.7, 3.3, and 3.4).

----------
assignee: docs at python
components: Documentation
files: recommend_requests.patch
keywords: patch
messages: 241349
nosy: VanL, docs at python
priority: normal
severity: normal
status: open
title: Add recommendation to use requests to the documentation, per summit
type: enhancement
versions: Python 3.5
Added file: http://bugs.python.org/file39088/recommend_requests.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23989>
_______________________________________

From report at bugs.python.org  Fri Apr 17 20:52:48 2015
From: report at bugs.python.org (Maries Ionel Cristian)
Date: Fri, 17 Apr 2015 18:52:48 +0000
Subject: [New-bugs-announce] [issue23990] Callable builtin doesn't respect
	descriptors
Message-ID: <1429296768.81.0.194105997153.issue23990@psf.upfronthosting.co.za>


New submission from Maries Ionel Cristian:

It appears that callable doesn't really care for the descriptor protocol, so it return True even if __call__ is actually an descriptor that raise AttributeError (clearly not callable at all).

Eg:

Python 3.4.3 (v3.4.3:9b73f1c3e601, Feb 24 2015, 22:44:40) [MSC v.1600 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> callable
<built-in function callable>
>>> class A:
...  @property
...  def __call__(self):
...   raise AttributeError('go away')
...
>>> a = A()
>>> a
<__main__.A object at 0x000000000365B5C0>
>>> a.__call__
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 4, in __call__
AttributeError: go away
>>> callable(a)
True
>>> # it should be False :(

----------
components: Interpreter Core
messages: 241352
nosy: ionel.mc
priority: normal
severity: normal
status: open
title: Callable builtin doesn't respect descriptors
type: behavior
versions: Python 2.7, Python 3.2, Python 3.3, Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23990>
_______________________________________

From report at bugs.python.org  Sat Apr 18 06:44:07 2015
From: report at bugs.python.org (Antony Lee)
Date: Sat, 18 Apr 2015 04:44:07 +0000
Subject: [New-bugs-announce] [issue23991] ZipFile sanity checks
Message-ID: <1429332247.71.0.71910598875.issue23991@psf.upfronthosting.co.za>


New submission from Antony Lee:

ZipFile.{open,read} could raise IsADirectoryError when called on a directory entry, rather than return an empty bytes.
ZipFile.writestr could fail writing non-empty bytes to a directory entry.

Use case for open and read: I was trying to write a zip merger (taking multiple zip files and merge them into a single zip).  Unfortunately, the obvious approach (iterate on the original infolists, extract each of them and writestr them to the new ZipFile object) created invalid zip files because it'll create empty files for each directory.  Raising IsADirectoryError would have brought attention to the issue much earlier.

The suggested modification to writestr is a bit different, but also makes sense because the bytes argument is not written anywhere anyways in the case of directory entries.

----------
components: Library (Lib)
messages: 241396
nosy: Antony.Lee
priority: normal
severity: normal
status: open
title: ZipFile sanity checks
versions: Python 3.4, Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23991>
_______________________________________

From report at bugs.python.org  Sat Apr 18 11:00:21 2015
From: report at bugs.python.org (=?utf-8?q?Charles-Fran=C3=A7ois_Natali?=)
Date: Sat, 18 Apr 2015 09:00:21 +0000
Subject: [New-bugs-announce] [issue23992] multiprocessing: MapResult
	shouldn't fail fast upon	exception
Message-ID: <1429347621.56.0.157795358647.issue23992@psf.upfronthosting.co.za>


New submission from Charles-Fran?ois Natali:

hanger.py
"""
from time import sleep


def hang(i):
    sleep(i)
    raise ValueError("x" * 1024**2)
"""


The following code will deadlock on pool.close():
"""
from multiprocessing import Pool
from time import sleep

from hanger import hang


with Pool() as pool:
    try:
        pool.map(hang, [0,1])
    finally:
        sleep(0.5)
        pool.close()
        pool.join()
"""

The problem is that when one of the tasks comprising a map result fails with an exception, the corresponding MapResult is removed from the result cache:

    def _set(self, i, success_result):
        success, result = success_result
        if success:
            [snip]
        else:
            self._success = False
            self._value = result
            if self._error_callback:
                self._error_callback(self._value)
<===
            del self._cache[self._job]
            self._event.set()
===>

Which means that when the pool is closed, the result handler thread terminates right away, because it doesn't see any task left to wait for.
Which means that it doesn't drain the result queue, and if some worker process is trying to write a large result to it (hence the large valuerrror to fill the socket/pipe buffer), it will hang, and the pool won't shut down (unless you call terminate()).

Although I can see the advantage of fail-fast behavior, I don't think it's correct because it breaks the invariant where results won't be deleted from the cache until they're actually done.

Also, the current fail-fast behavior breaks the semantics that the call only returns when it has completed.
Returning while some jobs part of the map are still running is potentially very bad, e.g. if the user call retries the same call, assuming that all the jobs are done. Retrying jobs that are idempotent but not parallel execution-safe would break with the current code.


The fix is trivial, use the same logic as in case of success to only signal failure when all jobs are done.

I'll provide a patch if it seems sensible :-)

----------
components: Library (Lib)
messages: 241404
nosy: neologix, pitrou
priority: normal
severity: normal
status: open
title: multiprocessing: MapResult shouldn't fail fast upon exception
type: behavior
versions: Python 2.7, Python 3.4, Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23992>
_______________________________________

From report at bugs.python.org  Sat Apr 18 11:25:51 2015
From: report at bugs.python.org (STINNER Victor)
Date: Sat, 18 Apr 2015 09:25:51 +0000
Subject: [New-bugs-announce] [issue23993] Use surrogateescape error handler
	by default in open()	if the locale is C
Message-ID: <1429349151.1.0.641370404.issue23993@psf.upfronthosting.co.za>


New submission from STINNER Victor:

As a following of the issue #19977, I propose to use also the surrogateescape error handler in open() by default if the locale is C.

Attached issue adds a new sys.getdefaulterrorhandler() function and use it in io.TextIOWrapper (and _pyio.TextIOWrapper).

We may use sys.getdefaulterrorhandler() in more places.

I don't think that it would be correct to use in for str.encode() or bytes.decode().

----------
components: Unicode
files: default_error_handler.patch
keywords: patch
messages: 241405
nosy: ezio.melotti, haypo, ncoghlan
priority: normal
severity: normal
status: open
title: Use surrogateescape error handler by default in open() if the locale is C
versions: Python 3.5
Added file: http://bugs.python.org/file39100/default_error_handler.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23993>
_______________________________________

From report at bugs.python.org  Sat Apr 18 19:48:03 2015
From: report at bugs.python.org (Mert Bora Alper)
Date: Sat, 18 Apr 2015 17:48:03 +0000
Subject: [New-bugs-announce] [issue23994] argparse fails to detect program
	name when there is a	slash at the end of the program's path
Message-ID: <1429379283.39.0.0511547854084.issue23994@psf.upfronthosting.co.za>


New submission from Mert Bora Alper:

Sorry if the title is not descriptive enough.

When I try to execute a program from a directory which contains an `__main__.py` file, argparse fails to detect programs name. For example:

    $ ls foo
    __main__.py
    $ python3 foo
    usage: foo [-h] [-c COUNT] length
    foo: error: the following arguments are required: length
    $ python3 foo/
    usage: [-h] [-c COUNT] length
    : error: the following arguments are required: length

----

    >>> argparse.__version__
    '1.1'

I followed same steps using Python 2.7.6 and got same result. It will probably be same on other versions too. Same result can also be obtained when using zip files instead of directories.

I don't know if this is a bug since I don't know if I do something wrong or is this a design decision or not.

Thank you,
Bora

----------
components: Library (Lib)
files: __main__.py
messages: 241434
nosy: boramalper
priority: normal
severity: normal
status: open
title: argparse fails to detect program name when there is a slash at the end of the program's path
type: behavior
versions: Python 2.7, Python 3.4
Added file: http://bugs.python.org/file39106/__main__.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23994>
_______________________________________

From report at bugs.python.org  Sat Apr 18 20:45:16 2015
From: report at bugs.python.org (petrikas)
Date: Sat, 18 Apr 2015 18:45:16 +0000
Subject: [New-bugs-announce] [issue23995] msvcrt could not be imported
Message-ID: <1429382716.1.0.015048526051.issue23995@psf.upfronthosting.co.za>


New submission from petrikas:

Python cannot access msvcrt's putwch() when using manage.py syncdb

To reproduce:
1. Call manage.py syncdb and try to create a new superuser
2. It crashes after inputting email (or before asking for the password)

Reproducible with 3.5a3, seems to be a regression from 3.4.3. Downgrading fixed the issue

----------
components: Library (Lib)
messages: 241438
nosy: petrikas
priority: normal
severity: normal
status: open
title: msvcrt could not be imported
type: crash
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23995>
_______________________________________

From report at bugs.python.org  Sat Apr 18 21:45:13 2015
From: report at bugs.python.org (Stefan Behnel)
Date: Sat, 18 Apr 2015 19:45:13 +0000
Subject: [New-bugs-announce] [issue23996] _PyGen_FetchStopIterationValue()
	crashes on unnormalised	exceptions
Message-ID: <1429386313.82.0.910239252026.issue23996@psf.upfronthosting.co.za>


New submission from Stefan Behnel:

The yield-from implementation calls _PyGen_FetchStopIterationValue() to get the exception value. If the StopIteration exception is not normalised, e.g. because it was set by PyErr_SetObject() in a C extension, then _PyGen_FetchStopIterationValue() will cast to (PyStopIterationObject*) whatever the exception value is and happily interpret an arbitrary memory position as PyObject*.

I attached a possible patch for the function. Another place to fix it would be in the yield-from code in ceval.c, but directly genobject.c seems the safer place.

----------
components: Interpreter Core
files: fix_stopiteration_crash.patch
keywords: patch
messages: 241454
nosy: scoder
priority: normal
severity: normal
status: open
title: _PyGen_FetchStopIterationValue() crashes on unnormalised exceptions
type: crash
versions: Python 3.3, Python 3.4, Python 3.5
Added file: http://bugs.python.org/file39108/fix_stopiteration_crash.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23996>
_______________________________________

From report at bugs.python.org  Sun Apr 19 00:32:38 2015
From: report at bugs.python.org (Christian Heimes)
Date: Sat, 18 Apr 2015 22:32:38 +0000
Subject: [New-bugs-announce] [issue23997] unicodedata_UCD_lookup() has
	theoretical buffer overflow
Message-ID: <1429396358.28.0.910833101692.issue23997@psf.upfronthosting.co.za>


New submission from Christian Heimes:

Coverity has found a potential buffer overflow in the unicodedata module. The function call _getcode() which calls _cmpname(). _cmpname() copies data into fixed size buffer of length NAME_MAXLEN. Neither lookup() nor _getcode() limit name_length to NAME_MAXLEN. Therefore the buffer could theoretical overflow.

In practice the buffer overflow can't be abused because Tools/unicode/makeunicodedata.py already limits max name length. I still like to fix the bug because it is a low hanging fruit. In most versions of Python the code already checks that name_length fits in INT_MAX.

CID 1295028 (#1 of 1): Out-of-bounds access (OVERRUN)
overrun-call: Overrunning callee's array of size 256 by passing argument (int)name_length (which evaluates to 2147483647) in call to _getcode

----------
files: unicode_name_maxlen.patch
keywords: patch
messages: 241461
nosy: benjamin.peterson, christian.heimes, ezio.melotti, haypo, lemburg, pitrou, serhiy.storchaka
priority: normal
severity: normal
stage: patch review
status: open
title: unicodedata_UCD_lookup() has theoretical buffer overflow
type: behavior
versions: Python 2.7, Python 3.3, Python 3.4, Python 3.5
Added file: http://bugs.python.org/file39109/unicode_name_maxlen.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23997>
_______________________________________

From report at bugs.python.org  Sun Apr 19 00:42:35 2015
From: report at bugs.python.org (Christian Heimes)
Date: Sat, 18 Apr 2015 22:42:35 +0000
Subject: [New-bugs-announce] [issue23998] PyImport_ReInitLock() doesn't
	check for allocation error
Message-ID: <1429396955.07.0.854315130889.issue23998@psf.upfronthosting.co.za>


New submission from Christian Heimes:

_PyImport_ReInitLock() doesn't check the return value of PyThread_allocate_lock(). A failed lock allocation can either lead to a NULL pointer dereference or to race conditions caused by a missing import lock.

As there is no way to recover from a failed lock allication I recommend to abort with a fatal error.

CID 1295025 (#1 of 1): Dereference after null check (FORWARD_NULL)
var_deref_model: Passing null pointer import_lock to PyThread_acquire_lock, which dereferences it.

----------
files: import_reinit_fatal.patch
keywords: buildbot, easy, patch
messages: 241462
nosy: christian.heimes
priority: normal
severity: normal
stage: patch review
status: open
title: PyImport_ReInitLock() doesn't check for allocation error
type: behavior
versions: Python 2.7, Python 3.3, Python 3.4, Python 3.5
Added file: http://bugs.python.org/file39110/import_reinit_fatal.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23998>
_______________________________________

From report at bugs.python.org  Sun Apr 19 00:52:13 2015
From: report at bugs.python.org (Christian Heimes)
Date: Sat, 18 Apr 2015 22:52:13 +0000
Subject: [New-bugs-announce] [issue23999] Undefined behavior in dtoa.c
	(rshift 32 of 32bit data	type)
Message-ID: <1429397533.86.0.186055088306.issue23999@psf.upfronthosting.co.za>


New submission from Christian Heimes:

Coverity has found undefined behavior in dtoa.c:d2b(). lo0bits() can return 32 which z >>= 32, where z is an uint32. I've talked to doku at PyCon. He suggested to update dtoa.c to a more recent version. Our copy is based on a version from 2001. There are more modern versions available, e.g. https://searchcode.com/codesearch/view/52748288/ from 2006.

CID 1202735 (#1 of 1): Bad bit shift operation (BAD_SHIFT)
large_shift: In expression z >>= k, right shifting by more than 31 bits has undefined behavior. The shift amount, k, is 32.

----------
messages: 241464
nosy: christian.heimes, doko, eric.smith, mark.dickinson, serhiy.storchaka
priority: normal
severity: normal
stage: needs patch
status: open
title: Undefined behavior in dtoa.c (rshift 32 of 32bit data type)
type: behavior
versions: Python 2.7, Python 3.3, Python 3.4, Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23999>
_______________________________________

From report at bugs.python.org  Sun Apr 19 03:00:33 2015
From: report at bugs.python.org (Larry Hastings)
Date: Sun, 19 Apr 2015 01:00:33 +0000
Subject: [New-bugs-announce] [issue24000] More fixes for the Clinic mapping
	of converters to	format units
Message-ID: <1429405233.93.0.203198864557.issue24000@psf.upfronthosting.co.za>


New submission from Larry Hastings:

I found another bug in the mapping of converters to format units.  (s#, z#, and y# all allow zeroes.)

I've redone the approach for str_converter in an attempt to make it easier to read.

It'd be swell if, after this gets checked in (or rejected), somebody *else* took a pass to see if they could find any bugs.

----------
assignee: larry
files: larry.one.more.clinic.format.unit.map.cleanup.1.txt
messages: 241468
nosy: larry, serhiy.storchaka, zach.ware
priority: normal
severity: normal
stage: patch review
status: open
title: More fixes for the Clinic mapping of converters to format units
type: behavior
versions: Python 3.5
Added file: http://bugs.python.org/file39113/larry.one.more.clinic.format.unit.map.cleanup.1.txt

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24000>
_______________________________________

From report at bugs.python.org  Sun Apr 19 03:06:17 2015
From: report at bugs.python.org (Larry Hastings)
Date: Sun, 19 Apr 2015 01:06:17 +0000
Subject: [New-bugs-announce] [issue24001] Clinic: use raw types in types= set
Message-ID: <1429405577.22.0.134829799978.issue24001@psf.upfronthosting.co.za>


New submission from Larry Hastings:

New proposed semantics for the types= parameter to converters: where possible, pass in actual types.  The resulting syntax:

  c: int(types={str}) # maps to 'U'
  s: str(types={str, robuffer}, length=True, zeroes=True) # maps to 's#'

Since "buffer", "robuffer", and "rwbuffer" are fake pseudotypes, I created empty classes with those names.

Serhiy: with this change in place, the types= parameter uses almost the same number of characters as it used to when it was a string.  (The new syntax requires commas between elements, so for two or more types it's slightly longer.)  Yet this makes the types= parameter far more accurate in illustrating what it's supposed to represent.

Does this make you happy? :)

----------
assignee: larry
components: Argument Clinic
files: larry.clinic.use.raw.types.1.txt
messages: 241469
nosy: larry, serhiy.storchaka, zach.ware
priority: normal
severity: normal
stage: patch review
status: open
title: Clinic: use raw types in types= set
type: enhancement
Added file: http://bugs.python.org/file39114/larry.clinic.use.raw.types.1.txt

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24001>
_______________________________________

From report at bugs.python.org  Sun Apr 19 07:39:10 2015
From: report at bugs.python.org (Larry Hastings)
Date: Sun, 19 Apr 2015 05:39:10 +0000
Subject: [New-bugs-announce] [issue24002] Add un-parse function to ast
Message-ID: <1429421950.27.0.647153175832.issue24002@psf.upfronthosting.co.za>


New submission from Larry Hastings:

Twice recently I've wanted a function that transforms an AST node tree back into text:

* In the hacked-up Tools/clinic/clinic.py for issue #24001
* In the hacked-up Lib/inspect.py for issue #23967

Both times I did a half-assed job just to get the patch working, so we could decide whether or not we want to go this route.

It seems useful and reasonable to do a proper job and add this functionality to ast.

----------
components: Library (Lib)
messages: 241477
nosy: benjamin.peterson, georg.brandl, larry
priority: normal
severity: normal
stage: needs patch
status: open
title: Add un-parse function to ast
type: enhancement
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24002>
_______________________________________

From report at bugs.python.org  Sun Apr 19 11:43:09 2015
From: report at bugs.python.org (john kaiser)
Date: Sun, 19 Apr 2015 09:43:09 +0000
Subject: [New-bugs-announce] [issue24003] variable naming
Message-ID: <1429436589.53.0.115033334607.issue24003@psf.upfronthosting.co.za>


New submission from john kaiser:

found error when naming variables with basic functions

how to replicate

while=123 #while should be treated as variable name
while True: #this should be treated as a function
	print while #this should be as a variable name

result:
  File "C:\Users\_you got served_\Desktop\bugs.py", line 1
    while = 123
          ^
SyntaxError: invalid syntax

----------
files: bugs.py
messages: 241502
nosy: kaiser
priority: normal
severity: normal
status: open
title: variable naming
type: behavior
versions: Python 2.7
Added file: http://bugs.python.org/file39117/bugs.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24003>
_______________________________________

From report at bugs.python.org  Sun Apr 19 12:21:02 2015
From: report at bugs.python.org (Stefan Behnel)
Date: Sun, 19 Apr 2015 10:21:02 +0000
Subject: [New-bugs-announce] [issue24004] avoid explicit generator type
	check in asyncio
Message-ID: <1429438862.32.0.692811371536.issue24004@psf.upfronthosting.co.za>


New submission from Stefan Behnel:

asyncio/coroutines.py contains this code:

"""
_COROUTINE_TYPES = (types.GeneratorType, CoroWrapper)

def iscoroutine(obj):
    """Return True if obj is a coroutine object."""
    return isinstance(obj, _COROUTINE_TYPES)
"""

In other places, it uses inspect.isgenerator() to do the same thing. This is inconsistent and should be changed. Code that wants to patch inspect.isgenerator() in order to support other generator types shouldn't also have to patch asyncio directly, and code that wants to support other generator types in asyncio shouldn't have to patch both places either.

----------
components: asyncio
messages: 241506
nosy: gvanrossum, haypo, scoder, yselivanov
priority: normal
severity: normal
status: open
title: avoid explicit generator type check in asyncio
type: behavior
versions: Python 3.4, Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24004>
_______________________________________

From report at bugs.python.org  Sun Apr 19 12:29:29 2015
From: report at bugs.python.org (Jaivish Kothari)
Date: Sun, 19 Apr 2015 10:29:29 +0000
Subject: [New-bugs-announce] [issue24005] Documentation Error: Extra line
	Break
Message-ID: <1429439369.0.0.722585874502.issue24005@psf.upfronthosting.co.za>


New submission from Jaivish Kothari:

https://docs.python.org/2/whatsnew/2.4.html?highlight=decorators#pep-318-decorators-for-functions-and-methods
'''
def require_int (func):
    def wrapper (arg):
        assert isinstance(arg, int)
        return func(arg)

    return wrapper
'''

New line is ommited before return wrapper 

'''
def require_int (func):
    def wrapper (arg):
        assert isinstance(arg, int)
        return func(arg)
    return wrapper

'''

----------
assignee: docs at python
components: Documentation
messages: 241508
nosy: docs at python, georg.brandl, janonymous
priority: normal
severity: normal
status: open
title: Documentation Error: Extra line Break
type: enhancement
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24005>
_______________________________________

From report at bugs.python.org  Sun Apr 19 19:21:01 2015
From: report at bugs.python.org (ppperry)
Date: Sun, 19 Apr 2015 17:21:01 +0000
Subject: [New-bugs-announce] [issue24006] Multiprocessing fails when using
	functions defined in	interactive interpreter.
Message-ID: <1429464061.97.0.0459539459978.issue24006@psf.upfronthosting.co.za>


New submission from ppperry:

An AttributeError is raised when starting a new process with an object defined in the interactive interpreter
>>>def test():print "test"
>>>test()
test
>>>from threading import Thread
>>>Thread(target=test).start()
test
>>>from multiprocessing import Process
>>>Process(target=test).start()
<traceback, ending with>
File <path to python>/Lib/pickle.py, line 1126, in find_class
      klass = getattr(mod, name)
AttributeError: 'module' object has no attribute 'test'
running similar code in a file works and does not raise an error.

----------
components: Interpreter Core, Library (Lib)
messages: 241521
nosy: ppperry
priority: normal
severity: normal
status: open
title: Multiprocessing fails when using functions defined in interactive interpreter.
type: behavior
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24006>
_______________________________________

From report at bugs.python.org  Sun Apr 19 20:30:47 2015
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sun, 19 Apr 2015 18:30:47 +0000
Subject: [New-bugs-announce] [issue24007] Write PyArg_Parse* format in a
	line with a function
Message-ID: <1429468247.73.0.580298527678.issue24007@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka:

Proposed patch makes Argument Clinic to output format argument of PyArg_Parse, PyArg_ParseTuple and PyArg_ParseTupleAndKeywords in a line with a function itself. First, this makes generated code more compact and easier to read and compare with old code. Second, it makes easier to grep sources for used format units. Format argument always (or almost always) is written in a line with a function.

----------
components: Argument Clinic, Demos and Tools
files: clinic_format_argument.patch
keywords: patch
messages: 241531
nosy: larry, serhiy.storchaka
priority: normal
severity: normal
stage: patch review
status: open
title: Write PyArg_Parse* format in a line with a function
type: enhancement
versions: Python 3.5
Added file: http://bugs.python.org/file39121/clinic_format_argument.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24007>
_______________________________________

From report at bugs.python.org  Sun Apr 19 20:53:57 2015
From: report at bugs.python.org (Anand Reddy Pandikunta)
Date: Sun, 19 Apr 2015 18:53:57 +0000
Subject: [New-bugs-announce] [issue24008] inspect.getsource is failing for
	sys.excepthook
Message-ID: <1429469637.6.0.278497450478.issue24008@psf.upfronthosting.co.za>


New submission from Anand Reddy Pandikunta:

>>> inspect.getsource(sys.excepthook)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/inspect.py", line 701, in getsource
    lines, lnum = getsourcelines(object)
  File "/usr/local/lib/python2.7/inspect.py", line 690, in getsourcelines
    lines, lnum = findsource(object)
  File "/usr/local/lib/python2.7/inspect.py", line 526, in findsource
    file = getfile(object)
  File "/usr/local/lib/python2.7/inspect.py", line 420, in getfile
    'function, traceback, frame, or code object'.format(object))
TypeError: <built-in function excepthook> is not a module, class, method, function, traceback, frame, or code object

----------
components: Library (Lib)
files: trace4.py
messages: 241537
nosy: ChillarAnand
priority: normal
severity: normal
status: open
title: inspect.getsource is failing for sys.excepthook
versions: Python 2.7
Added file: http://bugs.python.org/file39124/trace4.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24008>
_______________________________________

From report at bugs.python.org  Sun Apr 19 21:14:44 2015
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sun, 19 Apr 2015 19:14:44 +0000
Subject: [New-bugs-announce] [issue24009] Get rid of rare format units in
	PyArg_Parse*
Message-ID: <1429470884.11.0.538472438707.issue24009@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka:

There are a lot of format units supported in PyArg_Parse* functions, but some of them are rarely or never used in current CPython code. Some of format units are legacy from Python 2 and are not needed in modern Python 3 code or can be replaced with custom converter.

Here are results of grepping (not including Modules/_testcapimodule.c).

"es", "es#", "et#", "z*", "Z#" are not used.

"y#":
Modules/_io/textio.c:2334:        if (!PyArg_ParseTuple(_state, "y#i", &dec_buffer, &dec_buffer_len, &dec_flags)) { \

"z#":
Modules/_ctypes/_ctypes.c:3327:    if (!PyArg_ParseTuple(args, "is|Oz#", &index, &name, &paramflags, &iid, &iid_len))

"u#":
Modules/arraymodule.c:248:    if (!PyArg_Parse(v, "u#;array item must be unicode character", &p, &len))
PC/winreg.c:1547:    if (!PyArg_ParseTuple(args, "OZiu#:SetValue",

"y":
Modules/_io/textio.c:2334:        if (!PyArg_ParseTuple(_state, "y#i", &dec_buffer, &dec_buffer_len, &dec_flags)) { \
Modules/_cursesmodule.c:2790:    if (!PyArg_ParseTuple(args,"y;str", &str))
Modules/_cursesmodule.c:3026:    if (!PyArg_ParseTuple(args, "y|iiiiiiiii:tparm",
Modules/posixmodule.c:3767:    if (!PyArg_ParseTuple (args, "y:_getfullpathname",
Modules/posixmodule.c:3872:    if (!PyArg_ParseTuple(args, "y:_isdir", &path))
Modules/faulthandler.c:941:    if (!PyArg_ParseTuple(args, "y:fatal_error", &message))

"et":
Modules/socketmodule.c:4499:    if (!PyArg_ParseTuple(args, "et:gethostbyname", "idna", &name))
Modules/socketmodule.c:4667:    if (!PyArg_ParseTuple(args, "et:gethostbyname_ex", "idna", &name))
Modules/socketmodule.c:4744:    if (!PyArg_ParseTuple(args, "et:gethostbyaddr", "idna", &ip_num))
Modules/_tkinter.c:2099:    if (!PyArg_ParseTuple(args, "et:splitlist", "utf-8", &list))
Modules/_tkinter.c:2162:    if (!PyArg_ParseTuple(args, "et:split", "utf-8", &list))
Modules/_ssl.c:3038:        if (!PyArg_ParseTupleAndKeywords(args, kwds, "O!iet:_wrap_socket", kwlist,
Modules/_ssl.c:3070:        if (!PyArg_Parse(hostname_obj, "et", "idna", &hostname))

"s*":
Modules/_codecsmodule.c:188:    if (!PyArg_ParseTuple(args, "s*|z:escape_decode",
Modules/_codecsmodule.c:552:    if (!PyArg_ParseTuple(args, "s*|z:unicode_escape_decode",
Modules/_codecsmodule.c:569:    if (!PyArg_ParseTuple(args, "s*|z:raw_unicode_escape_decode",
Modules/_codecsmodule.c:696:    if (!PyArg_ParseTuple(args, "s*|z:readbuffer_encode",
Modules/_ssl.c:3734:    if (!PyArg_ParseTuple(args, "s*d:RAND_add", &view, &entropy))
Modules/fcntlmodule.c:225:        if (PyArg_Parse(ob_arg, "s*:ioctl", &pstr)) {
Modules/clinic/arraymodule.c.h:278:    if (!PyArg_Parse(arg, "s*:fromstring", &buffer))

"s#":
Modules/_gdbmmodule.c:128:    if (!PyArg_Parse(key, "s#", &krec.dptr, &krec.dsize) )
Modules/_gdbmmodule.c:176:    if (!PyArg_Parse(v, "s#", &krec.dptr, &krec.dsize) ) {
Modules/_gdbmmodule.c:194:        if (!PyArg_Parse(w, "s#", &drec.dptr, &drec.dsize)) {
Modules/fcntlmodule.c:71:        if (PyArg_Parse(arg, "s#", &str, &len)) {
Modules/_ctypes/_ctypes.c:2569:    if (!PyArg_ParseTuple(args, "Os#", &dict, &data, &len))
Modules/clinic/unicodedata.c.h:361:    if (!PyArg_Parse(arg, "s#:lookup", &name, &name_length))
Modules/clinic/_dbmmodule.c.h:62:    if (!PyArg_ParseTuple(args, "s#|O:get",
Modules/clinic/_dbmmodule.c.h:95:    if (!PyArg_ParseTuple(args, "s#|O:setdefault",
Modules/clinic/_gdbmmodule.c.h:150:    if (!PyArg_Parse(arg, "s#:nextkey", &key, &key_length))
Modules/_dbmmodule.c:108:    if (!PyArg_Parse(key, "s#", &krec.dptr, &tmp_size) )
Modules/_dbmmodule.c:132:    if ( !PyArg_Parse(v, "s#", &krec.dptr, &tmp_size) ) {
Modules/_dbmmodule.c:150:        if ( !PyArg_Parse(w, "s#", &drec.dptr, &tmp_size) ) {
Modules/_dbmmodule.c:336:        if ( !PyArg_Parse(default_value, "s#", &val.dptr, &tmp_size) ) {

In future may be we could deprecate some format units and remove them in 4.0.

This issue is a meta issue. Every case should be considered individually.

----------
assignee: serhiy.storchaka
components: Interpreter Core
messages: 241546
nosy: serhiy.storchaka
priority: low
severity: normal
status: open
title: Get rid of rare format units in PyArg_Parse*
type: enhancement
versions: Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24009>
_______________________________________

From report at bugs.python.org  Sun Apr 19 21:26:39 2015
From: report at bugs.python.org (Christian Heimes)
Date: Sun, 19 Apr 2015 19:26:39 +0000
Subject: [New-bugs-announce] [issue24010] Add error checks to
	PyInit__locale()
Message-ID: <1429471599.85.0.679825670624.issue24010@psf.upfronthosting.co.za>


New submission from Christian Heimes:

The init function of the locale module fails to check for errors in a couple of places. The patch replaces PyDict_SetItemString() calls with PyModule_AddIntMacro() and error checks. An exception is unlikely so I'm OK when the patch just lands in 3.4 and 3.5.

CID 1295027 (#8 of 8): Dereference null return value (NULL_RETURNS)
dereference: Dereferencing a pointer that might be null x when calling PyDict_SetItemString

----------
components: Extension Modules
files: localemodule.patch
keywords: patch
messages: 241550
nosy: christian.heimes, lemburg, loewis
priority: normal
severity: normal
stage: patch review
status: open
title: Add error checks to PyInit__locale()
type: behavior
versions: Python 3.4, Python 3.5
Added file: http://bugs.python.org/file39126/localemodule.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24010>
_______________________________________

From report at bugs.python.org  Sun Apr 19 21:28:15 2015
From: report at bugs.python.org (Christian Heimes)
Date: Sun, 19 Apr 2015 19:28:15 +0000
Subject: [New-bugs-announce] [issue24011] Add error checks to PyInit_signal()
Message-ID: <1429471695.96.0.523863138973.issue24011@psf.upfronthosting.co.za>


New submission from Christian Heimes:

The init function of the signal module fails to check for errors in a couple of places. The patch replaces PyDict_SetItemString() calls with PyModule_AddIntMacro() and error checks. An exception is unlikely so I'm OK when the patch just lands in 3.4 and 3.5.

CID 1295026 (#41 of 41): Dereference null return value (NULL_RETURNS)
dereference: Dereferencing a pointer that might be null x when calling PyDict_SetItemString

----------
components: Extension Modules
files: signalmodule.patch
keywords: patch
messages: 241551
nosy: christian.heimes
priority: normal
severity: normal
stage: patch review
status: open
title: Add error checks to PyInit_signal()
type: behavior
versions: Python 3.4, Python 3.5
Added file: http://bugs.python.org/file39127/signalmodule.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24011>
_______________________________________

From report at bugs.python.org  Sun Apr 19 21:33:15 2015
From: report at bugs.python.org (Christian Heimes)
Date: Sun, 19 Apr 2015 19:33:15 +0000
Subject: [New-bugs-announce] [issue24012] Add error checks to
	PyInit_pyexpat()
Message-ID: <1429471995.2.0.707709004529.issue24012@psf.upfronthosting.co.za>


New submission from Christian Heimes:

Similar to #24011 and #24010 the pyexpat module's init function fails to check some return values for NULL. The patch doesn't include proper reference cleanups as most of the other parts of PyInit_pyexpat() don't cleanup on error, too.

CID 982779 (#2 of 2): Dereference null return value (NULL_RETURNS)
dereference: Dereferencing a pointer that might be null sys_modules when calling PyDict_SetItem

CID 982240 (#2 of 2): Unchecked return value (CHECKED_RETURN)
check_return: Calling PyDict_SetItem without checking return value (as is done elsewhere 158 out of 174 times)

----------
components: Extension Modules
files: pyexpat.patch
keywords: patch
messages: 241555
nosy: christian.heimes, eli.bendersky, scoder
priority: normal
severity: normal
stage: patch review
status: open
title: Add error checks to PyInit_pyexpat()
type: behavior
versions: Python 3.4, Python 3.5
Added file: http://bugs.python.org/file39129/pyexpat.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24012>
_______________________________________

From report at bugs.python.org  Sun Apr 19 22:01:51 2015
From: report at bugs.python.org (Ben Hoyt)
Date: Sun, 19 Apr 2015 20:01:51 +0000
Subject: [New-bugs-announce] [issue24013] Improve os.scandir() and DirEntry
	documentation
Message-ID: <1429473711.62.0.571757122461.issue24013@psf.upfronthosting.co.za>


New submission from Ben Hoyt:

Victor Stinner's documentation for os.scandir and DirEntry is a great start (https://docs.python.org/dev/library/os.html#os.scandir), however there are a few mistakes in it, and a few ways I think it could be improved. Attaching a patch with the following overall changes:

1. Tweak the "see also" note on os.listdir() to mention *why* you'd want to use scandir -- namely that it includes file attribute info, and performance.
2. Move that str/bytes note in the scandir() docs down a bit, as I think that's really a detail and the other stuff is more important.
3. More details on why you'd want to use scandir -- to "increase the performance of code that also needs file type or file" -- and be more specific about what system calls are and are not required.
4. Replace "POSIX" with "Unix", per the rest of the os module docs when talking about generic Unix-like operating systems. It seems "POSIX" is used specifically when talking about the POSIX standard.
5. Remove half-true statement in DirEntry docs, "if a file is deleted between calling scandir and stat, a FileNotFoundError can be raised" -- but the method docs state that they specifically don't raise FileNotFoundError.
6. Removed "unfortunately, the behaviour of errors depends on the platform". This is always the case and doesn't add anything.
7. Tried to simplify and clarify the is_dir() and is_file() docs. Not sure I've succeeded here. This is hard!
8. Added "Availability: Unix, Windows." to scandir docs like listdir and most other os functions have.
9. Remove the see also about how "the listdir function returns the names of the directory entries" as that's already stated/implied above.

----------
assignee: docs at python
components: Documentation
files: scandir_doc_tweaks.patch
keywords: patch
messages: 241560
nosy: benhoyt, docs at python, haypo, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Improve os.scandir() and DirEntry documentation
versions: Python 3.5
Added file: http://bugs.python.org/file39131/scandir_doc_tweaks.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24013>
_______________________________________

From report at bugs.python.org  Mon Apr 20 11:09:20 2015
From: report at bugs.python.org (Joe Jevnik)
Date: Mon, 20 Apr 2015 09:09:20 +0000
Subject: [New-bugs-announce] [issue24014] Second pass of PyCode_Optimize
Message-ID: <1429520960.11.0.0723675041232.issue24014@psf.upfronthosting.co.za>


New submission from Joe Jevnik:

There are a lot of optimizations that are being missed by only running a single pass of PyCode_Optimize. I originally started by trying to optimize for De Morgan's Laws in if tests; however, I realized that the issue actually went away if you run the optimizer twice. I imagine that this would provide a lot of other performance increases because some of the patterns don't show up until the optimizer has already run once. For example:

>>> def f(a, b):
...     if not a and not b:
...         return True
...     return False
... 

On default, this generates:
>>> dis(f)
  2           0 LOAD_FAST                0 (a)
              3 UNARY_NOT
              4 POP_JUMP_IF_FALSE       18
              7 LOAD_FAST                1 (b)
             10 UNARY_NOT
             11 POP_JUMP_IF_FALSE       18

  3          14 LOAD_CONST               1 (True)
             17 RETURN_VALUE

  4     >>   18 LOAD_CONST               2 (False)
             21 RETURN_VALUE


If we run the optimizer again, we can collapse some of the UNARY_NOT -> POP_JUMP_IF_FALSE back into POP_JUMP_IF_TRUE, like this:
>>> dis(f)
  2           0 LOAD_FAST                0 (a)
              3 POP_JUMP_IF_TRUE        16
              6 LOAD_FAST                1 (b)
              9 POP_JUMP_IF_TRUE        16

  3          12 LOAD_CONST               1 (True)
             15 RETURN_VALUE

  4     >>   16 LOAD_CONST               2 (False)
             19 RETURN_VALUE


Also, Python/importlib.h blew up in the diff; should this be included in the patch?

----------
components: Interpreter Core
files: secondpass.patch
keywords: patch
messages: 241626
nosy: llllllllll
priority: normal
severity: normal
status: open
title: Second pass of PyCode_Optimize
type: performance
versions: Python 3.5
Added file: http://bugs.python.org/file39144/secondpass.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24014>
_______________________________________

From report at bugs.python.org  Mon Apr 20 15:28:04 2015
From: report at bugs.python.org (Joachim Breitner)
Date: Mon, 20 Apr 2015 13:28:04 +0000
Subject: [New-bugs-announce] [issue24015] timeit should start with 1 loop,
	not 10
Message-ID: <1429536484.03.0.469371276206.issue24015@psf.upfronthosting.co.za>


New submission from Joachim Breitner:

The docs for the timeit command line interface specify

If -n is not given, a suitable number of loops is calculated by trying successive powers of 10 until the total time is at least 0.2 seconds.

This sounds as if it it first tries 1, then 10, then 100 etc. But the code starts with 10 iterations. So even if the tested code already takes long enough (e.g. because it is a suitable loop itself), timit will by default test 10 loops.

I propose to change that, and replace

        # determine number so that 0.2 <= total time < 2.0
        for i in range(1, 10):
            number = 10**i

with

        # determine number so that 0.2 <= total time < 2.0
        for i in range(0, 10):
            number = 10**i

in Lib/timeit.py.

----------
components: Library (Lib)
messages: 241643
nosy: nomeata
priority: normal
severity: normal
status: open
title: timeit should start with 1 loop, not 10
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24015>
_______________________________________

From report at bugs.python.org  Mon Apr 20 18:02:16 2015
From: report at bugs.python.org (Carol Willing)
Date: Mon, 20 Apr 2015 16:02:16 +0000
Subject: [New-bugs-announce] [issue24016] Add a Sprints
	organization/preparation section to	devguide
Message-ID: <1429545736.46.0.134603387564.issue24016@psf.upfronthosting.co.za>


New submission from Carol Willing:

Based on feedback from the PyCon 2015 CPython sprints (thank you R. David Murray and all the CPython contributors there), a sprint organization section will be added to the devguide.

Here are some suggested subtopics in the Sprints organization section:
* Pre-sprint preparation (by core devs and those familiar with CPython devel)
  - Etherpad for issues and links to resources
  - Dev in a box environment preparation
  - Docker container for development (based on work done by Saul)
* At the sprint
  - Important info posted in room (IRC; etherpad; schedule)
  - Welcome talk as given by R. David
  - Introduction of core devs in attendance
  - Periodic announcements to encourage questions
* After the sprint
  - Capture feedback from the etherpad
  - Refine devguide as needed

Please suggest additional topics. Thanks!

----------
assignee: willingc
components: Devguide
messages: 241666
nosy: ezio.melotti, willingc
priority: normal
severity: normal
stage: needs patch
status: open
title: Add a Sprints organization/preparation section to devguide
type: enhancement

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24016>
_______________________________________

From report at bugs.python.org  Mon Apr 20 21:33:55 2015
From: report at bugs.python.org (STINNER Victor)
Date: Mon, 20 Apr 2015 19:33:55 +0000
Subject: [New-bugs-announce] [issue24017] Implemenation of the PEP 492 -
	Coroutines with async and	await syntax
Message-ID: <1429558435.86.0.159043579663.issue24017@psf.upfronthosting.co.za>


Changes by STINNER Victor <victor.stinner at gmail.com>:


----------
nosy: haypo
priority: normal
severity: normal
status: open
title: Implemenation of the PEP 492 - Coroutines with async and await syntax
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24017>
_______________________________________

From report at bugs.python.org  Mon Apr 20 21:36:25 2015
From: report at bugs.python.org (Stefan Behnel)
Date: Mon, 20 Apr 2015 19:36:25 +0000
Subject: [New-bugs-announce] [issue24018] add a Generator ABC
Message-ID: <1429558585.89.0.729845325991.issue24018@psf.upfronthosting.co.za>


New submission from Stefan Behnel:

Currently, CPython tends to assume that generators are always implemented by a Python function that uses the "yield" keyword. However, it is absolutely possible to implement generators as a protocol by adding send(), throw() and close() methods to an iterator.

The problem is that these will usually be rejected by code that needs to distinguish generators from other input, e.g. in asyncio, as this code will commonly do a type check against types.GeneratorType. Instead, it should check for the expected protocol. The best way to achieve this is to extend the existing ABCs with a Generator ABC that external generator implementations can register on.

Related to issue 24004 (asyncio). Asyncio could provide a backport copy of this for older Python versions.

I assume this is considered a new feature that cannot be added to 3.4 anymore?

----------
components: Library (Lib)
files: generator_abc.patch
keywords: patch
messages: 241675
nosy: gvanrossum, haypo, scoder, yselivanov
priority: normal
severity: normal
status: open
title: add a Generator ABC
type: enhancement
versions: Python 3.5
Added file: http://bugs.python.org/file39146/generator_abc.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24018>
_______________________________________

From report at bugs.python.org  Tue Apr 21 06:57:04 2015
From: report at bugs.python.org (Mahmoud Hashemi)
Date: Tue, 21 Apr 2015 04:57:04 +0000
Subject: [New-bugs-announce] [issue24019] str/unicode encoding kwarg causes
	exceptions
Message-ID: <1429592224.08.0.0993607651074.issue24019@psf.upfronthosting.co.za>


New submission from Mahmoud Hashemi:

The encoding keyword argument to the Python 3 str() and Python 2 unicode() constructors is excessively constraining to the practical use of these core types.

Looking at common usage, both these constructors' primary mode is to convert various objects into text:

>>> str(2)
'2'

But adding an encoding yields:

>>> str(2, encoding='utf8')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: coercing to str: need bytes, bytearray or buffer-like object, int found

While the error message is fine for an experienced developer, I would like to raise the question: is it necessary at all? Even harmlessly getting a str from a str is punished, but leaving off encoding is fine again:

>>> str('hi', encoding='utf8')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: decoding str is not supported
>>> str('hi')
'hi'

Merging and simplifying the two modes of these constructors would yield much more predictable results for experienced and beginning Pythonists alike. Basically, the encoding argument should be ignored if the argument is already a unicode/str instance, or if it is a non-string object. It should only be consulted if the primary argument is a bytestring. Bytestrings already have a .decode() method on them, another, obscurer version of it isn't necessary.

Furthermore, despite the core nature and widespread usage of these types, changing this behavior should break very little existing code and understanding. unicode() and str() will simply behave as expected more often, returning text versions of the arguments passed to them. 

Appendix: To demonstrate the expected behavior of the proposed unicode/str, here is a code snippet we've employed to sanely and safely get a text version of an arbitrary object:

def to_unicode(obj, encoding='utf8', errors='strict'):
    # the encoding default should look at sys's value
    try:
        return unicode(obj)
    except UnicodeDecodeError:
        return unicode(obj, encoding=encoding, errors=errors)

After many years of writing Python and teaching it to developers of all experience levels, I firmly believe that this is the right interaction pattern for Python's core text type. I'm also happy to expand on this issue, turn it into a PEP, or submit a patch if there is interest.

----------
components: Unicode
messages: 241699
nosy: ezio.melotti, haypo, mahmoud
priority: normal
severity: normal
status: open
title: str/unicode encoding kwarg causes exceptions
type: behavior
versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24019>
_______________________________________

From report at bugs.python.org  Tue Apr 21 15:42:25 2015
From: report at bugs.python.org (Ethan Furman)
Date: Tue, 21 Apr 2015 13:42:25 +0000
Subject: [New-bugs-announce] [issue24020] threading.local() must be run at
	module level (doc	improvement)
Message-ID: <1429623745.88.0.884525571571.issue24020@psf.upfronthosting.co.za>


New submission from Ethan Furman:

In order to work correctly, threading.local() must be run in global scope, yet that tidbit is missing from both the docs and the _threading_local.py file.

Something like:

.. note::
   threading.local() must be run at global scope to function properly.

That would have saved me hours of time.  Thank goodness for SO!  ;)

----------
assignee: docs at python
messages: 241713
nosy: docs at python, ethan.furman
priority: normal
severity: normal
status: open
title: threading.local() must be run at module level (doc improvement)
versions: Python 2.7, Python 3.4, Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24020>
_______________________________________

From report at bugs.python.org  Tue Apr 21 17:43:36 2015
From: report at bugs.python.org (Karl Richter)
Date: Tue, 21 Apr 2015 15:43:36 +0000
Subject: [New-bugs-announce] [issue24021] document urllib.urlretrieve
Message-ID: <1429631016.67.0.905121028412.issue24021@psf.upfronthosting.co.za>


Changes by Karl Richter <krichter722 at aol.de>:


----------
assignee: docs at python
components: Documentation
nosy: docs at python, krichter
priority: normal
severity: normal
status: open
title: document urllib.urlretrieve
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24021>
_______________________________________

From report at bugs.python.org  Tue Apr 21 18:04:01 2015
From: report at bugs.python.org (Benjamin Peterson)
Date: Tue, 21 Apr 2015 16:04:01 +0000
Subject: [New-bugs-announce] [issue24022] Python heap corruption issue
Message-ID: <1429632241.9.0.734206220056.issue24022@psf.upfronthosting.co.za>


New submission from Benjamin Peterson:

Reported by "Hug Bounter" to security@

Hello,

I would like to report a heap corruption issue in Python/Parser/tokenizer.c:922, affecting latest Python 3.4.3 (from python.org) and also 2.7 ( tested 2.7.9-r1 on Gentoo ). The latest version available - 3.5.0a3 is also affected. It doesn't seem to affect 3.3 branch (tested with 3.3.5-r1 on Gentoo).
The issue occurs when a malformed python script is executed by python binary, which results in a out-of-bound read access of heap and therefore a segmentation fault.
I couldn't confirm nor deny its exploitability, to my knowledge this would be more of a infoleak, if anything. Nevertheless, as Google Project Zero proved many times, no heap corruption issue should be treated lightheartedly. :-) Hence the reason why I'm reporting it to security at python.org

I tried to dig into the details of the bug and I have to admit the defeat - the Python Parser is quite a complex beast...
What I was able to determine was that given malformed script (attached), the infinite 'for' loop defined in tokenizer.c:900 never reaches any of the exit conditions, which causes a infinite incrementation of *tok->cur and thus reading character by character of the heap, until the heap segment boundary is reached and segmentation fault occurrs.

There seem to be a race condition involved as well, as the malformed script does not always result in crash sometimes producing the error below:

./python ~/Fuzz/crashes/python_stuff/heap_pattern.py
  File "/home/user/Fuzz/crashes/python_stuff/heap_pattern.py", line 44
SyntaxError: Non-UTF-8 code starting with '\x9e' in file /home/user/Fuzz/crashes/python_stuff/heap_pattern.py on line 45, but no encoding declared; see http://python.org/dev/peps/pep-0263/ for details

I acknowledge that attack scenario is somehow limited, because one has to be in a position to provide their own script for execution. Nevertheless, at the very least, a malicious user could crash python environment.


Depending on the particular script, ASAN detects either as a 'heap-use-after-free' or 'heap-buffer-overflow'.

HEAP-BUFFER-OVERFLOW according to asan:

$ ./python ~/heap3.py
=================================================================
==23461==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x62500001e0ff at pc 0xc90075 bp 0x7ffe53018fd0 sp 0x7ffe53018fc0
READ of size 1 at 0x62500001e0ff thread T0
    #0 0xc90074 in tok_nextc Parser/tokenizer.c:1021
    #1 0xc9a6ef in tok_get Parser/tokenizer.c:1341
    #2 0xca0640 in PyTokenizer_Get Parser/tokenizer.c:1738
    #3 0xc81109 in parsetok Parser/parsetok.c:208
    #4 0xa0c449 in PyParser_ASTFromFileObject Python/pythonrun.c:2356
    #5 0xa0c449 in PyRun_FileExFlags Python/pythonrun.c:2126
    #6 0xa15f0b in PyRun_SimpleFileExFlags Python/pythonrun.c:1606
    #7 0x43a1aa in run_file Modules/main.c:319
    #8 0x43a1aa in Py_Main Modules/main.c:751
    #9 0x4234d3 in main Modules/python.c:69
    #10 0x7efcd1cf1f9f in __libc_start_main (/lib64/libc.so.6+0x1ff9f)
    #11 0x426a7c (/home/user/Fuzz/targets/Python-3.4.3_ASAN/python+0x426a7c)

0x62500001e0ff is located 1 bytes to the left of 8192-byte region [0x62500001e100,0x625000020100)
allocated by thread T0 here:
    #0 0x7efcd29eb7c7 in malloc (/usr/lib/gcc/x86_64-pc-linux-gnu/4.9.2/libasan.so.1+0x577c7)
    #1 0xc9997a in PyTokenizer_FromFile Parser/tokenizer.c:852

SUMMARY: AddressSanitizer: heap-buffer-overflow Parser/tokenizer.c:1021 tok_nextc
Shadow bytes around the buggy address:
  0x0c4a7fffbbc0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c4a7fffbbd0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c4a7fffbbe0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c4a7fffbbf0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c4a7fffbc00: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
=>0x0c4a7fffbc10: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa[fa]
  0x0c4a7fffbc20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c4a7fffbc30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c4a7fffbc40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c4a7fffbc50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c4a7fffbc60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07
  Heap left redzone:       fa
  Heap right redzone:      fb
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack partial redzone:   f4
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Contiguous container OOB:fc
  ASan internal:           fe
==23461==ABORTING


Below is an example of ASAN detecting a 'use-after-free':

./python ~/heap4_asan.py
=================================================================
==23465==ERROR: AddressSanitizer: heap-use-after-free on address 0x62500001e101 at pc 0xc8f7c4 bp 0x7ffc35552000 sp 0x7ffc35551ff0
READ of size 1 at 0x62500001e101 thread T0
    #0 0xc8f7c3 in tok_nextc Parser/tokenizer.c:902
    #1 0xc9a96f in tok_get Parser/tokenizer.c:1429
    #2 0xca0640 in PyTokenizer_Get Parser/tokenizer.c:1738
    #3 0xc81109 in parsetok Parser/parsetok.c:208
    #4 0xa0c449 in PyParser_ASTFromFileObject Python/pythonrun.c:2356
    #5 0xa0c449 in PyRun_FileExFlags Python/pythonrun.c:2126
    #6 0xa15f0b in PyRun_SimpleFileExFlags Python/pythonrun.c:1606
    #7 0x43a1aa in run_file Modules/main.c:319
    #8 0x43a1aa in Py_Main Modules/main.c:751
    #9 0x4234d3 in main Modules/python.c:69
    #10 0x7f71d129ef9f in __libc_start_main (/lib64/libc.so.6+0x1ff9f)
    #11 0x426a7c (/home/user/Fuzz/targets/Python-3.4.3_ASAN/python+0x426a7c)

0x62500001e101 is located 1 bytes inside of 8192-byte region [0x62500001e100,0x625000020100)
freed by thread T0 here:
    #0 0x7f71d1f98aa6 in __interceptor_realloc (/usr/lib/gcc/x86_64-pc-linux-gnu/4.9.2/libasan.so.1+0x57aa6)
    #1 0xc8edb1 in tok_nextc Parser/tokenizer.c:1041

previously allocated by thread T0 here:
    #0 0x7f71d1f987c7 in malloc (/usr/lib/gcc/x86_64-pc-linux-gnu/4.9.2/libasan.so.1+0x577c7)
    #1 0xc9997a in PyTokenizer_FromFile Parser/tokenizer.c:852

SUMMARY: AddressSanitizer: heap-use-after-free Parser/tokenizer.c:902 tok_nextc
Shadow bytes around the buggy address:
  0x0c4a7fffbbd0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c4a7fffbbe0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c4a7fffbbf0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c4a7fffbc00: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c4a7fffbc10: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
=>0x0c4a7fffbc20:[fd]fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c4a7fffbc30: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c4a7fffbc40: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c4a7fffbc50: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c4a7fffbc60: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c4a7fffbc70: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07
  Heap left redzone:       fa
  Heap right redzone:      fb
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack partial redzone:   f4
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Contiguous container OOB:fc
  ASan internal:           fe
==23465==ABORTING

Without AddressSanitizer, this particular script does not crash, but causes one of two errors:

File "/home/user/heap4_asan.py", line 5
SyntaxError: Non-UTF-8 code starting with '\x9e' in file /home/user/heap4_asan.py on line 6, but no encoding declared; see http://python.org/dev/peps/pep-0263/ for details

or:

File "/home/user/heap4_asan.py", line 5
SyntaxError: unknown decode error


In all cases, the crash always occurs in Parser/tokenizer.c at line no. 922, where *tok->curr is incremented, regardless where it currently points. Eventually, it will reach heap boundary and the *tok->cur++ will cause python to crash.

Program received signal SIGSEGV, Segmentation fault.
0x0000000000573657 in tok_nextc (tok=tok at entry=0x8fb250) at Parser/tokenizer.c:922
922                 return Py_CHARMASK(*tok->cur++);


Sample GDB session can be found below:

$ gdb --args ./python ~/heap1.py
GNU gdb (Gentoo 7.9 vanilla) 7.9
Copyright (C) 2015 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-pc-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://bugs.gentoo.org/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ./python...done.
warning: File "/home/user/Fuzz/targets/Python-3.4.3/python-gdb.py" auto-loading has been declined by your `auto-load safe-path' set to "$debugdir:$datadir/auto-load".
To enable execution of this file add
    add-auto-load-safe-path /home/user/Fuzz/targets/Python-3.4.3/python-gdb.py
line to your configuration file "/home/user/.gdbinit".
To completely disable this security protection add
    set auto-load safe-path /
line to your configuration file "/home/user/.gdbinit".
For more information about this security protection see the
"Auto-loading safe path" section in the GDB manual.  E.g., run from the shell:
    info "(gdb)Auto-loading safe path"
gdb-peda$ r
Starting program: /home/user/Fuzz/targets/Python-3.4.3/python /home/user/heap1.py
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".

Program received signal SIGSEGV, Segmentation fault.
[----------------------------------registers-----------------------------------]
RAX: 0x995001
RBX: 0x963c40 --> 0x0
RCX: 0x0
RDX: 0x27 ("'")
RSI: 0x0
RDI: 0x963c40 --> 0x0
RBP: 0x0
RSP: 0x7fffffffdf40 --> 0x7ffff6f14660 --> 0x0
RIP: 0x573657 (<tok_nextc+1367>:    movzx  eax,BYTE PTR [r12])
R8 : 0x1bdf0
R9 : 0x1bde0
R10: 0x1bdd0
R11: 0x4
R12: 0x995000
R13: 0x0
R14: 0x7fffffffe010 --> 0x0
R15: 0x0
EFLAGS: 0x10216 (carry PARITY ADJUST zero sign trap INTERRUPT direction overflow)
[-------------------------------------code-------------------------------------]
   0x57364a <tok_nextc+1354>:    mov    QWORD PTR [rbx+0x10],rax
   0x57364e <tok_nextc+1358>:    lea    rax,[r12+0x1]
   0x573653 <tok_nextc+1363>:    mov    QWORD PTR [rbx+0x8],rax
=> 0x573657 <tok_nextc+1367>:    movzx  eax,BYTE PTR [r12]
   0x57365c <tok_nextc+1372>:    add    rsp,0x18
   0x573660 <tok_nextc+1376>:    pop    rbx
   0x573661 <tok_nextc+1377>:    pop    rbp
   0x573662 <tok_nextc+1378>:    pop    r12
[------------------------------------stack-------------------------------------]
0000| 0x7fffffffdf40 --> 0x7ffff6f14660 --> 0x0
0008| 0x7fffffffdf48 --> 0x57107e (<PyNode_AddChild+318>:    mov    rsi,rax)
0016| 0x7fffffffdf50 --> 0x7ffff6f14660 --> 0x0
0024| 0x7fffffffdf58 --> 0x27 ("'")
0032| 0x7fffffffdf60 --> 0x963c40 --> 0x0
0040| 0x7fffffffdf68 --> 0x3
0048| 0x7fffffffdf70 --> 0x0
0056| 0x7fffffffdf78 --> 0x7fffffffe010 --> 0x0
[------------------------------------------------------------------------------]
Legend: code, data, rodata, value
Stopped reason: SIGSEGV
0x0000000000573657 in tok_nextc (tok=tok at entry=0x963c40)
    at Parser/tokenizer.c:922
922                return Py_CHARMASK(*tok->cur++);


Thank you for reading this.
Please let me know if you need more information.

----------
components: Interpreter Core
messages: 241720
nosy: benjamin.peterson
priority: high
severity: normal
status: open
title: Python heap corruption issue
type: crash
versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24022>
_______________________________________

From report at bugs.python.org  Wed Apr 22 10:17:50 2015
From: report at bugs.python.org (Johan Pretorius)
Date: Wed, 22 Apr 2015 08:17:50 +0000
Subject: [New-bugs-announce] [issue24023] Django tutorial 2 not able to
	create a superuser on	Windows 7
Message-ID: <1429690670.7.0.624735318839.issue24023@psf.upfronthosting.co.za>


New submission from Johan Pretorius:

When running the $python manage.py createsuperuser command the following is returned:
C:\Users\johanP\mysite>python manage.py cre
Username (leave blank to use 'johanp'):
Email address: ja****rius at gmail.com
Traceback (most recent call last):
  File "manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "C:\Program Files\Python 3.5\lib\site-packages\djan
nit__.py", line 338, in execute_from_command_line
    utility.execute()
  File "C:\Program Files\Python 3.5\lib\site-packages\djan
nit__.py", line 330, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv
  File "C:\Program Files\Python 3.5\lib\site-packages\djan
e.py", line 390, in run_from_argv
    self.execute(*args, **cmd_options)
  File "C:\Program Files\Python 3.5\lib\site-packages\djan
ment\commands\createsuperuser.py", line 50, in execute
    return super(Command, self).execute(*args, **options)
  File "C:\Program Files\Python 3.5\lib\site-packages\djan
e.py", line 441, in execute
    output = self.handle(*args, **options)
  File "C:\Program Files\Python 3.5\lib\site-packages\djan
ment\commands\createsuperuser.py", line 124, in handle
    password = getpass.getpass()
  File "C:\Program Files\Python 3.5\lib\getpass.py", line
    msvcrt.putwch(c)
AttributeError: module 'msvcrt' has no attribute 'putwch'

The command picks up my AD credentials and doesn't request a password. I can create a super user in python using the following link:https://docs.djangoproject.com/en/1.8/topics/auth/default/
But the admin site won't start using this method.
I'm running Python 3.5, Django 1.8 on Windows 7 Professional.

----------
components: Demos and Tools
messages: 241787
nosy: JohanPretorius
priority: normal
severity: normal
status: open
title: Django tutorial 2 not able to create a superuser on Windows 7
type: compile error
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24023>
_______________________________________

From report at bugs.python.org  Wed Apr 22 15:13:46 2015
From: report at bugs.python.org (Marc-Andre Lemburg)
Date: Wed, 22 Apr 2015 13:13:46 +0000
Subject: [New-bugs-announce] [issue24024] str.__doc__ needs an update
Message-ID: <1429708426.75.0.208318155296.issue24024@psf.upfronthosting.co.za>


New submission from Marc-Andre Lemburg:

In Python 3.4.3, the doc string of the str() builtin still reads:

"str(object='') -> str\nstr(bytes_or_buffer[, encoding[, errors]]) -> str\n\nCreate a new string object from the given object. If encoding or\nerrors is specified, then the object must expose a data buffer\nthat will be decoded using the given encoding and error handler.\nOtherwise, returns the result of object.__str__() (if defined)\nor repr(object).\nencoding defaults to sys.getdefaultencoding().\nerrors defaults to 'strict'."

This no longer matches what str() does. sys.getdefaultencoding() always returns 'utf-8' and str() accepts bytes, bytearray or buffer-like objects as input when using an encoding, any object with .__str__() method defined or repr(obj) otherwise.

----------
components: Interpreter Core, Unicode
messages: 241798
nosy: ezio.melotti, haypo, lemburg
priority: normal
severity: normal
status: open
title: str.__doc__ needs an update
versions: Python 3.4, Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24024>
_______________________________________

From report at bugs.python.org  Wed Apr 22 15:23:32 2015
From: report at bugs.python.org (Marc-Andre Lemburg)
Date: Wed, 22 Apr 2015 13:23:32 +0000
Subject: [New-bugs-announce] [issue24025] str(bytes_obj) should raise an
	error
Message-ID: <1429709012.06.0.232941400287.issue24025@psf.upfronthosting.co.za>


New submission from Marc-Andre Lemburg:

In Python 2, the unicode() constructor does not accept bytes arguments, unless an encoding argument is given:

>>> unicode(u'abc???'.encode('utf-8'))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 3: ordinal not in range(128)

In Python 3, the str() constructor masks this programming error by returning the repr() of the bytes object:

>>> str('abc???'.encode('utf-8'))
"b'abc\\xc3\\xa4\\xc3\\xb6\\xc3\\xbc'"

I think it would be more helpful to point the programmer to the most probably missing encoding argument by raising an error.

Also note that you get a different output with encoding argument set:

>>> str('abc???'.encode('utf-8'), 'utf-8')
'abc???'

I know this is documented, but it is still not very helpful and can easily hide errors.

----------
components: Interpreter Core, Unicode
messages: 241800
nosy: ezio.melotti, haypo, lemburg
priority: normal
severity: normal
status: open
title: str(bytes_obj) should raise an error
versions: Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24025>
_______________________________________

From report at bugs.python.org  Wed Apr 22 16:17:44 2015
From: report at bugs.python.org (appidman)
Date: Wed, 22 Apr 2015 14:17:44 +0000
Subject: [New-bugs-announce] [issue24026] Python hangs forever in wait() of
	threading.py
Message-ID: <1429712264.48.0.476930197364.issue24026@psf.upfronthosting.co.za>


New submission from appidman:

I use paramiko to create SSH sessions to Linux machines, which works great 95% of the time. But sometimes, the Python process which creates the SSH session hangs forever in wait() of threading.py (see attachment 'gdb.txt' for full output).

{code}
(gdb) py-list
 239            waiter.acquire()
 240            self.__waiters.append(waiter)
 241            saved_state = self._release_save()
 242            try:    # restore state no matter what (e.g., KeyboardInterrupt) 
 243                if timeout is None:
>244                    waiter.acquire()
 245                    if __debug__:
 246                        self._note("%s.wait(): got it", self)
 247                else:
 248                    # Balancing act:  We can't afford a pure busy loop, so we
 249                    # have to sleep; but if we sleep the whole timeout time,


(gdb) py-bt
#5 Frame 0xb459a5c, for file /usr/lib/python2.7/threading.py, line 244, in wait (self=<_Condition(_Verbose__verbose=False, _Condition__lock=<thread.lock at remote 0xaf7e3a0>, acquire=<built-in method acquire of thread.lock object at remote 0xaf7e3a0>, _Condition__waiters=[<thread.lock at remote 0xaf7e100>], release=<built-in method release of thread.lock object at remote 0xaf7e3a0>) at remote 0xb73708c>, timeout=None, waiter=<thread.lock at remote 0xaf7e100>, saved_state=None)
    waiter.acquire()
#9 Frame 0xb66118c, for file /usr/lib/python2.7/dist-packages/paramiko/buffered_pipe.py, line 137, in read (self=<BufferedPipe(_closed=False, _lock=<thread.lock at remote 0xaf7e3a0>, _buffer=<array.array at remote 0xb737ea0>, _event=None, _cv=<_Condition(_Verbose__verbose=False, _Condition__lock=<thread.lock at remote 0xaf7e3a0>, acquire=<built-in method acquire of thread.lock object at remote 0xaf7e3a0>, _Condition__waiters=[<thread.lock at remote 0xaf7e100>], release=<built-in method release of thread.lock object at remote 0xaf7e3a0>) at remote 0xb73708c>) at remote 0xb737a4c>, nbytes=8192, timeout=None, out='', then=<float at remote 0xb57efe4>)
    self._cv.wait(timeout)
{code}

----------
components: Library (Lib)
files: gdb.txt
messages: 241806
nosy: appidman
priority: normal
severity: normal
status: open
title: Python hangs forever in wait() of threading.py
type: behavior
versions: Python 2.7
Added file: http://bugs.python.org/file39167/gdb.txt

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24026>
_______________________________________

From report at bugs.python.org  Wed Apr 22 17:19:43 2015
From: report at bugs.python.org (Pablo)
Date: Wed, 22 Apr 2015 15:19:43 +0000
Subject: [New-bugs-announce] [issue24027] IMAP library lacks documentation
	about expected	parameter types
Message-ID: <1429715983.31.0.608966950375.issue24027@psf.upfronthosting.co.za>


New submission from Pablo:

I used the IMAP library and I feel it lacks a lot of documentation regarding types and encoding.

Example:
IMAP4.list([directory[, pattern]]):
It doesn't state if it expects the directory argument to be an string (unicode) or a imap_utf7 encoded string or bytes.
It also expects names with spaces to be between double quotation marks and it doesn't state that it returns a bytes raw output.

This is valid for every function that receives a directory or mailbox argument. Also for most return values.

Documentation aside, I think it would be a nice improvement in the implementation to add an imap_utf7 convertion function, and check every directory or mailbox argument to see if it has characters outside the imap_utf7 charset and attempts to convert them automatically.

----------
assignee: docs at python
components: Documentation
messages: 241812
nosy: docs at python, pmoleri
priority: normal
severity: normal
status: open
title: IMAP library lacks documentation about expected parameter types
type: enhancement
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24027>
_______________________________________

From report at bugs.python.org  Wed Apr 22 18:58:00 2015
From: report at bugs.python.org (Terry J. Reedy)
Date: Wed, 22 Apr 2015 16:58:00 +0000
Subject: [New-bugs-announce] [issue24028] Idle: add doc subsection on
	calltips
Message-ID: <1429721880.11.0.245757615571.issue24028@psf.upfronthosting.co.za>


New submission from Terry J. Reedy:

The doc currently says, in the Edit menu section,

Show call tip
    After an unclosed parenthesis for a function, open a small window with function parameter hints.

I propose to add the additional information I gave in answer to https://stackoverflow.com/questions/29784273/python-idle-wont-show-docstring, in a subsubsection before or after the current one on completions https://docs.python.org/3/library/idle.html#completions  

'''
Calltips are shown when one types '(' after the name of an *acccessible* function.  The calltip should stay displayed until one types ')' or clicks the mouse or otherwise moves the cursor to dismiss it.  Cntl-\ brings it back.

A calltip consists of the function signature and the first line of the docstring.  For builtins without an accessible signature (such as, in 3.4.3, `int` or `bytes`), the calltip consists of all lines up the fifth line or the first blank line.

The set of *accessible* functions depends on what modules have been imported into the user process (where your code is executed), including those imported by Idle itself, and what code has been run (since the last restart).  For example, restart the Shell (Cntl-F6), open a new editor window, and enter

    itertools.count(

A calltip appears because Idle imports itertools into the user process for its own use.  Enter

    turtle.write(

and nothing appears, because the Idle does not import turtle.  Cntl-\ does nothing either.  Entering

    import turtle

above the function call does not immediately help, but if one runs the file to perform the import, calltips for turtle functions become available.

The suggests that one might want to run a file after writing the import statements at the top, or immediately run an existing file before editing.
'''

----------
assignee: terry.reedy
messages: 241815
nosy: terry.reedy
priority: normal
severity: normal
stage: needs patch
status: open
title: Idle: add doc subsection on calltips
type: enhancement
versions: Python 2.7, Python 3.4, Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24028>
_______________________________________

From report at bugs.python.org  Wed Apr 22 19:14:50 2015
From: report at bugs.python.org (Barry A. Warsaw)
Date: Wed, 22 Apr 2015 17:14:50 +0000
Subject: [New-bugs-announce] [issue24029] Surprising name binding behavior
	of submodule imports	needs documenting
Message-ID: <1429722890.23.0.504523422079.issue24029@psf.upfronthosting.co.za>


New submission from Barry A. Warsaw:

As described here: http://news.gmane.org/find-root.php?message_id=20150422115959.1ff2ee58%40limelight.wooz.org

Importing a submodule binds the submodule's name in the parent module's namespace.  This is surprising, but it seems intentional and it's relied upon by existing code, e.g. asyncio/__init__.py in the stdlib.

It's also not documented afaict.  It should be documented in the Language Reference's section on the import system.  After a little more discussion on import-sig, I plan on doing that.

----------
assignee: barry
components: Documentation
messages: 241816
nosy: barry, brett.cannon
priority: normal
severity: normal
status: open
title: Surprising name binding behavior of submodule imports needs documenting
type: behavior
versions: Python 3.4, Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24029>
_______________________________________

From report at bugs.python.org  Wed Apr 22 20:14:30 2015
From: report at bugs.python.org (Pablo)
Date: Wed, 22 Apr 2015 18:14:30 +0000
Subject: [New-bugs-announce] [issue24030] IMAP library encoding enhancement
Message-ID: <1429726470.47.0.351699535866.issue24030@psf.upfronthosting.co.za>


New submission from Pablo:

IMAP library doesn't encode parameters to the required charset.
The library is useful, but when it comes to complex mailbox names, the user needs to convert strings to and from the custom imap_utf7 encoding.
I think this conversion could be implemented in the library and applied transparently to all the arguments that need it.

Example:
IMAP4.select(mailbox='INBOX', readonly=False):
For the method to work, the mailbox argument needs to be encoded to imap_utf7 and if it has spaces it needs to be double quoted. All this hassle could be handled by the library.

The same applies to every function that uses a mailbox or directory argument.

When it comes to the mailbox argument I can identify the following cases:
a. bytes: It should be treated as an already imap_utf7 encoded string. If necessary it can be converted to string using ascii charset.
b. string:
  b.1: It's a valid imap_utf7 string without '&' -> doesn't need encoding. Eg.: INBOX
  b.2: An already encoded imap_utf7 string with '&' character -> doesn't need encoding. Eg.: Test&AOk-
  b.3: Any other case (invalid imap_utf7 string) -> needs to be encoded

Proposal:
1. Impelement an imap_utf7_encode() method
2. Implement a strict imap_utf7_decode() method, it must return an error if the input doesn't conform to imap_utf7 encoding.
3. Implement a method to ensure arguments are in imap_utf7 encoding:
   * bytes -> arg.decode('ascii')
   * string && imap_utf7_decode(arg) -> arg
   * otherwise -> imap_utf7_encode(arg)
   * In every case if it has spaces double quote the whole string
5. In every method that receives a mailbox or directory argument call this new method to ensure it's imap_utf7 encoded.

----------
components: Library (Lib), email
messages: 241821
nosy: barry, pmoleri, r.david.murray
priority: normal
severity: normal
status: open
title: IMAP library encoding enhancement
type: enhancement
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24030>
_______________________________________

From report at bugs.python.org  Wed Apr 22 22:29:35 2015
From: report at bugs.python.org (Christian Heimes)
Date: Wed, 22 Apr 2015 20:29:35 +0000
Subject: [New-bugs-announce] [issue24031] Add git support to make patchcheck
Message-ID: <1429734575.53.0.935403070835.issue24031@psf.upfronthosting.co.za>


New submission from Christian Heimes:

'make patchcheck' supports only Mercurial checkouts. The tool uses the 'hg status' command to find modified files. My patch implements 'git status' support in order to make the patch check tool available on git checkouts.

----------
components: Build
files: patchcheck_git.patch
keywords: patch
messages: 241824
nosy: christian.heimes
priority: normal
severity: normal
stage: patch review
status: open
title: Add git support to make patchcheck
type: enhancement
versions: Python 3.4, Python 3.5
Added file: http://bugs.python.org/file39174/patchcheck_git.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24031>
_______________________________________

From report at bugs.python.org  Wed Apr 22 23:17:03 2015
From: report at bugs.python.org (albertsmuktupavels)
Date: Wed, 22 Apr 2015 21:17:03 +0000
Subject: [New-bugs-announce] [issue24032] urlparse.urljoin does not add
	query part
Message-ID: <1429737423.73.0.596485063186.issue24032@psf.upfronthosting.co.za>


New submission from albertsmuktupavels:

>From documentation:
"Construct a full (?absolute?) URL by combining a ?base URL? (base) with another URL (url). Informally, this uses components of the base URL, in particular the addressing scheme, the network location and (part of) the path, to provide missing components in the relative URL."

base = http://www.example.com/example/foo.php?param=10
url = bar.php

I am expecting this result:
http://www.example.com/example/bar.php?param=10

But real result is:
http://www.example.com/example/bar.php

This should be easy fixable. Right now query= bquery is done only in one case - when path and params are empty.

I think that
if not query:
    query = bquery
should be moved before query might be used for first time. that means above code should be right after these lines:
if scheme != bscheme or scheme not in uses_relative:
    return url

----------
messages: 241827
nosy: albertsmuktupavels
priority: normal
severity: normal
status: open
title: urlparse.urljoin does not add query part
type: behavior
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24032>
_______________________________________

From report at bugs.python.org  Thu Apr 23 02:26:47 2015
From: report at bugs.python.org (Christie)
Date: Thu, 23 Apr 2015 00:26:47 +0000
Subject: [New-bugs-announce] [issue24033] Update _test_multiprocessing.py to
	use script helpers
Message-ID: <1429748807.96.0.769250379827.issue24033@psf.upfronthosting.co.za>


New submission from Christie:

As described in Issue9517, many test modules do not make use of the helpers in script_helpers.py to invoke the python interpreter in a subprocess. Issue9517 will be broken down into several smaller issues so we can address smaller change sets.

This issue is to update _test_multiprocessing.py to use script_helpers.py.

----------
components: Tests
messages: 241837
nosy: bobcatfish
priority: normal
severity: normal
status: open
title: Update _test_multiprocessing.py to use script helpers
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24033>
_______________________________________

From report at bugs.python.org  Thu Apr 23 06:47:10 2015
From: report at bugs.python.org (Masayuki Yamamoto)
Date: Thu, 23 Apr 2015 04:47:10 +0000
Subject: [New-bugs-announce] [issue24034] Make fails Objects/typeslots.inc
Message-ID: <1429764430.06.0.333500600321.issue24034@psf.upfronthosting.co.za>


New submission from Masayuki Yamamoto:

Objects/typeslots.inc is broken, and make stop on cygwin-1.7.35-i686.

If built-in python is not found in make processing, Objects/typeslots.inc generating will fail.
Because Objects/typeslots.inc is generated using $(PYTHON) in Makefile, but  $(PYTHON) variable is a hard-coding value that isn't check a built-in python existent.

make steps to fail and error message:
$ ./configure --prefix=/opt/py34
$ touch Include/typeslots.h
$ make
...
python.exe ./Objects/typeslots.py < ./Include/typeslots.h > Objects/typeslots.inc
/bin/sh: python.exe: command not found
Makefile:851: recipe for target 'Objects/typeslots.inc' failed
make: *** [Objects/typeslots.inc] Error 127

Cygwin python executable files:
$ find /usr/bin/ -name 'python*.exe'
/usr/bin/python2.7.exe
/usr/bin/python3.2m.exe

----------
components: Build, Interpreter Core
messages: 241843
nosy: masamoto
priority: normal
severity: normal
status: open
title: Make fails Objects/typeslots.inc
type: compile error
versions: Python 3.4, Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24034>
_______________________________________

From report at bugs.python.org  Thu Apr 23 07:05:31 2015
From: report at bugs.python.org (James)
Date: Thu, 23 Apr 2015 05:05:31 +0000
Subject: [New-bugs-announce] [issue24035] When Caps Locked,
	<Shift> + alpha-character still displayed as uppercase
Message-ID: <1429765531.25.0.768760341138.issue24035@psf.upfronthosting.co.za>


New submission from James:

Referring to Python 2.7 running on Windows (7/8): 

At the interactive interpreter, if either 1) Caps are Locked OR 2) <Shift> is held while an alpha-character is selected, the character is output and displayed as uppercase, as one would expect. However, in Python 2.7, when Caps are Locked AND <Shift> is held while an alpha-character is selected, the output is still uppercase, when it should be lowercase.

This behavior seems to be limited to Python 2(.7), and only when Python is run interactively using the Windows console. When Python 2.7 is started through Cygwin using mintty on Windows 7/8, the behavior is not reproducible. The same goes for IDLE, as well as when running python interactively on Ubuntu. The behavior is reproducible using IPython when run in the standard Windows console, but is not reproducible when Python is run using the IPython QtConsole.

To summarize:

-----------------------------       -------------------------------
AFFECTED                    |      |NOT AFFECTED                 |
------------------------------      -------------------------------
python 2 via cmd (Windows 7/8)     python 3 via cmd    (Windows 7/8)
ipython via cmd  (Windows 7/8)     IDLE python 2 & 3   (Windows 7/8)         
                                   python 2 via mintty (Windows 7/8)
                                   python 2 via bash  (Ubuntu 14.04)
                                   ipython qtconsole   (Windows 7/8)

----------
components: IO
messages: 241845
nosy: principia1687
priority: normal
severity: normal
status: open
title: When Caps Locked, <Shift> + alpha-character still displayed as uppercase
type: behavior
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24035>
_______________________________________

From report at bugs.python.org  Thu Apr 23 11:29:41 2015
From: report at bugs.python.org (Ma Lin)
Date: Thu, 23 Apr 2015 09:29:41 +0000
Subject: [New-bugs-announce] [issue24036] GB2312 codec is using a wrong
	covert table
Message-ID: <1429781381.54.0.113233811407.issue24036@psf.upfronthosting.co.za>


New submission from Ma Lin:

While I was trying to optimize GB2312/GBK/GB18030-2000 codecs (they are three encodings that widely used in China), I found there is a bug.

The three encodings, their relation should be: GB2312 ? GBK ? GB18030-2000.
However, in Python's implement: GB2312 ? GBK ? GB18030-2000.
GBK should be backward compatible with GB2312, but in Python's implement, it's not.

----
I digged into, I found the Python's GB2312 codec is using a wrong convert table.
In this file /Modules/cjkcodecs/_codecs_cn.c , there is a comment block, I paste it here: 

/* GBK and GB2312 map differently in few code points that are listed below:
 *
 *              gb2312                          gbk
 * A1A4         U+30FB KATAKANA MIDDLE DOT      U+00B7 MIDDLE DOT
 * A1AA         U+2015 HORIZONTAL BAR           U+2014 EM DASH
 * A844         undefined                       U+2015 HORIZONTAL BAR
 */
 
 In fact the second column (GB2312 column) is wrong, this column should be deleted.
 
 The four involved unicode codepoints are:
 U+30FB  ?     KATAKANA MIDDLE DOT
 U+00B7  ?      MIDDLE DOT
 U+2015  ?      HORIZONTAL BAR
 U+2014  ?      EM DASH

So, GB2312 codec decodes b'0xA1, 0xA4' to U+30FB.
U+30FB is a Japanese symbol, but looks quite similar to U+00B7.

I searched "GB2312 Unicode Table" with Google, there are right verson and wrong version on the Internet, unfortunately we are using the wrong verson.

libiconv-1.14 is also using the wrong version.

----
Hold an example of bad behavior. 

Using GBK encoder, encode U+30FB to bytes, UnicodeEncodeError exception occurred, becase U+30FB is not in GBK.

In Simplified Chinese version of Microsoft Windows, console's default encoding is GBK[1].
If using GB2312 decoder to decode b'0xA1, 0xA4', then print U+30FB to console, UnicodeEncodeError raised.
Since DASH is a common character, this bug is annoying.

----
If we fix this, I don't know how many disasters will happen.
However, if we don't fix this, it's a bug.

I already made a patch, but I think we need a discussion, should we fix this?

-----------------------
Annotate:
[1] In fact console's default encoding is cp936, cp936 almost same as GBK, but not entirely same. Using GBK in here is not a problem.

----------
components: Unicode
files: fixgb2312.patch
keywords: patch
messages: 241858
nosy: Ma Lin, ezio.melotti, haypo
priority: normal
severity: normal
status: open
title: GB2312 codec is using a wrong covert table
type: behavior
versions: Python 3.5, Python 3.6
Added file: http://bugs.python.org/file39182/fixgb2312.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24036>
_______________________________________

From report at bugs.python.org  Thu Apr 23 12:33:08 2015
From: report at bugs.python.org (Serhiy Storchaka)
Date: Thu, 23 Apr 2015 10:33:08 +0000
Subject: [New-bugs-announce] [issue24037] Argument Clinic: add the boolint
	converter
Message-ID: <1429785188.86.0.806042190219.issue24037@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka:

The 'i' format unit is often used for semantically boolean values. When the parameter has a default value, in Argument clinic you should specify it twice, as Python and C values:

    closefd: int(c_default="1") = True

or

    keepends: int(py_default="False") = 0

Proposed patch adds the boolint converter which makes a conversion for you.

    closefd: boolint = True
    keepends: boolint = False

----------
components: Argument Clinic
files: clinic_boolint_converter.patch
keywords: patch
messages: 241860
nosy: larry, serhiy.storchaka
priority: normal
severity: normal
stage: patch review
status: open
title: Argument Clinic: add the boolint converter
type: enhancement
versions: Python 3.5
Added file: http://bugs.python.org/file39183/clinic_boolint_converter.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24037>
_______________________________________

From report at bugs.python.org  Thu Apr 23 16:22:58 2015
From: report at bugs.python.org (Christian Heimes)
Date: Thu, 23 Apr 2015 14:22:58 +0000
Subject: [New-bugs-announce] [issue24038] Missing cleanup in list.sort()
	with key function
Message-ID: <1429798978.55.0.726391099196.issue24038@psf.upfronthosting.co.za>


New submission from Christian Heimes:

One error path in list.sort() doesn't do proper cleanup. It neither sets an exception nor resets the internal pointers to its former values. In case of failed malloc() it leaves the list object in an invalid state.

----------
components: Interpreter Core
files: list_keyfunc.patch
keywords: patch
messages: 241865
nosy: christian.heimes
priority: normal
severity: normal
stage: patch review
status: open
title: Missing cleanup in list.sort() with key function
type: behavior
versions: Python 3.3, Python 3.4, Python 3.5
Added file: http://bugs.python.org/file39184/list_keyfunc.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24038>
_______________________________________

From report at bugs.python.org  Thu Apr 23 19:48:57 2015
From: report at bugs.python.org (Prince)
Date: Thu, 23 Apr 2015 17:48:57 +0000
Subject: [New-bugs-announce] [issue24039] Minimize option doesn't work on
	Search Dialog box for	idle
Message-ID: <1429811337.7.0.183903934829.issue24039@psf.upfronthosting.co.za>


New submission from Prince:

I tried to search for a keyword in idle with help of search dialog box, it worked just fine however when I tried to minimize the box it didn't minimize. All other options like expand and close works as expected.

Steps To Reproduce:
1) open idle
2) press cntrl+f to open search dialog box.
3) press minimize button on the box.

expected - It should minimize
Actual - it doesn't
 
Tested on windows7-64bit

----------
components: IDLE
messages: 241871
nosy: prince09cs
priority: normal
severity: normal
status: open
title: Minimize option doesn't work on Search Dialog box for idle
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24039>
_______________________________________

From report at bugs.python.org  Thu Apr 23 20:35:08 2015
From: report at bugs.python.org (Behdad Esfahbod)
Date: Thu, 23 Apr 2015 18:35:08 +0000
Subject: [New-bugs-announce] [issue24040] plistlib assumes dict_type is
	descendent of dict
Message-ID: <1429814108.42.0.103611722711.issue24040@psf.upfronthosting.co.za>


New submission from Behdad Esfahbod:

Please replace instances of type({}) in plistlib.py with self._dict_type.

----------
components: Library (Lib)
messages: 241875
nosy: Behdad.Esfahbod
priority: normal
severity: normal
status: open
title: plistlib assumes dict_type is descendent of dict
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24040>
_______________________________________

From report at bugs.python.org  Thu Apr 23 20:52:10 2015
From: report at bugs.python.org (Behdad Esfahbod)
Date: Thu, 23 Apr 2015 18:52:10 +0000
Subject: [New-bugs-announce] [issue24041] Implement Mac East Asian encodings
	properly
Message-ID: <1429815130.87.0.76039167858.issue24041@psf.upfronthosting.co.za>


New submission from Behdad Esfahbod:

encodings.aliases has this in it's tail, even master today [0]

    # temporary mac CJK aliases, will be replaced by proper codecs in 3.1
    'x_mac_japanese'      : 'shift_jis',
    'x_mac_korean'        : 'euc_kr',
    'x_mac_simp_chinese'  : 'gb2312',
    'x_mac_trad_chinese'  : 'big5',

A full implementation is appreciated.

[0] https://github.com/python/cpython/blob/master/Lib/encodings/aliases.py#L539

----------
components: Library (Lib)
messages: 241876
nosy: Behdad.Esfahbod
priority: normal
severity: normal
status: open
title: Implement Mac East Asian encodings properly

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24041>
_______________________________________

From report at bugs.python.org  Thu Apr 23 21:01:02 2015
From: report at bugs.python.org (Serhiy Storchaka)
Date: Thu, 23 Apr 2015 19:01:02 +0000
Subject: [New-bugs-announce] [issue24042] Convert os._getfullpathname() and
	os._isdir() to	Argument Clinic
Message-ID: <1429815662.86.0.935887508266.issue24042@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka:

Proposed patch converts os._getfullpathname() and os._isdir() to Argument Clinic. This simplifies a code and as a sided effect adds a check for embedded null character in os._getfullpathname(). The patch is not tested and can contain bugs.

----------
components: Argument Clinic, Windows
messages: 241878
nosy: larry, loewis, serhiy.storchaka, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
stage: patch review
status: open
title: Convert os._getfullpathname() and os._isdir() to Argument Clinic
type: enhancement
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24042>
_______________________________________

From report at bugs.python.org  Thu Apr 23 21:02:04 2015
From: report at bugs.python.org (Behdad Esfahbod)
Date: Thu, 23 Apr 2015 19:02:04 +0000
Subject: [New-bugs-announce] [issue24043] Implement mac_romanian and
	mac_croatian encodings
Message-ID: <1429815724.46.0.756339237301.issue24043@psf.upfronthosting.co.za>


New submission from Behdad Esfahbod:

They are used in OpenType fonts, but not implemented by Python at this time.  Here's are the Unicode mappings for them:

http://unicode.org/Public/MAPPINGS/VENDORS/APPLE/ROMANIAN.TXT
http://unicode.org/Public/MAPPINGS/VENDORS/APPLE/CROATIAN.TXT

Thanks.

----------
components: Library (Lib)
messages: 241879
nosy: Behdad.Esfahbod
priority: normal
severity: normal
status: open
title: Implement mac_romanian and mac_croatian encodings
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24043>
_______________________________________

From report at bugs.python.org  Thu Apr 23 23:03:08 2015
From: report at bugs.python.org (Benjamin Peterson)
Date: Thu, 23 Apr 2015 21:03:08 +0000
Subject: [New-bugs-announce] [issue24044] NULL pointer dereference in
	listsort() with key function
Message-ID: <1429822988.7.0.684416288083.issue24044@psf.upfronthosting.co.za>


New submission from Benjamin Peterson:

Found by Christian Heimes:

Coverity has found a flaw in Objects/listobject.c:listsort() that
eventually leads to a NULL pointer dereference. Because NULL pointer
dereferences can lead to exploits or DoS vulnerabilities I'm reporting
the error on PSRT first. The error is on a code path that can be
triggered by a remote attacker, although not that easily. All Python 3
versions are affected, Python 2.7 looks save.

The problematic code line is
https://hg.python.org/cpython/file/bc1a178b3bc8/Objects/listobject.c#l19
65
. The code fails to restore self->ob_item to saved_ob_item when
PyMem_MALLOC() fails. Subsequent access to the same list object will
dereference self->ob_item (which is still NULL) and cause a segfault.

A remote attack might be able to trigger the segfault with a large
data set. All it takes is an application that sorts this large data
set with list.sort() and a custom key function. When Python runs out
of memory just in the right spot ... CRASH.

Additionally there is another bug, too. list.sort() doesn't set an
exception when PyMem_MALLOC() fails. A fix for both issues is simple
and straight forward:

diff -r bc1a178b3bc8 Objects/listobject.c
- --- a/Objects/listobject.c      Sat Apr 18 05:54:02 2015 +0200
+++ b/Objects/listobject.c      Sat Apr 18 06:29:02 2015 +0200
@@ -1961,8 +1961,10 @@
             keys = &ms.temparray[saved_ob_size+1];
         else {
             keys = PyMem_MALLOC(sizeof(PyObject *) * saved_ob_size);
- -            if (keys == NULL)
- -                return NULL;
+            if (keys == NULL) {
+                PyErr_NoMemory();
+                goto keyfunc_fail;
+            }
         }

         for (i = 0; i < saved_ob_size ; i++) {

----------
components: Interpreter Core
messages: 241889
nosy: benjamin.peterson, christian.heimes
priority: high
severity: normal
status: open
title: NULL pointer dereference in listsort() with key function
type: crash
versions: Python 2.7, Python 3.2, Python 3.3, Python 3.4, Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24044>
_______________________________________

From report at bugs.python.org  Fri Apr 24 02:47:09 2015
From: report at bugs.python.org (Ethan Furman)
Date: Fri, 24 Apr 2015 00:47:09 +0000
Subject: [New-bugs-announce] [issue24045] Behavior of large returncodes
	(sys.exit(nn))
Message-ID: <1429836429.07.0.888331603842.issue24045@psf.upfronthosting.co.za>


New submission from Ethan Furman:

Not sure if this is a bug, or just One of Those Things:

sys.exit(large_value) can wrap around if the value is too large, but this is O/S dependent.

linux (ubuntu 14.04)

  $ python
  Python 2.7.8 (default, Oct 20 2014, 15:05:29) 
  [GCC 4.9.1] on linux2
  Type "help", "copyright", "credits" or "license" for more information.
  --> import sys
  --> sys.exit(256)
  $ echo $?
  0

  $ python
  Python 2.7.8 (default, Oct 20 2014, 15:05:29) 
  [GCC 4.9.1] on linux2
  Type "help", "copyright", "credits" or "license" for more information.
  --> import sys
  --> sys.exit(257)
  $ echo $?
  1

M$ (Windows 7)

  > python
  Python 2.7... 
  --> import sys
  --> sys.exit(65535)
  > echo %errorlevel%
  65535

  > python
  Python 2.7...
  --> import sys
  --> sys.exit(100000)
  > echo %errorlevel%
  100000

Perhaps a minor doc update that talks about return codes and why they might not be exactly what was given to Python?

----------
assignee: docs at python
messages: 241903
nosy: docs at python, ethan.furman
priority: normal
severity: normal
status: open
title: Behavior of large returncodes  (sys.exit(nn))
versions: Python 2.7, Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24045>
_______________________________________

From report at bugs.python.org  Fri Apr 24 04:54:41 2015
From: report at bugs.python.org (aixtools)
Date: Fri, 24 Apr 2015 02:54:41 +0000
Subject: [New-bugs-announce] [issue24046] Incomplete build on AIX
Message-ID: <1429844081.77.0.468875149896.issue24046@psf.upfronthosting.co.za>


New submission from aixtools:

Actually, I have been building and using my builds of Python, when needed for ./configure requirements for a long time. In short, it is quite nice that make "completes" even when there are missing and/or failed modules.

I have just resolved the problem with ctypes not building (see https://bugs.python.org/issue6006) and that got me started to research others.

Failed to build these modules:
_elementtree       _sqlite3           _ssl            
bz2                pyexpat                            


While there are several - I am looking first at ssl.

My first attempt comes up with some failed defines - probably because the latest openssl provided by IBM is openssl-1.0.0 and openssl-1.0.1 is needed.

Rather than wait for that to happen I decided to experiment with LibreSSL. If you are not familiar with LibreSSL - I shall be quick - openbsd (who also maintains openssh) has been cutting out insecure and/or superfluous code.

One of the more insecure (because it can be a predictable source of enthropy) is RAND_egd() - so it is unavoidable that this occurs:

ld: 0711-317 ERROR: Undefined symbol: .RAND_egd

After patching _ssl.c to this:
--- _ssl.c.orig 2014-06-30 02:05:42 +0000
+++ _ssl.c      2015-04-24 02:47:00 +0000
@@ -1604,6 +1604,7 @@
 static PyObject *
 PySSL_RAND_egd(PyObject *self, PyObject *arg)
 {
+#ifndef LIBRESSL_VERSION_NUMBER
     int bytes;
 
     if (!PyString_Check(arg))
@@ -1618,6 +1619,12 @@
         return NULL;
     }
     return PyInt_FromLong(bytes);
+#else
+        PyErr_SetString(PySSLErrorObject,
+                        "external EGD connection not allowed when using LibreSSL:"
+                        "no data to seed the PRNG via PySSL_RAND_egd");
+        return NULL;
+#endif
 }
 
 PyDoc_STRVAR(PySSL_RAND_egd_doc,

The end result is:
Failed to build these modules:
_elementtree       _sqlite3           bz2             
pyexpat 

In short, you can get ahead of the curve by depreciating/removing PySSL_RAND_egd() because any code that uses it may be receiving predictable input and thereafter everything may be predictable.

If you do not believe openbsd (or me) - just read the code. It calls anything configured (handy when /dev/urandom was hard to find anno 1999) but these days a backdoor waiting to be opened.

p.s. As I get time I shall continue with the other modules that do not build - just let me know if you prefer that I continue posting in this "issue", or make new one(s) for each module as I find a solution.

----------
components: Extension Modules
messages: 241908
nosy: aixtools at gmail.com
priority: normal
severity: normal
status: open
title: Incomplete build on AIX
type: compile error
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24046>
_______________________________________

From report at bugs.python.org  Fri Apr 24 07:49:07 2015
From: report at bugs.python.org (Elizabeth Myers)
Date: Fri, 24 Apr 2015 05:49:07 +0000
Subject: [New-bugs-announce] [issue24047] str.startswith and str.endswith
	should accept multiple	arguments.
Message-ID: <1429854547.35.0.046111286022.issue24047@psf.upfronthosting.co.za>


New submission from Elizabeth Myers:

str.startswith and str.endswith should accept multiple arguments when passing in strings. This makes it easier to check if the first character of a string is one of a given option, versus this awkward construction:

>>> f = 'abc'
>>> 'test'.startswith(tuple(f))
False

With my proposal, this could be shortened to 'test'.startswith(*f) for easy testing.

This also makes it easier to check if a string begins with one of any combination of matches.

----------
components: Interpreter Core, Unicode
messages: 241912
nosy: Elizacat, ezio.melotti, haypo
priority: normal
severity: normal
status: open
title: str.startswith and str.endswith should accept multiple arguments.
type: enhancement
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24047>
_______________________________________

From report at bugs.python.org  Fri Apr 24 07:58:38 2015
From: report at bugs.python.org (Davis Herring)
Date: Fri, 24 Apr 2015 05:58:38 +0000
Subject: [New-bugs-announce] [issue24048] remove_module() needs to
	save/restore exception state
Message-ID: <1429855118.66.0.488857300303.issue24048@psf.upfronthosting.co.za>


New submission from Davis Herring:

import.c's remove_module() is always called with an exception set and can invoke arbitrary code via deallocation; if that code calls PyErr_Clear() (or is sensitive to PyErr_Occurred()) it will lose (or be damaged by) the preexisting exception.

----------
components: Interpreter Core
messages: 241914
nosy: herring
priority: normal
severity: normal
status: open
title: remove_module() needs to save/restore exception state
type: behavior
versions: Python 2.7, Python 3.2, Python 3.3, Python 3.4, Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24048>
_______________________________________

From report at bugs.python.org  Fri Apr 24 14:36:02 2015
From: report at bugs.python.org (Nick Craig-Wood)
Date: Fri, 24 Apr 2015 12:36:02 +0000
Subject: [New-bugs-announce] [issue24049] Remove unused code in symtable.c
	and fix docs for import	* checking
Message-ID: <1429878962.91.0.558127171831.issue24049@psf.upfronthosting.co.za>


New submission from Nick Craig-Wood:

Here is a patch to remove some unused code in `symtable.c`

In Python3 `from x import *` was banned from use in functions completely.

This is detected by `symtable_visit_alias` 

        if (st->st_cur->ste_type != ModuleBlock) {
            int lineno = st->st_cur->ste_lineno;
            int col_offset = st->st_cur->ste_col_offset;
            PyErr_SetString(PyExc_SyntaxError, IMPORT_STAR_WARNING);

However in `check_unoptimized` it checks for `import *` being used in a nested function etc.  This is the python2 behaviour which wasn't removed at the time the new python3 behaviour was added.

According to my analysis and tests it is now impossible for `check_unoptimized` to raise an error, since only valid uses of `import *` are left at the point it is called.

I propose to remove that function entirely, its call, and fix some stray documentation in this patch.

----------
components: Interpreter Core
files: python3.5-symtable.patch
keywords: patch
messages: 241937
nosy: ncw
priority: normal
severity: normal
status: open
title: Remove unused code in symtable.c and fix docs for import * checking
type: enhancement
versions: Python 3.5
Added file: http://bugs.python.org/file39195/python3.5-symtable.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24049>
_______________________________________

From report at bugs.python.org  Fri Apr 24 15:34:45 2015
From: report at bugs.python.org (nivin)
Date: Fri, 24 Apr 2015 13:34:45 +0000
Subject: [New-bugs-announce] [issue24050] Segmentation fault (core dumped)
Message-ID: <1429882485.21.0.520202259055.issue24050@psf.upfronthosting.co.za>


New submission from nivin:

Got an error as Segmentation fault (core dumped) when executed a python script . Error getting only for a specific python script only

----------
components: Interpreter Core
messages: 241940
nosy: nivin
priority: normal
severity: normal
status: open
title: Segmentation fault (core dumped)
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24050>
_______________________________________

From report at bugs.python.org  Fri Apr 24 15:48:50 2015
From: report at bugs.python.org (Serhiy Storchaka)
Date: Fri, 24 Apr 2015 13:48:50 +0000
Subject: [New-bugs-announce] [issue24051] Argument Clinic no longer works
	with single optional	argument
Message-ID: <1429883330.67.0.210732695704.issue24051@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka:

Argument Clinic no longer works single optional argument. For example see _tkinter.tkapp.wantobjects in the patch in issue20168.

/*[clinic input]
_tkinter.tkapp.wantobjects

    [
    value: int
    ]
    /

[clinic start generated code]*/

It is converted to the methoddef of type METH_O.

#define _TKINTER_TKAPP_WANTOBJECTS_METHODDEF    \
    {"wantobjects", (PyCFunction)_tkinter_tkapp_wantobjects, METH_O, _tkinter_tkapp_wantobjects__doc__},

static PyObject *
_tkinter_tkapp_wantobjects(PyObject *self, PyObject *arg)
{
    PyObject *return_value = NULL;
    int group_right_1 = 0;
    int value = 0;

    if (!PyArg_Parse(arg, "i:wantobjects", &value))
        goto exit;
    return_value = _tkinter_tkapp_wantobjects_impl(self, group_right_1, value);

exit:
    return return_value;
}

As result wantobjects() can't be called without an argument.

----------
components: Argument Clinic
messages: 241943
nosy: larry, serhiy.storchaka
priority: normal
severity: normal
stage: needs patch
status: open
title: Argument Clinic no longer works with single optional argument
type: behavior
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24051>
_______________________________________

From report at bugs.python.org  Fri Apr 24 17:29:13 2015
From: report at bugs.python.org (Alexander Belopolsky)
Date: Fri, 24 Apr 2015 15:29:13 +0000
Subject: [New-bugs-announce] [issue24052] sys.exit(code) returns "success"
	to the OS for some	values of code
Message-ID: <1429889353.58.0.941477823236.issue24052@psf.upfronthosting.co.za>


New submission from Alexander Belopolsky:

$ python3
Python 3.4.3 (default, Mar  2 2015, 11:08:35)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-4)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> import subprocess
>>> subprocess.call([sys.executable, '-c', "import sys;sys.exit(512)"])
0
>>> subprocess.call([sys.executable, '-c', "import sys;sys.exit(256)"])
0

See also #14376.

----------
components: Interpreter Core
messages: 241946
nosy: belopolsky
priority: normal
severity: normal
status: open
title: sys.exit(code) returns "success" to the OS for some values of code
type: behavior
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24052>
_______________________________________

From report at bugs.python.org  Fri Apr 24 17:47:49 2015
From: report at bugs.python.org (Alexander Belopolsky)
Date: Fri, 24 Apr 2015 15:47:49 +0000
Subject: [New-bugs-announce] [issue24053] Define EXIT_SUCCESS and
	EXIT_FAILURE constants in sys
Message-ID: <1429890469.94.0.27310602141.issue24053@psf.upfronthosting.co.za>


New submission from Alexander Belopolsky:

Python defines some BSDish exit codes in the os module:

    EX_CANTCREAT = 73
    EX_CONFIG = 78
    EX_DATAERR = 65
    EX_IOERR = 74
    EX_NOHOST = 68
    EX_NOINPUT = 66
    EX_NOPERM = 77
    EX_NOUSER = 67
    EX_OK = 0
    EX_OSERR = 71
    EX_OSFILE = 72
    EX_PROTOCOL = 76
    EX_SOFTWARE = 70
    EX_TEMPFAIL = 75
    EX_UNAVAILABLE = 69
    EX_USAGE = 64

but these are documented as only available on UNIX and may not be portable across all flavors. 


POSIX [1] and C99 defines EXIT_SUCCESS and EXIT_FAILURE constants.  I propose adding those to sys.

Having these constants in sys will make them more discoverable because they will be next to sys.exit().


[1] http://pubs.opengroup.org/onlinepubs/009695399/functions/exit.html

----------
messages: 241949
nosy: belopolsky
priority: normal
severity: normal
status: open
title: Define EXIT_SUCCESS and EXIT_FAILURE constants in sys
type: enhancement

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24053>
_______________________________________

From report at bugs.python.org  Fri Apr 24 18:45:33 2015
From: report at bugs.python.org (David D. Riddle)
Date: Fri, 24 Apr 2015 16:45:33 +0000
Subject: [New-bugs-announce] [issue24054] Invalid syntax in
	inspect_fodder2.py (on Python 2.x)
Message-ID: <1429893933.08.0.150501464744.issue24054@psf.upfronthosting.co.za>


New submission from David D. Riddle:

test_linecache.py reads from three files namely inspect_fodder.py, inspect_fodder2.py, and mapping_tests.py. It reads the py files directly as text files. This patch copies these files to linecache_fodder, linecache_fodder2, and linecache_mapping_fodder respectively, and updates test_linecache.py accordingly.

The reason I do this is so that the these files are not compiled. This is desirable for me because I use linecache2 on python 2.7. Python 2.7 can not compile inspect_fodder2.py nor is it necessary as this file is only used as a text file never as an object file in the test_linecache.py tests.

This issue came up for me when I attempted to make an RPM out of linecache2. rpmbuild compiles all py files in the rpm and fails when it tries to compile inspect_fodder2.py. If these files have the .py removed from the test files then they will not be compiled which fixes my issue. Nor do they need to be compiled to successfully run the test_linecache.py tests.

----------
components: Tests
files: mywork.patch
keywords: patch
messages: 241955
nosy: ddriddle
priority: normal
severity: normal
status: open
title: Invalid syntax in inspect_fodder2.py (on Python 2.x)
type: compile error
Added file: http://bugs.python.org/file39198/mywork.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24054>
_______________________________________

From report at bugs.python.org  Fri Apr 24 22:25:56 2015
From: report at bugs.python.org (Demian Brecht)
Date: Fri, 24 Apr 2015 20:25:56 +0000
Subject: [New-bugs-announce] [issue24055] unittest package-level set up &
	tear down module
Message-ID: <1429907156.24.0.517799178005.issue24055@psf.upfronthosting.co.za>


New submission from Demian Brecht:

There's a feature available via nose that might be nice to have in unittest: package-level setup and teardown functions.

Using nose, I can define a setUpModule() method in a test package's __init__.py and it will execute it during the test run. This is helpful for test packages that test related features and may have expensive setup/teardown routines. By having the test runner pick up setup/teardown methods in __init__.py, I can now incur this expense once rather than once per module.

I don't mind putting a patch together for this if others think this might be beneficial.

----------
components: Library (Lib)
messages: 241981
nosy: demian.brecht, ezio.melotti, michael.foord, rbcollins
priority: normal
severity: normal
stage: needs patch
status: open
title: unittest package-level set up & tear down module
type: enhancement
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24055>
_______________________________________

From report at bugs.python.org  Sat Apr 25 08:13:53 2015
From: report at bugs.python.org (Nick Coghlan)
Date: Sat, 25 Apr 2015 06:13:53 +0000
Subject: [New-bugs-announce] [issue24056] Expose closure & generator status
	in function repr()
Message-ID: <1429942433.91.0.521457748741.issue24056@psf.upfronthosting.co.za>


New submission from Nick Coghlan:

>From https://mail.python.org/pipermail/python-ideas/2015-April/033177.html, there are some additional details about functions that could be usefully exposed in the function repr, specifically whether or not it's a closure, and whether or not it's a generator function.

Proposed display:

    <function f at 0x7f7dad9f7bf8 (closure)>
    <function f at 0x7f7dad9f7bf8 (generator)>
    <function f at 0x7f7dad9f7bf8 (closure,generator)>

The Python level checks for the two flags:

closure: f.__closure__ is not None
generator: c.__code__.co_flags & inspect.CO_GENERATOR

Actual implementation would be in the C code at https://hg.python.org/cpython/file/default/Objects/funcobject.c#l569

----------
messages: 241994
nosy: ncoghlan
priority: normal
severity: normal
status: open
title: Expose closure & generator status in function repr()
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24056>
_______________________________________

From report at bugs.python.org  Sat Apr 25 08:55:58 2015
From: report at bugs.python.org (Novice Live)
Date: Sat, 25 Apr 2015 06:55:58 +0000
Subject: [New-bugs-announce] [issue24057] trivial typo in datetime.rst:
	needing a preceding dot
Message-ID: <1429944958.51.0.607005729778.issue24057@psf.upfronthosting.co.za>


New submission from Novice Live:

a non-trivial dot is needed for the hyperlink to make sense, or the link will jump to the start of the documentation, which doesn't make sense. 

the same typo as in http://bugs.python.org/issue23561.

----------
assignee: docs at python
components: Documentation
files: datetime_typo.patch
keywords: patch
messages: 241998
nosy: docs at python, n0vicelive
priority: normal
severity: normal
status: open
title: trivial typo in datetime.rst: needing a preceding dot
versions: Python 3.4, Python 3.5
Added file: http://bugs.python.org/file39202/datetime_typo.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24057>
_______________________________________

From report at bugs.python.org  Sat Apr 25 16:21:10 2015
From: report at bugs.python.org (Masayuki Yamamoto)
Date: Sat, 25 Apr 2015 14:21:10 +0000
Subject: [New-bugs-announce] [issue24058] Compiler warning for readline
	extension
Message-ID: <1429971670.78.0.634220240768.issue24058@psf.upfronthosting.co.za>


New submission from Masayuki Yamamoto:

Compiler warns case of define HAVE_DECLSPEC_DLL.
In Modules/readline.c:1065, _PyOS_ReadlineTState variable declaration is different to Include/pythonrun.h:275.

----------
components: Build, Extension Modules
hgrepos: 307
messages: 242013
nosy: masamoto
priority: normal
severity: normal
status: open
title: Compiler warning for readline extension
type: compile error
versions: Python 3.4, Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24058>
_______________________________________

From report at bugs.python.org  Sat Apr 25 17:35:48 2015
From: report at bugs.python.org (Raymond Hettinger)
Date: Sat, 25 Apr 2015 15:35:48 +0000
Subject: [New-bugs-announce] [issue24059] Minor speed and readability
	improvement to the random	module
Message-ID: <1429976148.71.0.723418409386.issue24059@psf.upfronthosting.co.za>


New submission from Raymond Hettinger:

Replace all the _sqrt(x) calls with x ** 0.5, improving the visual appearance and providing a modest speed improvement.

$ python3 -m timeit -s 'from math import sqrt' 'sqrt(3.14)'
10000000 loops, best of 3: 0.032 usec per loop
$ python3 -m timeit -s 'from math import sqrt' '3.14 ** 0.5'
100000000 loops, best of 3: 0.0101 usec per loop

----------
components: Library (Lib)
files: random_sqrt.diff
keywords: patch
messages: 242017
nosy: rhettinger
priority: low
severity: normal
stage: patch review
status: open
title: Minor speed and readability improvement to the random module
type: performance
versions: Python 3.5
Added file: http://bugs.python.org/file39206/random_sqrt.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24059>
_______________________________________

From report at bugs.python.org  Sat Apr 25 23:17:04 2015
From: report at bugs.python.org (Karl Richter)
Date: Sat, 25 Apr 2015 21:17:04 +0000
Subject: [New-bugs-announce] [issue24060] Clearify necessities for logging
	with timestamps
Message-ID: <1429996624.15.0.693068386122.issue24060@psf.upfronthosting.co.za>


New submission from Karl Richter:

The `Formatter` section of the `logging` module at https://docs.python.org/2/library/logging.html#formatter-objects reads like it's sufficient to create an instance of `Formatter` with default arguments (and set it as formatter of the `Handler` of a `Logger`) to have add timestamps to logging output.

----------
assignee: docs at python
components: Documentation
messages: 242024
nosy: docs at python, krichter
priority: normal
severity: normal
status: open
title: Clearify necessities for logging with timestamps
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24060>
_______________________________________

From report at bugs.python.org  Sun Apr 26 15:17:30 2015
From: report at bugs.python.org (Hanno Boeck)
Date: Sun, 26 Apr 2015 13:17:30 +0000
Subject: [New-bugs-announce] [issue24061] Python 2.x breaks with address
	sanitizer
Message-ID: <1430054250.12.0.0817440488321.issue24061@psf.upfronthosting.co.za>


New submission from Hanno Boeck:

Right now it is not possible to build python 2.7.9 with address sanitizer. This issue has been worked around for python 3 in bug #18596 by marking some functions with attributes to tell address sanitizer to ignore them.

I have attached a patch that will apply the same workaround to python 2.7. I'd apprechiate if this could be applied before the next python 2 release. Although it's the "old" python 2 version I think this is important enough to be backported.

----------
files: python-2.7.9-workaround-asan.diff
keywords: patch
messages: 242055
nosy: hanno
priority: normal
severity: normal
status: open
title: Python 2.x breaks with address sanitizer
versions: Python 2.7
Added file: http://bugs.python.org/file39208/python-2.7.9-workaround-asan.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24061>
_______________________________________

From report at bugs.python.org  Sun Apr 26 21:14:18 2015
From: report at bugs.python.org (July Tikhonov)
Date: Sun, 26 Apr 2015 19:14:18 +0000
Subject: [New-bugs-announce] [issue24062] links to os.stat() in
	documentation lead to stat module	instead
Message-ID: <1430075658.37.0.617374885608.issue24062@psf.upfronthosting.co.za>


New submission from July Tikhonov:

Documentation of os.fstat()
https://docs.python.org/3/library/os.html#os.fstat
has a "See also:" section, which features a wrong link. The same with os.lstat().

Some of this problem was fixed (among other things) in issue 10960. But since then, two more wrong links appeared.

Attached patch applies to 3.5, although 3.4 has the same problem.

----------
assignee: docs at python
components: Documentation
files: doc-library-os-stat-links.diff
keywords: patch
messages: 242070
nosy: docs at python, july
priority: normal
severity: normal
status: open
title: links to os.stat() in documentation lead to stat module instead
type: enhancement
versions: Python 3.4, Python 3.5, Python 3.6
Added file: http://bugs.python.org/file39209/doc-library-os-stat-links.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24062>
_______________________________________

From report at bugs.python.org  Sun Apr 26 21:43:46 2015
From: report at bugs.python.org (=?utf-8?q?R=C3=A9mi_Verschelde?=)
Date: Sun, 26 Apr 2015 19:43:46 +0000
Subject: [New-bugs-announce] [issue24063] Support Mageia and Arch Linux in
	the platform module
Message-ID: <1430077426.39.0.0830591110576.issue24063@psf.upfronthosting.co.za>


New submission from R?mi Verschelde:

Support for Mageia and Arch Linux was added in the platform module for Python 3.x [1, 2], but the fix was not backported to the 2.7.x branch.

It would be nice to see these fixes cherry-picked for the incoming 2.7.10 release.

[1] https://hg.python.org/cpython/rev/97a098aa4205
[2] https://hg.python.org/cpython/rev/85d827cbabfa

----------
components: Library (Lib)
messages: 242073
nosy: akien
priority: normal
severity: normal
status: open
title: Support Mageia and Arch Linux in the platform module
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24063>
_______________________________________

From report at bugs.python.org  Mon Apr 27 06:53:07 2015
From: report at bugs.python.org (Raymond Hettinger)
Date: Mon, 27 Apr 2015 04:53:07 +0000
Subject: [New-bugs-announce] [issue24064] Make the property doctstring
	writeable
Message-ID: <1430110387.37.0.412217606886.issue24064@psf.upfronthosting.co.za>


New submission from Raymond Hettinger:

I can't see any reason for property docstrings to be readonly:

    >>> p = property(doc='basic')
    >>> p.__doc__
    'basic'
    >>> p.__doc__ = 'extended'
    Traceback (most recent call last):
      File "<pyshell#46>", line 1, in <module>
        p.__doc__ = 'extended'
    AttributeError: readonly attribute

Among other things, making it writeable would simplify the ability to update the docstrings produced by namedtuple;

    Time.mtime.__doc__ = 'modification time'
    Score.max = 'all time cumulative high score for a single season'

----------
components: Interpreter Core
keywords: easy
messages: 242098
nosy: rhettinger
priority: normal
severity: normal
stage: patch review
status: open
title: Make the property doctstring writeable
type: enhancement
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24064>
_______________________________________

From report at bugs.python.org  Mon Apr 27 19:40:19 2015
From: report at bugs.python.org (Berker Peksag)
Date: Mon, 27 Apr 2015 17:40:19 +0000
Subject: [New-bugs-announce] [issue24065] Outdated *_RESTRICTED flags in
	structmember.h
Message-ID: <1430156419.01.0.516849484938.issue24065@psf.upfronthosting.co.za>


New submission from Berker Peksag:

Looks like READ_RESTRICTED, PY_WRITE_RESTRICTED and RESTRICTED flags were used for "restricted mode" [1] in Python 2. "restricted mode" has been deprecated in Python 2.3. Also, the current documentation is outdated. WRITE_RESTRICTED is now PY_WRITE_RESTRICTED: https://docs.python.org/3/extending/newtypes.html#generic-attribute-management

There are a few usages of these flags in the CPython source:

PY_WRITE_RESTRICTED

* Objects/funcobject.c
* Objects/methodobject.c

RESTRICTED

* Objects/funcobject.c
* Objects/classobject.c

Are they still useful or can we deprecate/remove them now?

[1] https://github.com/python/cpython/blob/2.7/Python/structmember.c#L180

----------
components: Interpreter Core
messages: 242134
nosy: berker.peksag
priority: normal
severity: normal
status: open
title: Outdated *_RESTRICTED flags in structmember.h
type: enhancement
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24065>
_______________________________________

From report at bugs.python.org  Mon Apr 27 23:26:12 2015
From: report at bugs.python.org (Kirill Elagin)
Date: Mon, 27 Apr 2015 21:26:12 +0000
Subject: [New-bugs-announce] [issue24066] send_message should take all the
	addresses in the To:	header into account
Message-ID: <1430169972.74.0.338342255395.issue24066@psf.upfronthosting.co.za>


New submission from Kirill Elagin:

If I have a message with multiple `To` headers and I send it using `send_message` not specifying `to_addrs`, the message gets sent only to one of the recipients.

I?m attaching my patch that makes it send to _all_ the addresses listed in `To`, `Cc` and `Bcc`.

I didn?t add any new tests as the existing ones already cover those cases and I have no idea how on Earth do they pass.

----------
components: Library (Lib)
messages: 242158
nosy: kirelagin
priority: normal
severity: normal
status: open
title: send_message should take all the addresses in the To: header into account

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24066>
_______________________________________

From report at bugs.python.org  Mon Apr 27 23:42:46 2015
From: report at bugs.python.org (Eyal Reuveni)
Date: Mon, 27 Apr 2015 21:42:46 +0000
Subject: [New-bugs-announce] [issue24067] Weakproxy is an instance of
	collections.Iterator
Message-ID: <1430170966.34.0.831827810985.issue24067@psf.upfronthosting.co.za>


New submission from Eyal Reuveni:

Calling weakref.proxy() on an object returns something that is an instance of collections.Iterator, regardless of whether the object is an instance of collections.Iterator or even if the object itself is iterable.

To reproduce, run the following code (verified in python2.7 and 3.4 on my Mac):

import collections
import weakref

class NotIterator:
    pass

not_iterator = NotIterator()
isinstance(not_iterator, collections.Iterator) # returns False

proxy_object = weakref.proxy(not_iterator)
isinstance(proxy_object, collections.Iterator) # returns True, should be False

----------
components: Library (Lib)
messages: 242159
nosy: ereuveni
priority: normal
severity: normal
status: open
title: Weakproxy is an instance of collections.Iterator
type: behavior
versions: Python 2.7, Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24067>
_______________________________________

From report at bugs.python.org  Tue Apr 28 10:53:26 2015
From: report at bugs.python.org (Wolfgang Maier)
Date: Tue, 28 Apr 2015 08:53:26 +0000
Subject: [New-bugs-announce] [issue24068] statistics module - incorrect
	results with boolean input
Message-ID: <1430211206.61.0.689765465928.issue24068@psf.upfronthosting.co.za>


New submission from Wolfgang Maier:

the mean function in the statistics module gives nonsensical results with boolean values in the input, e.g.:

>>> mean([True, True, False, False])
0.25

>>> mean([True, 1027])
0.5

This is an issue with the module's internal _sum function that mean relies on. Other functions relying on _sum are affected more subtly, e.g.:

>>> variance([1, 1027, 0])
351234.3333333333

>>> variance([True, 1027, 0])
351234.3333333334

The problem with _sum is that it will try to coerce its result to any non-int type found in the input (so bool in the examples), but bool(1028) is just True so information gets lost.

I've attached a patch preventing the type cast when it would be to bool.
I don't have time to write a separate test though so if somebody wants to take over .. :)

----------
components: Library (Lib)
files: statistics._sum.patch
keywords: patch
messages: 242169
nosy: steven.daprano, wolma
priority: normal
severity: normal
status: open
title: statistics module - incorrect results with boolean input
type: behavior
versions: Python 3.4, Python 3.5
Added file: http://bugs.python.org/file39220/statistics._sum.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24068>
_______________________________________

From report at bugs.python.org  Tue Apr 28 18:44:19 2015
From: report at bugs.python.org (Sworddragon)
Date: Tue, 28 Apr 2015 16:44:19 +0000
Subject: [New-bugs-announce] [issue24069] Option to delete obsolete bytecode
	files
Message-ID: <1430239459.0.0.172566043873.issue24069@psf.upfronthosting.co.za>


New submission from Sworddragon:

The library compileall has the option -f to force the rebuilding of the bytecode files so I thought maybe there could also be an option to delete all bytecode files which haven't a non-bytecode library anymore.

----------
components: Library (Lib)
messages: 242189
nosy: Sworddragon
priority: normal
severity: normal
status: open
title: Option to delete obsolete bytecode files
type: enhancement
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24069>
_______________________________________

From report at bugs.python.org  Tue Apr 28 19:57:25 2015
From: report at bugs.python.org (Roy Hyunjin Han)
Date: Tue, 28 Apr 2015 17:57:25 +0000
Subject: [New-bugs-announce] [issue24070] Exceptions and arguments disappear
	when using argparse	inside with statement
Message-ID: <1430243845.68.0.464856289952.issue24070@psf.upfronthosting.co.za>


New submission from Roy Hyunjin Han:

Exceptions and arguments disappear when using argparse inside a "with" statement.  The behavior was confusing and frustrating because I could not pinpoint why certain arguments were missing or unrecognized.

Unhandled exceptions inside the with statement typically trigger a traceback, unless __exit__ returns True, in which the exception is "swallowed" (https://www.python.org/dev/peps/pep-0343/).

However, the NameError exception and tile_indices argument disappear because of a premature sys.exit (https://hg.python.org/cpython/file/default/Lib/argparse.py#l2385).

```
"""
$ python exception-in-with.py
EXPECTED
Traceback (most recent call last):
  File "exception-in-with.py", line 51, in <module>
    abc  # !!!
NameError: name 'abc' is not defined
ACTUAL
usage: exception-in-with.py [-h] --image_path PATH
exception-in-with.py: error: argument --image_path is required

$ python exception-in-with.py --image_path x
EXPECTED == ACTUAL
Traceback (most recent call last):
  File "exception-in-with.py", line 51, in <module>
    abc  # !!!
NameError: name 'abc' is not defined

$ python exception-in-with.py --image_path x --tile_indices 1
EXPECTED
Traceback (most recent call last):
  File "exception-in-with.py", line 51, in <module>
    abc  # !!!
NameError: name 'abc' is not defined
ACTUAL
usage: exception-in-with.py [-h] --image_path PATH
exception-in-with.py: error: unrecognized arguments: --tile_indices 1
"""
from argparse import ArgumentParser


class Starter(object):

    def __init__(self):
        self.argument_parser = ArgumentParser()

    def __enter__(self):
        return self

    def __exit__(self, exception_type, exception_value, exception_traceback):
        self.argument_parser.parse_args()

    def add_argument(self, *args, **kw):
        self.argument_parser.add_argument(*args, **kw)


with Starter() as starter:
    starter.add_argument('--image_path', metavar='PATH', required=True)
    abc  # !!!
    starter.add_argument('--tile_indices', metavar='INTEGER')
```

----------
components: Library (Lib)
files: argparse-with-exception.py
messages: 242192
nosy: invisibleroads
priority: normal
severity: normal
status: open
title: Exceptions and arguments disappear when using argparse inside with statement
type: behavior
versions: Python 2.7, Python 3.3
Added file: http://bugs.python.org/file39227/argparse-with-exception.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24070>
_______________________________________

From report at bugs.python.org  Tue Apr 28 21:25:28 2015
From: report at bugs.python.org (Stephen Evans)
Date: Tue, 28 Apr 2015 19:25:28 +0000
Subject: [New-bugs-announce] [issue24071] Python 2.7.8,
	2.7.9  re.MULTILINE failure
Message-ID: <1430249128.2.0.387298041566.issue24071@psf.upfronthosting.co.za>


New submission from Stephen Evans:

A simple multiline regex fails when just the re.MULTILINE argument is used, but works when equivalent alternative methods are used. This was tested on Python2.7.8 on FreeBSD and Win32 Python2.7.9

data = re.sub(r'#.*', '', text, re.MULTILINE) # fails

data = re.sub(r'(?m)#.*', '', text) # Ok
data = re.sub(r'#.*', '', text, re.MULTILINE|re.DEBUG) # Ok

All the expressions work correctly with Win64 Python3.4.3
The attached file has the code and with a sample of text that fails.

----------
components: Regular Expressions
files: refail.py
messages: 242205
nosy: Stephen.Evans, ezio.melotti, mrabarnett
priority: normal
severity: normal
status: open
title: Python 2.7.8, 2.7.9  re.MULTILINE failure
type: behavior
versions: Python 2.7
Added file: http://bugs.python.org/file39229/refail.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24071>
_______________________________________

From report at bugs.python.org  Wed Apr 29 02:42:01 2015
From: report at bugs.python.org (=?utf-8?q?J=C3=A9r=C3=B4me_Laurens?=)
Date: Wed, 29 Apr 2015 00:42:01 +0000
Subject: [New-bugs-announce] [issue24072] xml.etree.ElementTree.Element does
	not catch text
Message-ID: <1430268121.66.0.962770633094.issue24072@psf.upfronthosting.co.za>


New submission from J?r?me Laurens:

text is not catcher in case 3 below

INPUT

import xml.etree.ElementTree as ET
root1 = ET.fromstring('<a>TEXT</a>')
print(root1.text)
root2 = ET.fromstring('<a>TEXT<b/></a>')
print(root2.text)
root3 = ET.fromstring('<a><b/>TEXT</a>')
print(root3.text)

CURRENT OUTPUT

TEXT
TEXT
None <---------- ERROR HERE

EXPECTED OUTPUT

TEXT
TEXT
TEXT

----------
messages: 242207
nosy: jlaurens
priority: normal
severity: normal
status: open
title: xml.etree.ElementTree.Element does not catch text
type: behavior
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24072>
_______________________________________

From report at bugs.python.org  Wed Apr 29 04:52:14 2015
From: report at bugs.python.org (Xiang Zhang)
Date: Wed, 29 Apr 2015 02:52:14 +0000
Subject: [New-bugs-announce] [issue24073] sys.stdin.mode can not give the
	right mode and os.fdopen	does not check argument
Message-ID: <1430275934.06.0.903599257127.issue24073@psf.upfronthosting.co.za>


New submission from Xiang Zhang:

The problem is what the title tells and can be produced by the snippet below.

import sys
import os

sys.stdout.write("%s\n" % sys.stdin.mode)
sys.stdout.flush()
f = os.fdopen(sys.stdin.fileno(), "r")
f.write("aaaa")
f.flush()
f.read()
f.close()

When running this snippet with nohup, which changes the stdin's mode to O_WRONLY(which can also be shown below because the write operation will fail), this snippet will still give sys.stdin.mode as r, both in 2.7 and 3.4.

In 2.7, the os.fdopen will fail with Invalid Argument error because the mode r given to fdopen conflicts with stdin's real mode w. In 3.4, os.fdopen won't give any error.

Both in 2.7 and 3.4, if I change the mode given to os.fdopen to w:
    f = os.fdopen(sys.stdin.fileno(), "w")
The write operation will succeed and the read operation will fail.

The output produced in nohup.out for 2.7 is:
    r
    Traceback (most recent call last):
      File "test.py", line 9, in <module>
        f.read()
    IOError: File not open for reading
For 3.4:
    r
    Traceback (most recent call last):
      File "test.py", line 9, in <module>
        f.read()
    io.UnsupportedOperation: not readable

I run the snippet with nohup on Gnome Terminal, bash, Ubuntu 15.04. The Python version is 2.7.9 and 3.4.3.

----------
components: Library (Lib)
messages: 242211
nosy: angwer
priority: normal
severity: normal
status: open
title: sys.stdin.mode can not give the right mode and os.fdopen does not check argument
type: behavior
versions: Python 2.7, Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24073>
_______________________________________

From report at bugs.python.org  Wed Apr 29 08:44:04 2015
From: report at bugs.python.org (Glen Fletcher)
Date: Wed, 29 Apr 2015 06:44:04 +0000
Subject: [New-bugs-announce] [issue24074] string, center, ljust, rjust,
	width paramter should accept None
Message-ID: <1430289844.83.0.648544192653.issue24074@psf.upfronthosting.co.za>


New submission from Glen Fletcher:

I've only list python 2.7, as I'm not sure that version 3 doesn't accept None, if so this should be changed there too.

If these function are passed None, as the width they should return the string unchanged, just as they would for with a width set to 0.

Reasoning, A common use of these would be in list comprehensions and such, i.e. (string.center(encode(i), w) for i, w in items, widths), given the that items and widths may be of differing length in theory i should be a empty string and width 0 it not specified however the best way of dealing with this would be to use itertools.izip_longest(items, widths, default=None) and None would be encode as No Value, however this would require writing (string.center(encode(i), 0 if w is None else w) for i, w in itertools.izip_longest(items, widths, default=None)), which is far harder to follow, when its quite reasonable to expect these string alignment functions to return an unpadded string if passed None as the width.

----------
components: Library (Lib)
messages: 242215
nosy: glenflet
priority: normal
severity: normal
status: open
title: string, center, ljust, rjust, width paramter should accept None
type: behavior
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24074>
_______________________________________

From report at bugs.python.org  Wed Apr 29 15:30:43 2015
From: report at bugs.python.org (Sergey B Kirpichev)
Date: Wed, 29 Apr 2015 13:30:43 +0000
Subject: [New-bugs-announce] [issue24075] list.sort() should do quick exit
	if len(list) <= 1
Message-ID: <1430314243.47.0.686981874079.issue24075@psf.upfronthosting.co.za>


New submission from Sergey B Kirpichev:

If there is nothing to sort (i.e. one item), why call key function at all?

In my practical situation, simplest key() function will lead to recursion in case of such trivial lists.  I can make similar cmp-type function (i.e. mycmp=lambda a, b: cmp(key(a), key(b))) and then wrap this with cmp_to_key.  But that looks silly.

Simple test case:
$ cat a.py
a = [1]

def spam(x):
    raise Exception

a.sort(key=spam)
print(a)
$ python a.py 
Traceback (most recent call last):
  File "a.py", line 6, in <module>
    a.sort(key=spam)
  File "a.py", line 4, in spam
    raise Exception
Exception

----------
components: Interpreter Core
files: trivial-sorting-py3.patch
keywords: patch
messages: 242222
nosy: Sergey.Kirpichev
priority: normal
severity: normal
status: open
title: list.sort() should do quick exit if len(list) <= 1
type: behavior
versions: Python 2.7, Python 3.2, Python 3.3, Python 3.4, Python 3.5, Python 3.6
Added file: http://bugs.python.org/file39231/trivial-sorting-py3.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24075>
_______________________________________

From report at bugs.python.org  Wed Apr 29 20:36:58 2015
From: report at bugs.python.org (=?utf-8?q?=C5=81ukasz_Langa?=)
Date: Wed, 29 Apr 2015 18:36:58 +0000
Subject: [New-bugs-announce] [issue24076] sum() several times slower on
	Python 3
Message-ID: <1430332618.4.0.236828536627.issue24076@psf.upfronthosting.co.za>


New submission from ?ukasz Langa:

I got a report that summing numbers is noticably slower on Python 3. This is easily reproducible:


$ time python2.7 -c "print sum(xrange(3, 10**9, 3)) + sum(xrange(5, 10**9, 5)) - sum(xrange(15, 10**9, 15))"
233333333166666668

real    0m6.165s
user    0m6.100s
sys     0m0.032s


$ time python3.4 -c "print(sum(range(3, 10**9, 3)) + sum(range(5, 10**9, 5)) - sum(range(15, 10**9, 15)))"
233333333166666668

real    0m16.413s
user    0m16.086s
sys     0m0.089s


I can't tell from initial poking what's the core issue here. Both examples produce equivalent bytecode, the builtin_sum() function is only noticably different in the fact that it uses PyLong_* across the board, including PyLong_AsLongAndOverlow. We'll need to profile this, which I didn't have time for yet.

----------
components: Interpreter Core
messages: 242238
nosy: lukasz.langa, pitrou, rhettinger
priority: normal
severity: normal
stage: needs patch
status: open
title: sum() several times slower on Python 3
type: performance
versions: Python 3.4, Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24076>
_______________________________________

From report at bugs.python.org  Wed Apr 29 23:19:52 2015
From: report at bugs.python.org (John Beck)
Date: Wed, 29 Apr 2015 21:19:52 +0000
Subject: [New-bugs-announce] [issue24077] man page says -I implies -S. code
	says -s. Should it be	both?
Message-ID: <1430342392.4.0.32328442705.issue24077@psf.upfronthosting.co.za>


New submission from John Beck:

The man page for Python (3.4 and 3.5) says:

   -I     Run Python in isolated mode. This also implies  -E  and  -S.  In
          isolated  mode  sys.path  contains neither the scripts directory
          nor the users site-packages directory. All  PYTHON*  environment
          variables are ignored, too.  Further restrictions may be imposed
          to prevent the user from injecting malicious code.

But the code says:

-I     : isolate Python from the user's environment (implies -E and -s)

and the code to handle -I does:

        case 'I':
            Py_IsolatedFlag++;
            Py_NoUserSiteDirectory++;
            Py_IgnoreEnvironmentFlag++;
            break;

where Py_NoUserSiteDirectory is the variable corresponding to the -s flag
rather than the -S flag.  But it seems like -I should really imply both
-s and -S.  So I am filing this bug primarily to find out whether or not
it really should be both.  If so, great: a patch is attached; details
below.  But if not, then the man page should be corrected.

The rest of this is written under the assumption that -I should imply -S
as well as -s.

Background: depending on which packages are installed on different Solaris
systems, test_site passes or not.  Certain packages (e.g., dogpile.core,
dogpile.cache, repoze.lru) that have a .pth file with "import types"
which results in test_site.StartupImportTests failing because types has
been imported which is in the list of collections modules, none of which
are expected to be imported.  So we thought "well, -S should fix that"
then noticed the man page saying -I implied -S which is how we got here.

Tweaking the code and man page so -I does imply -S was trivial.  But three
other changes were needed:
1. In test_site.py, test_startup_imports() asserted that 'site' was in the
   list of modules that had been imported.  This is no longer true, so I
   deleted the assert.
2. test_inspect failed because of a name error, that turned out to be
   inspect.py calling exit instead of sys.exit.  So the attached patch
   corrects both of those.  This fix is probably generally applicable
   even if the "-I should imply both -S and -s" assumption turns out to
   be false.
3. test_venv failed because it and the venv module were using -I to imply
   -s and -E but not -S.  Changing three instances of "-Im" to "-Esm"
   (one in Lib/venv/__init__.py, the other two in Lib/test/test_venv.py)
   fixed this.  However, even if the "-I should imply both -S and -s"
   assumption is true, this change may not be desirable in the general
   case, but needed because of Solaris' hacky work-around for issue
   1298835 not yet being fixed.'

   I.e., we ship /usr/lib/python3.4/site-packages/vendor-packages.pth
   with the one line:
   import site; site.addsitedir('/usr/lib/python3.4/vendor-packages')
   (likewise for other versions).

   So this may not be desirable in general, but I mention it for the
   sake of completeness.

----------
components: Interpreter Core
files: 25-site-module.patch
keywords: patch
messages: 242248
nosy: dhduvall, jbeck
priority: normal
severity: normal
status: open
title: man page says -I implies -S. code says -s. Should it be both?
versions: Python 3.4, Python 3.5
Added file: http://bugs.python.org/file39233/25-site-module.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24077>
_______________________________________

From report at bugs.python.org  Thu Apr 30 01:04:32 2015
From: report at bugs.python.org (Siming Yuan)
Date: Wed, 29 Apr 2015 23:04:32 +0000
Subject: [New-bugs-announce] [issue24078] inspect.getsourcelines ignores
	context and returns wrong	line #
Message-ID: <1430348672.59.0.934013457033.issue24078@psf.upfronthosting.co.za>


New submission from Siming Yuan:

if the same class name is used within a module, but defined in different contexts (either class in class or class in function), inspect.getsourcelines() on the class object ignores the object context and only returns the first matched name.

reproduce:

a.py
----
class A(object):
    class B(object):
        pass

class C(object):
    class B(object):
        pass

------------------
>>> import inspect
>>> import a
>>> inspect.getsourcelines(a.C.B)
(['    class B(object):\n', '        pass\n'], 2)

----------
components: Library (Lib)
messages: 242254
nosy: siyuan
priority: normal
severity: normal
status: open
title: inspect.getsourcelines ignores context and returns wrong line #
type: behavior
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24078>
_______________________________________

From report at bugs.python.org  Thu Apr 30 01:34:55 2015
From: report at bugs.python.org (=?utf-8?q?J=C3=A9r=C3=B4me_Laurens?=)
Date: Wed, 29 Apr 2015 23:34:55 +0000
Subject: [New-bugs-announce] [issue24079] xml.etree.ElementTree.Element.text
	does not conform to	the documentation
Message-ID: <1430350495.43.0.91352789573.issue24079@psf.upfronthosting.co.za>


New submission from J?r?me Laurens:

The documentation for xml.etree.ElementTree.Element.text reads "If the element is created from an XML file the attribute will contain any text found between the element tags."

import xml.etree.ElementTree as ET
root3 = ET.fromstring('<a><b/>TEXT</a>')
print(root3.text)

CURRENT OUTPUT

None

"TEXT" is between the elements tags but does not appear in the output

BTW : this is well formed xml and has nothing to do with tail.

----------
components: XML
messages: 242256
nosy: jlaurens
priority: normal
severity: normal
status: open
title: xml.etree.ElementTree.Element.text does not conform to the documentation
type: behavior
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24079>
_______________________________________

From report at bugs.python.org  Thu Apr 30 15:59:58 2015
From: report at bugs.python.org (Matt Johnston)
Date: Thu, 30 Apr 2015 13:59:58 +0000
Subject: [New-bugs-announce] [issue24080] asyncio.Event.wait() Task was
	destroyed but it is pending
Message-ID: <1430402398.63.0.270088920542.issue24080@psf.upfronthosting.co.za>


New submission from Matt Johnston:

asyncio.Event.wait() doesn't seem to be cancelled by asyncio.wait_for(). Ctrl-c in the attached example produces output below. I'm not certain the code is correct though the documentation for wait_for() suggests it should work. Without the wait_for() it doesn't suffer from pending task destruction. This is Python 3.4.3. 


Hello World!
Hello World!
^CTraceback (most recent call last):
  File "./loop2.py", line 27, in <module>
    loop.run_until_complete(hello_world())
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/asyncio/base_events.py", line 304, in run_until_complete
    self.run_forever()
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/asyncio/base_events.py", line 276, in run_forever
    self._run_once()
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/asyncio/base_events.py", line 1136, in _run_once
    event_list = self._selector.select(timeout)
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/selectors.py", line 502, in select
    kev_list = self._kqueue.control(None, max_ev, timeout)
KeyboardInterrupt
Task was destroyed but it is pending!
task: <Task pending coro=<wait() running at /Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/asyncio/locks.py:242> wait_for=<Future pending cb=[Task._wakeup()]> cb=[_release_waiter(<Future pendi...sk._wakeup()]>)() at /Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/asyncio/tasks.py:335]>
[1]    8134 exit 1     ../venv3/bin/python3 ./loop2.py

----------
components: asyncio
files: loop2.py
messages: 242269
nosy: gvanrossum, haypo, matt, yselivanov
priority: normal
severity: normal
status: open
title: asyncio.Event.wait() Task was destroyed but it is pending
versions: Python 3.4
Added file: http://bugs.python.org/file39234/loop2.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24080>
_______________________________________

From report at bugs.python.org  Thu Apr 30 16:12:34 2015
From: report at bugs.python.org (Petr Viktorin)
Date: Thu, 30 Apr 2015 14:12:34 +0000
Subject: [New-bugs-announce] [issue24081] Obsolete caveat in reload() docs
Message-ID: <1430403154.98.0.829139703368.issue24081@psf.upfronthosting.co.za>


New submission from Petr Viktorin:

imp.reload() and importlib.reload() docs state::

    If a module is syntactically correct but its initialization fails, the first
    :keyword:`import` statement for it does not bind its name locally, but does
    store a (partially initialized) module object in ``sys.modules``.  To reload
    the module you must first :keyword:`import` it again (this will bind the name
    to the partially initialized module object) before you can :func:`reload` it.

If I reading that correctly, "initialization" refers to executing the module, so for module containing just::

    uninitialized_variable

the following::

    >>> import sys
    >>> import x
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "/tmp/x.py", line 1, in <module>
        uninitialized_variable
    NameError: name 'uninitialized_variable' is not defined

should leave me with a initialized module in sys.modules['x']. However, this is not what happens, in either Python 3.4 or 2.7::

    >>> sys.modules['x']
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    KeyError: 'x'

Here's a patch to remove the caveat in Python 3 docs.
If I missed something, and "initialization" refers to something else, it should be clarified.

----------
assignee: docs at python
components: Documentation
files: 0001-Remove-obsolete-caveat-from-reload-docs.patch
keywords: patch
messages: 242270
nosy: docs at python, encukou
priority: normal
severity: normal
status: open
title: Obsolete caveat in reload() docs
versions: Python 2.7, Python 3.4
Added file: http://bugs.python.org/file39235/0001-Remove-obsolete-caveat-from-reload-docs.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24081>
_______________________________________

From report at bugs.python.org  Thu Apr 30 16:19:30 2015
From: report at bugs.python.org (Petr Viktorin)
Date: Thu, 30 Apr 2015 14:19:30 +0000
Subject: [New-bugs-announce] [issue24082] Obsolete note in argument parsing
	(c-api/arg.rst)
Message-ID: <1430403570.61.0.800757161817.issue24082@psf.upfronthosting.co.za>


New submission from Petr Viktorin:

A note in the docs for the "u" format unit saus NULs are not allowed, but the previous sentence says they aren't accepted.

----------
assignee: docs at python
components: Documentation
files: 0002-Remove-obsolete-note-in-argument-parsing-docs.patch
keywords: patch
messages: 242271
nosy: docs at python, encukou
priority: normal
severity: normal
status: open
title: Obsolete note in argument parsing (c-api/arg.rst)
versions: Python 3.4
Added file: http://bugs.python.org/file39236/0002-Remove-obsolete-note-in-argument-parsing-docs.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24082>
_______________________________________

From report at bugs.python.org  Thu Apr 30 17:15:48 2015
From: report at bugs.python.org (Marc-Andre Lemburg)
Date: Thu, 30 Apr 2015 15:15:48 +0000
Subject: [New-bugs-announce] [issue24083] MSVCCompiler.get_msvc_path()
	doesn't work on Win x64
Message-ID: <1430406948.34.0.89684475541.issue24083@psf.upfronthosting.co.za>


New submission from Marc-Andre Lemburg:

Trying to use .get_msvc_path() on an distutils.msvccompiler.MSVCCompiler() instance raises an AttributeError:

MSVCCompiler instance has no attribute '_MSVCCompiler__root'

The reason seems to be that self.__root is not set for Win x64 in .__init__().

Here's an example from Python 2.7.9:

>>> from distutils.msvccompiler import *
>>> c = OldMSVCCompiler()
>>> vars(c)
{'force': 0, 'verbose': 0, 'dry_run': 0, 'runtime_library_dirs': [], 'libraries': [], 'macros': [], 'objects': [], 'output_dir': None, '_MSVCCompiler__product': 'Microsoft SDK compiler 15.0', 'initialized': False, '_MSVCCompiler__arch': 'AMD64', '_MSVCCompiler__version': 9.0, 'library_dirs': [], 'include_dirs': []}
>>> c.get_msvc_paths('include')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "D:\Python27\lib\distutils\msvccompiler.py", line 615, in get_msvc_paths
    % (self.__root, self.__version))
AttributeError: MSVCCompiler instance has no attribute '_MSVCCompiler__root'

The newer implementation of MSVCCompiler in msvc9compiler doesn't have this method, so cannot be used to find the include and library paths.

----------
components: Distutils
messages: 242274
nosy: dstufft, eric.araujo, lemburg
priority: normal
severity: normal
status: open
title: MSVCCompiler.get_msvc_path() doesn't work on Win x64
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24083>
_______________________________________

From report at bugs.python.org  Thu Apr 30 18:27:51 2015
From: report at bugs.python.org (Romuald Brunet)
Date: Thu, 30 Apr 2015 16:27:51 +0000
Subject: [New-bugs-announce] [issue24084] pstats: sub-millisecond display
Message-ID: <1430411271.93.0.999844602202.issue24084@psf.upfronthosting.co.za>


New submission from Romuald Brunet:

When running pstats with functions that take less than 0.5 millisecond (per call for example), the value shown is '0.000, which isn't really helpful.

This patch aims to show the value in microseconds instead of seconds for values that would otherwise be displayed as '0.000'

(the display may have to be tweaked for non-utf8 users, as the ? character is used)


Example output:

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
     9827    0.007      7?s    0.007      7?s {method 'get' of 'dict' objects}
     3427    0.009     25?s    0.017     49?s {map}
    10701    0.006      5?s    0.006      5?s {method 'lower' of 'str' objects}
    31410    0.045     14?s    0.045     14?s {method 'split' of 'str' objects}
    10023    0.006      6?s    0.006      6?s {_weakref.proxy}
    31801    0.009      2?s    0.009      2?s {len}
     3892    0.003      7?s    0.003      7?s {method 'extend' of 'list' objects}
     1397    0.001      8?s    0.001      8?s {method 'replace' of 'str' objects}
    10488    0.010      9?s    0.010      9?s {method 'string_literal' of '_mysql.connection' objects}
    10702    2.620   2447?s    2.620   2447?s {method 'readline' of 'file' objects}
    10023    0.004      3?s    0.004      3?s {method 'info' of '_mysql.connection' objects}
    10023    0.005      5?s    0.005      5?s {method 'insert_id' of '_mysql.connection' objects}
    14327    0.022     15?s    0.022     15?s {method 'rstrip' of 'str' objects}

----------
components: Library (Lib)
files: pstats-microseconds.patch
keywords: patch
messages: 242277
nosy: Romuald
priority: normal
severity: normal
status: open
title: pstats: sub-millisecond display
type: enhancement
Added file: http://bugs.python.org/file39237/pstats-microseconds.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24084>
_______________________________________

From report at bugs.python.org  Thu Apr 30 20:59:04 2015
From: report at bugs.python.org (Buck Evan)
Date: Thu, 30 Apr 2015 18:59:04 +0000
Subject: [New-bugs-announce] [issue24085] large memory overhead when pyc is
	recompiled
Message-ID: <1430420344.67.0.572478851339.issue24085@psf.upfronthosting.co.za>


New submission from Buck Evan:

In the attached example I show that there's a significant memory overhead present whenever a pre-compiled pyc is not present.

This only occurs with more than 5225 objects (dictionaries in this case)
allocated. At 13756 objects, the mysterious pyc overhead is 50% of memory
usage.

I've reproduced this issue in python 2.6, 2.7, 3.4. I imagine it's present in all cpythons.


$ python -c 'import repro'
16736
$ python -c 'import repro'
8964
$ python -c 'import repro'
8964

$ rm *.pyc; python -c 'import repro'
16740
$ rm *.pyc; python -c 'import repro'
16736
$ rm *.pyc; python -c 'import repro'
16740

----------
files: repro.py
messages: 242281
nosy: bukzor
priority: normal
severity: normal
status: open
title: large memory overhead when pyc is recompiled
versions: Python 3.4
Added file: http://bugs.python.org/file39238/repro.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24085>
_______________________________________

From report at bugs.python.org  Thu Apr 30 21:12:10 2015
From: report at bugs.python.org (Trevor Bekolay)
Date: Thu, 30 Apr 2015 19:12:10 +0000
Subject: [New-bugs-announce] [issue24086] Configparser interpolation is
	unexpected
Message-ID: <1430421130.2.0.837084015548.issue24086@psf.upfronthosting.co.za>


New submission from Trevor Bekolay:

I was having an issue installing a package in Python 3, which installed properly in Python 2. This is the error message I got:

    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):

    ... snip unhelpful traceback ...

      File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/configparser.py", line 423, in _interpolate_some

        "found: %r" % (rest,))

    configparser.Interpolation

    ----------------------------------------
    Command "python setup.py egg_info" failed with error code 1 in /private/var/folders/12/p4ngkfbx2pnb1ln81csjb19c0000gn/T/pip-build-6xhgg5x6/nengo

This wasn't a super helpful error message. I managed to figure out that Python 3 (or setuptools?) was attempting to parse my ~/.pypirc file, which raised a configparser.InterpolationSyntaxError because my password contained a percent-sign. Configparser doesn't try to interpolate this string in Python 2, so it worked fine. As a workaround, I've changed my PyPI password, but this failure was quite surprising to me.

As for what I would expect to happen, I would not expect interpolation to happen if I don't want it to happen. All of the examples shown on in the docs use the syntax "%(key)s" or "${key}". If this is the only way to interpolate, then why is "%" ambiguous? Surely we can look forward one character, and if it's not "(" then "%" should be interpreted literally. Instead, configparser requires me to disambiguate with "%%". But, this would cause my string to be parsed incorrectly in Python 2's configparser, so I'm unable to make that change.

----------
components: Library (Lib)
messages: 242283
nosy: tbekolay
priority: normal
severity: normal
status: open
title: Configparser interpolation is unexpected
type: behavior
versions: Python 3.2, Python 3.3, Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24086>
_______________________________________

From report at bugs.python.org  Thu Apr 30 22:25:52 2015
From: report at bugs.python.org (Paul Moore)
Date: Thu, 30 Apr 2015 20:25:52 +0000
Subject: [New-bugs-announce] [issue24087] Documentation doesn't explain the
	term "coroutine" (PEP	342)
Message-ID: <1430425552.31.0.779606074615.issue24087@psf.upfronthosting.co.za>


New submission from Paul Moore:

Although the new generator methods introduced in PEP 342 are documented, the term "coroutine" is not defined anywhere. In particular, the fact that Python coroutines work in conjunction with an event loop rather than transferring control directly between each other is not mentioned.

----------
assignee: docs at python
components: Documentation
files: coroutine_docs.patch
keywords: needs review, patch
messages: 242286
nosy: docs at python, gvanrossum, paul.moore
priority: normal
severity: normal
stage: patch review
status: open
title: Documentation doesn't explain the term "coroutine" (PEP 342)
type: enhancement
versions: Python 3.5
Added file: http://bugs.python.org/file39239/coroutine_docs.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24087>
_______________________________________