From report at bugs.python.org  Thu Jan  1 08:58:44 2015
From: report at bugs.python.org (Martin Panter)
Date: Thu, 01 Jan 2015 07:58:44 +0000
Subject: [New-bugs-announce] [issue23142] Integration of
	unittest.FunctionTestCase with automatic	discovery and loading
Message-ID: <1420099124.62.0.985279569808.issue23142@psf.upfronthosting.co.za>


New submission from Martin Panter:

It is not clear how you are meant to use unittest.FunctionTestCase with automatic test running. Unless a simple way to do this already exists, I wonder if it would be okay to automatically discover and run predefined test instances, such as the "test_module.testcase" object in my example module.

Currently the ?testcase? object is ignored by the automatic test discovery. Also, attempting to manually run it results in

$ python -m unittest test_module.testcase
[. . .]
TypeError: calling unittest.case.FunctionTestCase (testSomething) returned returned <unittest.result.TestResult run=1 errors=1 failures=0>, not a test

Apparently calling a TestCase instance invokes TestCase.run()!

The only way I can think of making such a test discoverable is my making a subclass. But as demonstrated by my example module, this is more cumbersome than bypassing FunctionTestCase altogether.

----------
components: Library (Lib)
files: test_module.py
messages: 233276
nosy: vadmium
priority: normal
severity: normal
status: open
title: Integration of unittest.FunctionTestCase with automatic discovery and loading
type: enhancement
versions: Python 3.5
Added file: http://bugs.python.org/file37579/test_module.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23142>
_______________________________________

From report at bugs.python.org  Thu Jan  1 12:55:39 2015
From: report at bugs.python.org (Antoine Pitrou)
Date: Thu, 01 Jan 2015 11:55:39 +0000
Subject: [New-bugs-announce] [issue23143] Remove some conditional code in
	_ssl.c
Message-ID: <1420113339.19.0.823124921107.issue23143@psf.upfronthosting.co.za>


New submission from Antoine Pitrou:

There's a lot of conditional code in _ssl.c, meant to address the unavailability of some features in old OpenSSL versions. I think for 3.5 we should require at least 0.9.8 (which is already old), and consequently remove some of those conditionals.

----------
components: Library (Lib)
keywords: easy
messages: 233280
nosy: pitrou
priority: low
severity: normal
stage: needs patch
status: open
title: Remove some conditional code in _ssl.c
type: enhancement
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23143>
_______________________________________

From report at bugs.python.org  Thu Jan  1 19:47:08 2015
From: report at bugs.python.org (Ross)
Date: Thu, 01 Jan 2015 18:47:08 +0000
Subject: [New-bugs-announce] [issue23144] html.parser.HTMLParser: setting
	'convert_charrefs =	True' leads to dropped text
Message-ID: <1420138028.63.0.996553076818.issue23144@psf.upfronthosting.co.za>


New submission from Ross:

If convert_charrefs is set to true the final data section is not return by feed(). It is held until the next tag is encountered.

---
from html.parser import HTMLParser

class MyHTMLParser(HTMLParser):
    def __init__(self):
        HTMLParser.__init__(self, convert_charrefs=True)
        self.fed = []
    def handle_starttag(self, tag, attrs):
        print("Encountered a start tag:", tag)
    def handle_endtag(self, tag):
        print("Encountered an end tag :", tag)
    def handle_data(self, data):
        print("Encountered some data  :", data)

parser = MyHTMLParser()

parser.feed("foo <a>link</a> bar")
print("")
parser.feed("spam <a>link</a> eggs")

---

gives

Encountered some data  : foo 
Encountered a start tag: a
Encountered some data  : link
Encountered an end tag : a

Encountered some data  :  barspam 
Encountered a start tag: a
Encountered some data  : link
Encountered an end tag : a


With 'convert_charrefs = False' it works as expected.

----------
components: Library (Lib)
messages: 233291
nosy: xkjq
priority: normal
severity: normal
status: open
title: html.parser.HTMLParser: setting 'convert_charrefs = True' leads to dropped text
type: behavior
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23144>
_______________________________________

From report at bugs.python.org  Thu Jan  1 23:38:59 2015
From: report at bugs.python.org (STINNER Victor)
Date: Thu, 01 Jan 2015 22:38:59 +0000
Subject: [New-bugs-announce] [issue23145] regrtest: log test loader errors
Message-ID: <1420151939.5.0.279924205041.issue23145@psf.upfronthosting.co.za>


New submission from STINNER Victor:

Currently, when regrtest fails to load a test submodule, no exception is raised, no error is logged.

I propose to at least log all loader errors: see attached regrtest.patch.

For an example, try to apply the attached test_asyncio_bug.patch and run:
./python -m test -m test_not_implemented test_asyncio

The current output is:
---
[1/1] test_asyncio
1 test OK.
---

The output with regrtest.patch:
---
haypo at selma$ ./python -m test -m test_not_implemented test_asyncio
[1/1] test_asyncio
Failed to import test module: test.test_asyncio.test_base_events
Traceback (most recent call last):
  File "/home/haypo/prog/python/default/Lib/unittest/loader.py", line 417, in _find_test_path
    module = self._get_module_from_name(name)
  File "/home/haypo/prog/python/default/Lib/unittest/loader.py", line 358, in _get_module_from_name
    __import__(name)
  File "/home/haypo/prog/python/default/Lib/test/test_asyncio/test_base_events.py", line 29
    class BaseEventLoopTests(test_utils.TestCase)
                                                ^
SyntaxError: invalid syntax

1 test OK.
---

Or should we raise an exception instead?

----------
components: Tests
files: regrtest.patch
keywords: patch
messages: 233298
nosy: haypo
priority: normal
severity: normal
status: open
title: regrtest: log test loader errors
versions: Python 2.7, Python 3.4, Python 3.5
Added file: http://bugs.python.org/file37583/regrtest.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23145>
_______________________________________

From report at bugs.python.org  Fri Jan  2 10:40:25 2015
From: report at bugs.python.org (Serhiy Storchaka)
Date: Fri, 02 Jan 2015 09:40:25 +0000
Subject: [New-bugs-announce] [issue23146] Incosistency in pathlib between /
	and \
Message-ID: <1420191625.18.0.999730856462.issue23146@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka:

>>> pathlib.PureWindowsPath('c:/a/b', '/x/y')
PureWindowsPath('c:/x/y')
>>> pathlib.PureWindowsPath('//?/c:/a/b', '/x/y')
PureWindowsPath('/x/y')
>>> pathlib.PureWindowsPath(r'\\?\c:\a\b', '/x/y')
PureWindowsPath('//?/c:/x/y')

I suppose this is because in parse_parts() altsep is replaced with sep before calling self.splitroot() in main loop but not in other loop (if drv or root). Line 76.

----------
components: Library (Lib)
messages: 233308
nosy: pitrou, serhiy.storchaka
priority: normal
severity: normal
stage: needs patch
status: open
title: Incosistency in pathlib between / and \
type: behavior
versions: Python 3.4, Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23146>
_______________________________________

From report at bugs.python.org  Fri Jan  2 11:31:19 2015
From: report at bugs.python.org (Serhiy Storchaka)
Date: Fri, 02 Jan 2015 10:31:19 +0000
Subject: [New-bugs-announce] [issue23147] Possible error in
	_header_value_parser.py
Message-ID: <1420194679.08.0.467668822135.issue23147@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka:

In UnstructuredTokenList._fold() at line 439 the fold() method is called with one positional argument.

            if part.has_fws:
                part.fold(folded)
                continue

But there are no fold() methods with one positional parameter. May be it should be _fold()?

----------
components: Library (Lib), email
messages: 233309
nosy: barry, r.david.murray, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Possible error in _header_value_parser.py
type: behavior
versions: Python 3.4, Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23147>
_______________________________________

From report at bugs.python.org  Fri Jan  2 11:38:19 2015
From: report at bugs.python.org (Serhiy Storchaka)
Date: Fri, 02 Jan 2015 10:38:19 +0000
Subject: [New-bugs-announce] [issue23148] Missing the charset parameter in
	as_encoded_word()
Message-ID: <1420195099.17.0.393236049779.issue23148@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka:

In Lib/email/_header_value_parser.py:460 the as_encoded_word() method is called without mandatory argument charset.

----------
components: Library (Lib), email
messages: 233310
nosy: barry, r.david.murray, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Missing the charset parameter in as_encoded_word()
type: behavior
versions: Python 3.4, Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23148>
_______________________________________

From report at bugs.python.org  Fri Jan  2 13:02:17 2015
From: report at bugs.python.org (Jonathan Sharpe)
Date: Fri, 02 Jan 2015 12:02:17 +0000
Subject: [New-bugs-announce] [issue23149] Typo in PEP-0008 - "this PEP do
	not"
Message-ID: <1420200137.59.0.98373116964.issue23149@psf.upfronthosting.co.za>


Changes by Jonathan Sharpe <j.r.sharpe+python at gmail.com>:


----------
assignee: docs at python
components: Documentation
files: fix_pep8_typo.patch
keywords: patch
nosy: docs at python, jonrsharpe
priority: normal
severity: normal
status: open
title: Typo in PEP-0008 - "this PEP do not"
type: enhancement
Added file: http://bugs.python.org/file37585/fix_pep8_typo.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23149>
_______________________________________

From report at bugs.python.org  Fri Jan  2 15:13:47 2015
From: report at bugs.python.org (Julian Reschke)
Date: Fri, 02 Jan 2015 14:13:47 +0000
Subject: [New-bugs-announce] [issue23150] urllib parse incorrect handing of
	params
Message-ID: <1420208027.4.0.951710161178.issue23150@psf.upfronthosting.co.za>


New submission from Julian Reschke:

urllib.parse tries to special-case params, which have been dropped from the general URI syntax back in RFC 2396 (16 years ago).

In most cases this can be worked around by reconstructing the path from both path and params; however this fails for paths that *end* in a semicolon (because it's not possible to distinguish an empty param from an absent param).

----------
components: Library (Lib)
messages: 233312
nosy: julian.reschke at gmx.de
priority: normal
severity: normal
status: open
title: urllib parse incorrect handing of params
type: behavior

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23150>
_______________________________________

From report at bugs.python.org  Fri Jan  2 17:04:33 2015
From: report at bugs.python.org (Serhiy Storchaka)
Date: Fri, 02 Jan 2015 16:04:33 +0000
Subject: [New-bugs-announce] [issue23151] _loggerClass is initialized twice
Message-ID: <1420214673.16.0.25178792027.issue23151@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka:

In Lib/logging/__init__.py at line 1089 _loggerClass is initialized to None. The code in Manager.getLogger() expects that it can be None. But at line 1549 _loggerClass is initialized to Logger. And there is no official way to set it to None. Looks as either initialization to None and the code in Manager.getLogger() are redundant or initialization to Logger is redundant.

----------
components: Library (Lib)
messages: 233313
nosy: serhiy.storchaka, vinay.sajip
priority: normal
severity: normal
status: open
title: _loggerClass is initialized twice

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23151>
_______________________________________

From report at bugs.python.org  Fri Jan  2 21:23:52 2015
From: report at bugs.python.org (Steve Dower)
Date: Fri, 02 Jan 2015 20:23:52 +0000
Subject: [New-bugs-announce] [issue23152] fstat64 required on Windows
Message-ID: <1420230232.09.0.10127303351.issue23152@psf.upfronthosting.co.za>


New submission from Steve Dower:

Currently test_largefile is failing on the Windows buildbots because an fstat() call in Modules/_io/fileio.c is failing. fstat() returns a 32-bit size, but the file being opened is larger than 2GB.

This appears to be a change in the CRT where it would previously succeed with an incorrect value but now returns an error. (While we're here, there is no exception set in this case, so you get "SystemError: NULL result without error in PyObject_Call", but that's not the core issue.)

I can fix this instance, but I suspect we may need to fix this in multiple places. fstat64 (or _fstat64) seems to exist on multiple platforms, and the docs I found (e.g. http://linux.die.net/man/2/fstat64) suggest that EOVERFLOW is always going to occur in this case, so is there any reason not to switch to fstat64 everywhere? Maybe enable it through pyport.h/pyconfig.h and have a #define that is set based on availability?

----------
components: IO
messages: 233327
nosy: benjamin.peterson, hynek, pitrou, steve.dower, stutzbach, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: fstat64 required on Windows
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23152>
_______________________________________

From report at bugs.python.org  Fri Jan  2 23:13:42 2015
From: report at bugs.python.org (John Potelle)
Date: Fri, 02 Jan 2015 22:13:42 +0000
Subject: [New-bugs-announce] [issue23153] Clarify Boolean Clause Results
Message-ID: <1420236822.81.0.553151868836.issue23153@psf.upfronthosting.co.za>


New submission from John Potelle:

>From v3.4 Tutorial section 5.7

It is possible to assign the result of a comparison or other Boolean expression to a variable. For example,
>>> string1, string2, string3 = '', 'Trondheim', 'Hammer Dance'
>>> string1 or string2 or string3
'Trondheim'
>>> bool(string1 or string2 or string3)
True

In most languages a Boolean clause (comparison) returns a Boolean or it's equivalent, not the last argument evaluated.  Please add a reference to bool function (or some better? method) in this section for clarification.

----------
assignee: docs at python
components: Documentation
messages: 233330
nosy: docs at python, jpotelle
priority: normal
severity: normal
status: open
title: Clarify Boolean Clause Results
type: enhancement
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23153>
_______________________________________

From report at bugs.python.org  Sat Jan  3 13:28:23 2015
From: report at bugs.python.org (Mark Lawrence)
Date: Sat, 03 Jan 2015 12:28:23 +0000
Subject: [New-bugs-announce] [issue23154] MSVC 2013 Express needlessly
	rebuilds code
Message-ID: <1420288103.47.0.693564742808.issue23154@psf.upfronthosting.co.za>


New submission from Mark Lawrence:

I've suspected that this is the case for some time but I've confirmed it this morning.  I ran synchronize and the highest revision was 94004 "Changes %s to %ls in wprintf in launcher.c for C99 compatibility."  As expected MSVC rebuilt the launcher.  Later I reran synchronize and the highest revision was 94009 "Update bundled pip and setuptools to 6.0.6 and 11.0.".  The output from the 64 bit Release build concluded "31 succeeded, 0 failed, 5 up-to-date, 1 skipped".  There also seems to be a toggle operating between the Release and Debug builds such that only one rebuilds at any one time.

----------
components: Build, Windows
messages: 233355
nosy: BreamoreBoy, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: MSVC 2013 Express needlessly rebuilds code
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23154>
_______________________________________

From report at bugs.python.org  Sat Jan  3 20:38:03 2015
From: report at bugs.python.org (Thomas Klausner)
Date: Sat, 03 Jan 2015 19:38:03 +0000
Subject: [New-bugs-announce] [issue23155] unittest: object has no attribute
	'_removed_tests'
Message-ID: <1420313883.47.0.122520965169.issue23155@psf.upfronthosting.co.za>


New submission from Thomas Klausner:

On NetBSD with python-3.4.2 I see the following issue when running the tests for py-flake8-2.2.5:

running build_ext
test_get_parser (flake8.tests.test_engine.TestEngine) ... ok
test_get_python_version (flake8.tests.test_engine.TestEngine) ... ok
test_get_style_guide (flake8.tests.test_engine.TestEngine) ... ok
test_get_style_guide_kwargs (flake8.tests.test_engine.TestEngine) ... ok
test_register_extensions (flake8.tests.test_engine.TestEngine) ... ok
test_stdin_disables_jobs (flake8.tests.test_engine.TestEngine) ... ok
test_windows_disables_jobs (flake8.tests.test_engine.TestEngine) ... ok
Traceback (most recent call last):
  File "setup.py", line 74, in <module>
    test_suite='nose.collector',
  File "/usr/pkg/lib/python3.4/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/usr/pkg/lib/python3.4/distutils/dist.py", line 955, in run_commands
    self.run_command(cmd)
  File "/usr/pkg/lib/python3.4/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/usr/pkg/lib/python3.4/site-packages/setuptools/command/test.py", line 142, in run
    self.with_project_on_sys_path(self.run_tests)
  File "/usr/pkg/lib/python3.4/site-packages/setuptools/command/test.py", line 122, in with_project_on_sys_path
    func()
  File "/usr/pkg/lib/python3.4/site-packages/setuptools/command/test.py", line 163, in run_tests
    testRunner=self._resolve_as_ep(self.test_runner),
  File "/usr/pkg/lib/python3.4/unittest/main.py", line 93, in __init__
    self.runTests()
  File "/usr/pkg/lib/python3.4/unittest/main.py", line 244, in runTests
    self.result = testRunner.run(self.test)
  File "/usr/pkg/lib/python3.4/unittest/runner.py", line 168, in run
    test(result)
  File "/usr/pkg/lib/python3.4/unittest/suite.py", line 87, in __call__
    return self.run(*args, **kwds)
  File "/usr/pkg/lib/python3.4/unittest/suite.py", line 130, in run
    self._removeTestAtIndex(index)
  File "/usr/pkg/lib/python3.4/unittest/suite.py", line 83, in _removeTestAtIndex
    self._removed_tests += test.countTestCases()
  File "/usr/pkg/lib/python3.4/unittest/suite.py", line 41, in countTestCases
    cases = self._removed_tests
AttributeError: 'FinalizingSuiteWrapper' object has no attribute '_removed_tests'
*** Error code 1


I have reported this

https://gitlab.com/pycqa/flake8/issues/19#note_712215

and Ian Cordasco said this looks like a bug in the unittest module, not py-flake8.

----------
components: Extension Modules
messages: 233365
nosy: wiz
priority: normal
severity: normal
status: open
title: unittest: object has no attribute '_removed_tests'
type: behavior
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23155>
_______________________________________

From report at bugs.python.org  Sat Jan  3 22:55:24 2015
From: report at bugs.python.org (Terry J. Reedy)
Date: Sat, 03 Jan 2015 21:55:24 +0000
Subject: [New-bugs-announce] [issue23156] Update tix install information in
	tkinter tix chapter of	doc
Message-ID: <1420322124.69.0.0233808198239.issue23156@psf.upfronthosting.co.za>


New submission from Terry J. Reedy:

Update tix install info in doc.  "Using tix" starts with 3 lines for testing one's tix install and continues 

'''If this fails, you have a Tk installation problem which must be resolved before proceeding. Use the environment variable TIX_LIBRARY to point to the installed Tix library directory, and make sure you have the dynamic object library (tix8183.dll or libtix8183.so) in the same directory that contains your Tk dynamic object library (tk8183.dll or libtk8183.so). The directory with the dynamic object library should also have a file called pkgIndex.tcl (case sensitive), which contains the line: package ifneeded Tix 8.1 [list load "[file join $dir tix8183.dll]" Tix]'''

Almost nothing above matches my working-with-tix 3.4.2 Win 7 install.  I do have a tix library directory: python34/tcl/tix8.4.3, but the version number is much newer.  Since it is in the right place, TIX_LIBRARY is not needed and there is none.  python34/DLLs contains tcl86t.dll and tk86t.dll and NO tix####.dll.  Is the once separate tix dll now part of tk dll?  I cannot find pkgIndex.tcl; it is certainly not in the DLLs directory nor in the /tcl.

The current doc seems useless to people who do not have tix working.  See, for example,
https://stackoverflow.com/questions/27751923/tix-widgets-installation-issue
which is a semi-repeat question and which claims seeing similar reports elsewhere on the net.

----------
assignee: docs at python
components: Documentation, Tkinter
messages: 233368
nosy: docs at python, serhiy.storchaka, terry.reedy, zach.ware
priority: normal
severity: normal
stage: needs patch
status: open
title: Update tix install information in tkinter tix chapter of doc
type: behavior
versions: Python 2.7, Python 3.4, Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23156>
_______________________________________

From report at bugs.python.org  Sat Jan  3 23:29:08 2015
From: report at bugs.python.org (Antoine Pitrou)
Date: Sat, 03 Jan 2015 22:29:08 +0000
Subject: [New-bugs-announce] [issue23157] Lib/test/time_hashlib.py doesn't
	work
Message-ID: <1420324148.28.0.0179005395687.issue23157@psf.upfronthosting.co.za>


New submission from Antoine Pitrou:

I suppose it was totally forgotten in the transition to 3.x, and nobody appears to have complained. Perhaps we should remove it.

----------
components: Demos and Tools, Library (Lib)
messages: 233375
nosy: christian.heimes, gregory.p.smith, pitrou
priority: low
severity: normal
status: open
title: Lib/test/time_hashlib.py doesn't work
type: behavior
versions: Python 3.4, Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23157>
_______________________________________

From report at bugs.python.org  Sun Jan  4 02:00:44 2015
From: report at bugs.python.org (Al Sweigart)
Date: Sun, 04 Jan 2015 01:00:44 +0000
Subject: [New-bugs-announce] [issue23158] IDLE's help.txt "corrent" typo
Message-ID: <1420333244.7.0.623275484679.issue23158@psf.upfronthosting.co.za>


New submission from Al Sweigart:

There is a typo in IDLE's help.txt file:

"into the corrent number of spaces"

----------
messages: 233379
nosy: Al.Sweigart
priority: normal
severity: normal
status: open
title: IDLE's help.txt "corrent" typo
type: behavior
versions: Python 3.4, Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23158>
_______________________________________

From report at bugs.python.org  Sun Jan  4 08:07:43 2015
From: report at bugs.python.org (Eric McDonald)
Date: Sun, 04 Jan 2015 07:07:43 +0000
Subject: [New-bugs-announce] [issue23159] argparse: Provide equivalent
	of	optparse.OptionParser.{option_groups, option_list, get_option}
Message-ID: <1420355263.97.0.969710352141.issue23159@psf.upfronthosting.co.za>


New submission from Eric McDonald:

There are several use cases for having the equivalent of the optparse.OptionParser 'get_option' method and the 'option_groups' and 'option_list' properties in argparse.ArgumentParser class.

(1) Easy alteration of the text of the default help option. With optparse, one could write:

>>> oparser.get_option( "-h" ).help
'show this help message and exit'
>>> oparser.get_option( "-h" ).help = "Show this help message and exit."
>>> oparser.get_option( "-h" ).help
'Show this help message and exit.'

The equivalent facility does not appear to exist in argparse. (Issue #19462, http://bugs.python.org/issue19462, is about a different use case. And, while https://docs.python.org/3/library/argparse.html#add-help suggests a workaround with add_help=False and then creating a new option with the 'help' action, it is still less convenient than altering the existing help option.)

(2) Inspection of all the argument declarations in an ArgumentParser object after it has been populated. This is particularly useful if you want to look for the equivalent of the available options in config files and don't want to have write explicit code which separately enumerates the available config file keys and how to handle them. With an OptionParser, one could use the 'option_groups' attribute to find all option groups (to correspond to config file sections) and the 'option_list' attribute to find all options (to correspond to config file keys); these are all part of the published interface and provide relatively simple data types to inspect. With an Argument Parser, one needs to rely upon the unpublished interface (the '_actions' attribute of a parser, etc...).

----------
components: Library (Lib)
messages: 233394
nosy: emcd
priority: normal
severity: normal
status: open
title: argparse: Provide equivalent of optparse.OptionParser.{option_groups,option_list,get_option}
type: enhancement
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23159>
_______________________________________

From report at bugs.python.org  Sun Jan  4 08:35:09 2015
From: report at bugs.python.org (Anselm Kruis)
Date: Sun, 04 Jan 2015 07:35:09 +0000
Subject: [New-bugs-announce] [issue23160] Respect the environment variable
	SVNROOT in	external-common.bat
Message-ID: <1420356909.91.0.987880987929.issue23160@psf.upfronthosting.co.za>


New submission from Anselm Kruis:

Issue #21907 introduced the environment variable SVNROOT in 
PCbuild/get_externals.bat.

I propose to use the same variable in Tools/buildbot/external-common.bat too. This batch contains many verbatim copies of the SVN-URL.
With the provided patch, it would become much simpler to use a (local) mirror of svn.python.org.

----------
components: Build
messages: 233395
nosy: anselm.kruis
priority: normal
severity: normal
status: open
title: Respect the environment variable SVNROOT in external-common.bat
type: enhancement
versions: Python 2.7, Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23160>
_______________________________________

From report at bugs.python.org  Sun Jan  4 08:57:16 2015
From: report at bugs.python.org (Devin Jeanpierre)
Date: Sun, 04 Jan 2015 07:57:16 +0000
Subject: [New-bugs-announce] [issue23161] collections.abc.MutableSet missing
	methods
Message-ID: <1420358236.89.0.629550370238.issue23161@psf.upfronthosting.co.za>


New submission from Devin Jeanpierre:

>>> set(dir(set)) - set(dir(collections.abc.MutableSet))
{'copy', 'update', '__rsub__', 'union', 'issubset', 'intersection', 'issuperset', '__rxor__', 'difference', 'symmetric_difference', 'difference_update', '__rand__', 'intersection_update', 'symmetric_difference_update', '__ror__'}

Most of these should be implemented on MutableSet rather than subclasses.

----------
messages: 233398
nosy: Devin Jeanpierre
priority: normal
severity: normal
status: open
title: collections.abc.MutableSet missing methods
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23161>
_______________________________________

From report at bugs.python.org  Sun Jan  4 09:05:06 2015
From: report at bugs.python.org (Devin Jeanpierre)
Date: Sun, 04 Jan 2015 08:05:06 +0000
Subject: [New-bugs-announce] [issue23162] collections.abc sequences don't
	check identity before	equality
Message-ID: <1420358706.85.0.46487014677.issue23162@psf.upfronthosting.co.za>


New submission from Devin Jeanpierre:

For sequence __contains__ and other scenarios, identity is checked before equality, which I've heard is so that "for x in y: assert x in y" doesn't ever fail with an AssertionError (even with NaN and so on). This is not the case for collections.abc-based sequences, which is a jarring inconsistency.

----------
components: Library (Lib)
messages: 233399
nosy: Devin Jeanpierre
priority: normal
severity: normal
status: open
title: collections.abc sequences don't check identity before equality
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23162>
_______________________________________

From report at bugs.python.org  Sun Jan  4 11:21:23 2015
From: report at bugs.python.org (Karl Richter)
Date: Sun, 04 Jan 2015 10:21:23 +0000
Subject: [New-bugs-announce] [issue23163] pdb docs need to contain a
	statement on	threads/multithreaded debugging
Message-ID: <1420366883.83.0.11703664982.issue23163@psf.upfronthosting.co.za>


New submission from Karl Richter:

Due to the fact that `pdb` currently simply ignores breakpoints which are set and hit in another than the main thread the docs need to contain a statement on behavior in a multithreaded environment.

----------
components: Library (Lib)
messages: 233409
nosy: krichter
priority: normal
severity: normal
status: open
title: pdb docs need to contain a statement on threads/multithreaded debugging
type: enhancement
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23163>
_______________________________________

From report at bugs.python.org  Sun Jan  4 14:19:32 2015
From: report at bugs.python.org (Eric O. LEBIGOT)
Date: Sun, 04 Jan 2015 13:19:32 +0000
Subject: [New-bugs-announce] [issue23164] "pydoc filter" documentation
	restrictive
Message-ID: <1420377572.43.0.335658121107.issue23164@psf.upfronthosting.co.za>


New submission from Eric O. LEBIGOT:

The pydoc documentation for filter reads:

    filter(function or None, sequence) -> list, tuple, or string
    
    Return those items of sequence for which function(item) is true.  If
    function is None, return the items that are true.  If sequence is a tuple
    or string, return the same type, else return a list.

It would be nicer to know (e.g. when offline and with no local access to the HTML documentation) that filter() can actually be used more generally with an iterable:

    filter(function or None, **iterable**) -> list, tuple, or string
    
    Return those items of **iterable** for which?

----------
assignee: docs at python
components: Documentation
messages: 233416
nosy: docs at python, lebigot
priority: normal
severity: normal
status: open
title: "pydoc filter" documentation restrictive
type: enhancement
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23164>
_______________________________________

From report at bugs.python.org  Sun Jan  4 17:48:31 2015
From: report at bugs.python.org (Guido Vranken)
Date: Sun, 04 Jan 2015 16:48:31 +0000
Subject: [New-bugs-announce] [issue23165] Heap overwrite in
	Python/fileutils.c:_Py_char2wchar() on	32 bit systems due to
	malloc parameter overflow
Message-ID: <1420390111.83.0.411085812385.issue23165@psf.upfronthosting.co.za>


New submission from Guido Vranken:

The vulnerability described here is exceedingly difficult to exploit, since there is no straight-forward way an "attacker" (someone who controls a Python script contents but not other values such as system environment variables), can control a relevant parameter to the vulnerable function (_Py_char2wchar in Python/fileutils.c). It is, however, important that it is remediated since unawareness of this vulnerability may cause an unsuspecting author to establish a link between user and the function parameter in future versions of Python.

Like I said, the vulnerability is caused by code in the _Py_char2wchar function. Indirectly this function is accessed through Objects/unicodeobject.c:PyUnicode_DecodeLocaleAndSize(), PyUnicode_DecodeFSDefaultAndSize(), PyUnicode_DecodeLocale, and some other functions.

As far as I know this can only be exploited on 32-bit architectures (whose overflow threshold of its registers is  2**32). The following description sets out from the latest Python 3.4 code retrieved from https://hg.python.org/cpython .

The problem lies in the computation of size of the buffer that will hold the wide char version of the input string:

--
Python/fileutils.c
--
 296 #ifdef HAVE_BROKEN_MBSTOWCS
 297     /* Some platforms have a broken implementation of
 298      * mbstowcs which does not count the characters that
 299      * would result from conversion.  Use an upper bound.
 300      */
 301     argsize = strlen(arg);
 302 #else
 303     argsize = mbstowcs(NULL, arg, 0);
 304 #endif
 ...
 ...
 306         res = (wchar_t *)PyMem_RawMalloc((argsize+1)*sizeof(wchar_t));

 and:

 331     argsize = strlen(arg) + 1;
 332     res = (wchar_t*)PyMem_RawMalloc(argsize*sizeof(wchar_t));

Both invocations to PyMem_RawMalloc are not preceded by code that asserts no overflow will occur as a result of multiplication of the length of 'arg' by sizeof(wchar_t), which is typically 4 bytes. It follows that on a 32-bit architecture, it is possible cause an internal overflow to occur through the supplication of a string whose size is >= ((2**32)-1) / 4, which is 1 gigabyte. The supplication of a 1 GB (minus one byte) string will therefore result in a value of 0 being passed to PyMem_RawMalloc, because:

        argsize = 1024*1024*1024-1
        malloc_argument = ((argsize+1) * 4
        print malloc_argument & 0xFFFFFFFF
        # prints '0'
        
Effectively this will result in an allocation of exactly 1 byte, since a parameter of 0 is automatically adjusted to 1 by the underlying _PyMem_RawMalloc():

--
Objects/obmalloc.c
--
  51 static void *
  52 _PyMem_RawMalloc(void *ctx, size_t size)
  53 {
  54     /* PyMem_Malloc(0) means malloc(1). Some systems would return NULL
  55        for malloc(0), which would be treated as an error. Some platforms would
  56        return a pointer with no memory behind it, which would break pymalloc.
  57        To solve these problems, allocate an extra byte. */
  58     if (size == 0)
  59         size = 1;
  60     return malloc(size);
  61 }


Once the memory has been allocated, mbstowcs() is invoked:

--
Python/fileutils.c
--

 306         res = (wchar_t *)PyMem_RawMalloc((argsize+1)*sizeof(wchar_t));
 307         if (!res)
 308             goto oom;
 309         count = mbstowcs(res, arg, argsize+1);

In my test setup (latest 32 bit Debian), mbstowcs returns '0', meaning no bytes were written to 'res'.

Then, 'res' is iterated over and the iteration is halted as soon as a null-wchar or a wchar which is a surrogate:

--
Python/fileutils.c
--

 310         if (count != (size_t)-1) {
 311             wchar_t *tmp;
 312             /* Only use the result if it contains no
 313                surrogate characters. */
 314             for (tmp = res; *tmp != 0 &&
 315                          !Py_UNICODE_IS_SURROGATE(*tmp); tmp++)
 316                 ;
 317             if (*tmp == 0) {
 318                 if (size != NULL)
 319                     *size = count;
 320                 return res;
 321             }
 322         }
 323         PyMem_RawFree(res);


Py_UNICODE_IS_SURROGATE is defined as follows:

--
Include/unicodeobject.h
--
 183 #define Py_UNICODE_IS_SURROGATE(ch) (0xD800 <= (ch) && (ch) <= 0xDFFF)

In the iteration over 'res', control is transferred back to the invoker of _Py_char2wchar() if a null-wchar is encountered first. If, however, a wchar that does satisfies the expression in Py_UNICODE_IS_SURROGATE() is encountered first, *tmp is not null and thus the conditional code on lines 318-320 is skipped.
The space that 'res' points to is unintialized. Uninitialized, however, does not not entail randomness in this case. If an attacker has sufficient freedom to manipulate the contents of the process memory prior to calling _Py_char2wchar() in order to scatter it with values that satisfy Py_UNICODE_IS_SURROGATE(), this could increase their odds of having _Py_char2wchar() encounter such a value before a null-wchar. These kinds of details are very dependant on system architecture, operating system, libc implementation and so forth.

The remainder of the function will perform a byte-per-byte conversion embedded in a loop, to manually convert the entire input string. Especially relevant to this vulnerability are lines 332, 339, 356 and 365, 366:

On line 332 memory is allocated, effectively only 1 byte as explained above. 'argsize', however, is 0x40000000 in our case, and the entire routine is repeated until argsize is 0.
On line 339 one or more characters are converted, and stored into 'out', which is 'res'. Lines 356 and 366 do the same.

--
Python/fileutils.c
--
 325     /* Conversion failed. Fall back to escaping with surrogateescape. */
 326 #ifdef HAVE_MBRTOWC
 327     /* Try conversion with mbrtwoc (C99), and escape non-decodable bytes. */
 328
 329     /* Overallocate; as multi-byte characters are in the argument, the
 330        actual output could use less memory. */
 331     argsize = strlen(arg) + 1;
 332     res = (wchar_t*)PyMem_RawMalloc(argsize*sizeof(wchar_t));
 333     if (!res)
 334         goto oom;
 335     in = (unsigned char*)arg;
 336     out = res;
 337     memset(&mbs, 0, sizeof mbs);
 338     while (argsize) {
 339         size_t converted = mbrtowc(out, (char*)in, argsize, &mbs);
 340         if (converted == 0)
 341             /* Reached end of string; null char stored. */
 342             break;
 343         if (converted == (size_t)-2) {
 344             /* Incomplete character. This should never happen,
 345                since we provide everything that we have -
 346                unless there is a bug in the C library, or I
 347                misunderstood how mbrtowc works. */
 348             PyMem_RawFree(res);
 349             if (size != NULL)
 350                 *size = (size_t)-2;
 351             return NULL;
 352         }
 353         if (converted == (size_t)-1) {
 354             /* Conversion error. Escape as UTF-8b, and start over
 355                in the initial shift state. */
 356             *out++ = 0xdc00 + *in++;
 357             argsize--;
 358             memset(&mbs, 0, sizeof mbs);
 359             continue;
 360         }
 361         if (Py_UNICODE_IS_SURROGATE(*out)) {
 362             /* Surrogate character.  Escape the original
 363                byte sequence with surrogateescape. */
 364             argsize -= converted;
 365             while (converted--)
 366                 *out++ = 0xdc00 + *in++;
 367             continue;
 368         }
 369         /* successfully converted some bytes */
 370         in += converted;
 371         argsize -= converted;
 372         out++;
 373     }
 374     if (size != NULL)
 375         *size = out - res;
 376 #else   /* HAVE_MBRTOWC */
 377     /* Cannot use C locale for escaping; manually escape as if charset
 378        is ASCII (i.e. escape all bytes > 128. This will still roundtrip
 379        correctly in the locale's charset, which must be an ASCII superset. */
 380     res = decode_ascii_surrogateescape(arg, size);
 381     if (res == NULL)
 382         goto oom;
 383 #endif   /* HAVE_MBRTOWC */

Suffice it to say that this leads to writing to memory that has not been allocated, thereby making this a heap overflow vulnerability. decode_ascii_surrogateescape() seems to suffer from the same issue.

Guido Vranken,

Intelworks
http://www.intelworks.com/

----------
files: _py_char2wchar_patches.tar.gz
messages: 233424
nosy: Guido
priority: normal
severity: normal
status: open
title: Heap overwrite in Python/fileutils.c:_Py_char2wchar() on 32 bit systems due to malloc parameter overflow
type: security
versions: Python 3.2, Python 3.3, Python 3.4
Added file: http://bugs.python.org/file37597/_py_char2wchar_patches.tar.gz

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23165>
_______________________________________

From report at bugs.python.org  Sun Jan  4 22:13:57 2015
From: report at bugs.python.org (Jurijs Vorotilovs)
Date: Sun, 04 Jan 2015 21:13:57 +0000
Subject: [New-bugs-announce] [issue23166] urllib2 ignores opener
	configuration under certain	circumstances
Message-ID: <1420406037.0.0.986455895168.issue23166@psf.upfronthosting.co.za>


New submission from Jurijs Vorotilovs:

Python 2.7.9 has a bug in urllib2.py:urlopen().
It creates HTTPSHandler instances by its own when it should not.
One may have assigned custom openers with subclassed HTTPSHandler or HTTPSHandler instance with debug enabled or etc.

----------
components: Library (Lib)
messages: 233427
nosy: crazyjurich
priority: normal
severity: normal
status: open
title: urllib2 ignores opener configuration under certain circumstances
type: behavior
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23166>
_______________________________________

From report at bugs.python.org  Sun Jan  4 23:12:26 2015
From: report at bugs.python.org (Dmitry Kazakov)
Date: Sun, 04 Jan 2015 22:12:26 +0000
Subject: [New-bugs-announce] [issue23167] Marshal docs say format version is
	3; actual format version is 4
Message-ID: <1420409546.12.0.515201542018.issue23167@psf.upfronthosting.co.za>


New submission from Dmitry Kazakov:

Documentation on marshal module says that format version is 3, but Py_MARSHAL_VERSION is set to 4. Search for marshal-related issues gave me an idea that this is a documentation bug.

----------
assignee: docs at python
components: Documentation
files: marshal_doc.diff
keywords: patch
messages: 233429
nosy: docs at python, vlth
priority: normal
severity: normal
status: open
title: Marshal docs say format version is 3; actual format version is 4
versions: Python 3.4
Added file: http://bugs.python.org/file37598/marshal_doc.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23167>
_______________________________________

From report at bugs.python.org  Mon Jan  5 13:18:38 2015
From: report at bugs.python.org (Dmitrijs Ledkovs)
Date: Mon, 05 Jan 2015 12:18:38 +0000
Subject: [New-bugs-announce] [issue23168] test_file2k.py depends on
	sys.stdin being unseekable
Message-ID: <1420460318.93.0.946671821792.issue23168@psf.upfronthosting.co.za>


New submission from Dmitrijs Ledkovs:

# LD_LIBRARY_PATH=`pwd` ./python Lib/test/regrtest.py test_file2k </dev/null
[1/1] test_file2k
test test_file2k failed -- Traceback (most recent call last):
  File "/builddir/build/BUILD/Python-2.7.9/Lib/test/test_file2k.py", line 236, in testStdin
    self.assertRaises(IOError, sys.stdin.seek, -1)
AssertionError: IOError not raised

1 test failed:
    test_file2k


I believe this is similar issue to http://bugs.python.org/issue14853

----------
messages: 233459
nosy: xnox
priority: normal
severity: normal
status: open
title: test_file2k.py depends on sys.stdin being unseekable

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23168>
_______________________________________

From report at bugs.python.org  Mon Jan  5 14:18:11 2015
From: report at bugs.python.org (Radek Simko)
Date: Mon, 05 Jan 2015 13:18:11 +0000
Subject: [New-bugs-announce] [issue23169] Reflect that PreReq and
	BuildPreReq are deprecated in	the latest RPM
Message-ID: <1420463891.72.0.774842309991.issue23169@psf.upfronthosting.co.za>


New submission from Radek Simko:

When I try to make custom package of Python 2.7, I use the spec file attached in `/Misc/RPM`. I don't really use it to build Python as I want to define some specific options, but I do use it as a source of RPM package meta data which I can simply reuse when creating the final RPM, e.g.:

rpm -q --specfile ./Misc/RPM/python-2.7.spec --queryformat '%{DESCRIPTION}'

Manipulation with that specfile in RPM 4.4.2.3 (CentOS 5.6) is fine, but there are warnings in RPM 4.8.0 (CentOS 6.5):

warning: line 71: buildprereq is deprecated: BuildPrereq: expat-devel
warning: line 72: buildprereq is deprecated: BuildPrereq: db4-devel
warning: line 73: buildprereq is deprecated: BuildPrereq: gdbm-devel
warning: line 74: buildprereq is deprecated: BuildPrereq: sqlite-devel
warning: line 91: prereq is deprecated: Prereq: python2.6 = %{PACKAGE_VERSION}
warning: line 122: prereq is deprecated: Prereq: python2.6 = %{PACKAGE_VERSION}-1pydotorg

Here's related thread: https://groups.google.com/forum/#!topic/comp.lang.python/R8ahiZ5wyhc
and most importantly here's proof/explanation: http://www.rpm.org/wiki/Releases/4.8.0#Packagebuilding

 - PreReq and BuildPreReq are now officially deprecated, with warnings at build-time
 - PreReq is mapped to Requires(pre,preun) at build-time

Requires(pre,preun) is backwards compatible, so it works in RPM 4.4 too, I would therefore suggest to change "[Build]Prereq" to "[Build]Requires(pre,preun)" respectively in the specfile to reflect this.

I'm happy to send a patch if some maintainers will agree with proposed solution.

----------
components: Build
messages: 233462
nosy: radeksimko
priority: normal
severity: normal
status: open
title: Reflect that PreReq and BuildPreReq are deprecated in the latest RPM
type: enhancement
versions: Python 2.7, Python 3.2, Python 3.3, Python 3.4, Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23169>
_______________________________________

From report at bugs.python.org  Mon Jan  5 18:12:35 2015
From: report at bugs.python.org (Dimitri John Ledkov)
Date: Mon, 05 Jan 2015 17:12:35 +0000
Subject: [New-bugs-announce] [issue23170] test_uuid test_ifconfig_getnode
	fails with Temporary	failure in name resolution
Message-ID: <1420477955.33.0.233419644536.issue23170@psf.upfronthosting.co.za>


New submission from Dimitri John Ledkov:

Building 3.4.2, running testsuite on linux,
test_uuid test_ifconfig_getnode fails as following:

[365/388] test_uuid
Warning -- sys.path was modified by test_site
test test_uuid failed -- Traceback (most recent call last):
  File "/builddir/build/BUILD/Python-3.4.2/Lib/test/test_uuid.py", line 318, in test_ifconfig_getnode
    node = uuid._ifconfig_getnode()
  File "/builddir/build/BUILD/Python-3.4.2/Lib/uuid.py", line 356, in _ifconfig_getnode
    ip_addr = socket.gethostbyname(socket.gethostname())
socket.gaierror: [Errno -3] Temporary failure in name resolution

Note that "Use of the 'network' resource not enabled"

----------
components: Installation
messages: 233467
nosy: xnox
priority: normal
severity: normal
status: open
title: test_uuid test_ifconfig_getnode fails with Temporary failure in name resolution
type: compile error
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23170>
_______________________________________

From report at bugs.python.org  Mon Jan  5 18:31:42 2015
From: report at bugs.python.org (Jon Dufresne)
Date: Mon, 05 Jan 2015 17:31:42 +0000
Subject: [New-bugs-announce] [issue23171] csv.writer.writerow() does not
	accept generator (must be	coerced to list)
Message-ID: <1420479102.66.0.846752030326.issue23171@psf.upfronthosting.co.za>


New submission from Jon Dufresne:

The csv.writer.writerow() does not accept a generator as input. I find this counter-intuitive and against the spirit of similar APIs. If the generator is coerced to a list, everything works as expected. See the following test script which fails on the line "w.writerow(g)". In my opinion, this line should work identically to the line "w.writerow(list(g))".

---
import csv

f = open('foo.csv', 'w')
w = csv.writer(f)
g = (i for i in ['a', 'b', 'c'])
w.writerow(list(g))
g = (i for i in ['a', 'b', 'c'])
w.writerow(g)
---

----------
components: Library (Lib)
messages: 233470
nosy: jdufresne
priority: normal
severity: normal
status: open
title: csv.writer.writerow() does not accept generator (must be coerced to list)
type: behavior
versions: Python 3.3

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23171>
_______________________________________

From report at bugs.python.org  Mon Jan  5 22:12:41 2015
From: report at bugs.python.org (Wayne Song)
Date: Mon, 05 Jan 2015 21:12:41 +0000
Subject: [New-bugs-announce] [issue23172] Bad file descriptor error occurs
	if random library is	imported before closing FDs
Message-ID: <1420492361.28.0.389548963786.issue23172@psf.upfronthosting.co.za>


New submission from Wayne Song:

The following script:

import os
import resource
import random

print("1")

for fd in range(resource.getrlimit(resource.RLIMIT_NOFILE)[0]):
    try:
        if fd not in range(0, 3):
            os.close(fd)
    except os.error:
        pass

print("2")
print(os.urandom(32))
print("3")

Crashes with the following output:

1
2
Traceback (most recent call last):
  File "test.py", line 17, in <module>
    print(os.urandom(32))
OSError: [Errno 9] Bad file descriptor

On an Ubuntu 14.04 install (in VirtualBox).  It seems to run correctly on Mac OS X.  The script runs fine if I don't import random at the top.

----------
components: Library (Lib)
messages: 233478
nosy: waynesong
priority: normal
severity: normal
status: open
title: Bad file descriptor error occurs if random library is imported before closing FDs
type: behavior
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23172>
_______________________________________

From report at bugs.python.org  Tue Jan  6 01:00:45 2015
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 06 Jan 2015 00:00:45 +0000
Subject: [New-bugs-announce] [issue23173] asyncio: kill the subprocess if
	the creation failed
Message-ID: <1420502445.76.0.9797827832.issue23173@psf.upfronthosting.co.za>


New submission from STINNER Victor:

The creation of a subprocess in asyncio is complex, it is composed of multiple steps (callbacks scheduled by call_soon, coroutines, etc.). The creation can fail because of different reasons. I guess that the most common reason is a cancellation of a task, explicitly or because of a timeout (asyncio.wait_for).

I propose to ensure that everything is cleaned up on failure, especially to kill the subprocess. This choice comes from the subprocess module, it kills the subprocess on error.

Attached patch implements this change. It also fixes a bug noticed in the issue #23140: a call to waiter.set_result() in SubprocessStreamProtocol.connection_made() is not protected by a test on the future state.

When I wrote the unit tests, I tried to mock subprocess.Popen to ensure that the Popen.kill() method is called, but mocking a subprocess for asyncio is complex. I chose to keep the test simple.

----------
components: asyncio
files: asyncio_subprocess_kill.patch
keywords: patch
messages: 233486
nosy: gvanrossum, haypo, yselivanov
priority: normal
severity: normal
status: open
title: asyncio: kill the subprocess if the creation failed
versions: Python 3.4, Python 3.5
Added file: http://bugs.python.org/file37608/asyncio_subprocess_kill.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23173>
_______________________________________

From report at bugs.python.org  Tue Jan  6 01:05:22 2015
From: report at bugs.python.org (Karl Richter)
Date: Tue, 06 Jan 2015 00:05:22 +0000
Subject: [New-bugs-announce] [issue23174] shelve.open fails with error
	"anydbm.error: db type	could not be determined"
Message-ID: <1420502722.34.0.451688646321.issue23174@psf.upfronthosting.co.za>


New submission from Karl Richter:

`shelve.open(tempfile.mkstemp()[1])` fails with error "anydbm.error: db type could not be determined" which is not explainable with the docs. Traceback is

    Traceback (most recent call last):
      File "./cudaminer_param_checker.py", line 720, in <module>
        plac.call(cudaminer_param_checker)
      File "/usr/local/lib/python2.7/dist-packages/plac_core.py", line 309, in call
        cmd, result = parser_from(obj).consume(arglist)
      File "/usr/local/lib/python2.7/dist-packages/plac_core.py", line 195, in consume
        return cmd, self.func(*(args + varargs + extraopts), **kwargs)
      File "./cudaminer_param_checker.py", line 715, in cudaminer_param_checker
        visualize_cudaminer_param_checker_results_wxpython_gui()
      File "./cudaminer_param_checker.py", line 365, in visualize_cudaminer_param_checker_results_wxpython_gui
        frame = CudaminerParamChecker(None, )
      File "./cudaminer_param_checker.py", line 378, in __init__
        self.generator = CudaminerParamCheckerGenerator()
      File "./cudaminer_param_checker.py", line 160, in __init__
        self.result_dict= shelve.open(storage_file_path) 
      File "/usr/lib/python2.7/shelve.py", line 239, in open
        return DbfilenameShelf(filename, flag, protocol, writeback)
      File "/usr/lib/python2.7/shelve.py", line 223, in __init__
        Shelf.__init__(self, anydbm.open(filename, flag), protocol, writeback)
      File "/usr/lib/python2.7/anydbm.py", line 82, in open
        raise error, "db type could not be determined"
    anydbm.error: db type could not be determined

----------
assignee: docs at python
components: Documentation
messages: 233488
nosy: docs at python, krichter
priority: normal
severity: normal
status: open
title: shelve.open fails with error "anydbm.error: db type could not be determined"
type: behavior
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23174>
_______________________________________

From report at bugs.python.org  Tue Jan  6 01:54:53 2015
From: report at bugs.python.org (Laimis)
Date: Tue, 06 Jan 2015 00:54:53 +0000
Subject: [New-bugs-announce] [issue23175] logging.exception doesn't accept
	custom exc_info
Message-ID: <1420505693.63.0.867428622164.issue23175@psf.upfronthosting.co.za>


New submission from Laimis:

Documentation says, that "The arguments are interpreted as for debug()." But it's not true, because no matter what exc_info is passed to logging.exception(), exc_info is overwritten (kwargs['exc_info'] = 1) and later self.error is called. 
This is either documentation issue or behavior issue, because in the current implementation it's not possible to pass custom exc_info (e.g. full traceback) to logging.exception(), although documentation implies it should work the same as for info, warning, error, debug and others.

----------
components: Library (Lib)
messages: 233496
nosy: laimis
priority: normal
severity: normal
status: open
title: logging.exception doesn't accept custom exc_info
type: behavior
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23175>
_______________________________________

From report at bugs.python.org  Tue Jan  6 03:58:37 2015
From: report at bugs.python.org (Sworddragon)
Date: Tue, 06 Jan 2015 02:58:37 +0000
Subject: [New-bugs-announce] [issue23176] socket.recvfrom(0) waits for data
Message-ID: <1420513117.73.0.631111386058.issue23176@psf.upfronthosting.co.za>


New submission from Sworddragon:

For example on sending ICMP packets and receiving the data socket.recv(1) does wait for data while socket.recv(0) doesn't. socket.recvfrom(1) does wait for data too but I'm also seeing that socket.recvfrom(0) is waiting for data which doesn't look correct (at least it seems not to be consistent with socket.recv()).

----------
components: Library (Lib)
messages: 233500
nosy: Sworddragon
priority: normal
severity: normal
status: open
title: socket.recvfrom(0) waits for data
type: behavior
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23176>
_______________________________________

From report at bugs.python.org  Tue Jan  6 11:54:16 2015
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 06 Jan 2015 10:54:16 +0000
Subject: [New-bugs-announce] [issue23177] test_ssl: failures on OpenBSD with
	LibreSSL
Message-ID: <1420541656.17.0.750310163886.issue23177@psf.upfronthosting.co.za>


New submission from STINNER Victor:

(See also the issue #21356.)

http://buildbot.python.org/all/builders/x86%20OpenBSD%205.5%203.x/builds/1242/steps/test/logs/stdio

======================================================================
FAIL: test_options (test.test_ssl.ContextTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/python-builds/3.x.borja-openbsd-x86/build/Lib/test/test_ssl.py", line 764, in test_options
    ctx.options)
AssertionError: -2130705409 != -2097150977

======================================================================
FAIL: test_openssl_version (test.test_ssl.BasicSocketTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/python-builds/3.x.borja-openbsd-x86/build/Lib/test/test_ssl.py", line 315, in test_openssl_version
    (s, t))
AssertionError: False is not true : ('LibreSSL 2.1', (2, 0, 0, 0, 0))

----------
components: Tests
messages: 233514
nosy: haypo, rpointel
priority: normal
severity: normal
status: open
title: test_ssl: failures on OpenBSD with LibreSSL
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23177>
_______________________________________

From report at bugs.python.org  Tue Jan  6 20:05:05 2015
From: report at bugs.python.org (Jon Dufresne)
Date: Tue, 06 Jan 2015 19:05:05 +0000
Subject: [New-bugs-announce] [issue23178] csv.reader does not handle BOM
Message-ID: <1420571105.9.0.00648910593011.issue23178@psf.upfronthosting.co.za>


New submission from Jon Dufresne:

The following test script demonstrates that Python's csv library does not handle a BOM. I would expect the returned row to be equal to expected and to print 'True' to stdout.

In the wild, it is typical for other CSV writers to add a BOM. MS Excel is especially picky about the BOM when reading a utf-8 encoded file. So many writers add a BOM for interopability with MS Excel.

If a python program accepts a CSV file as input (often the case in web apps), these files will not be handled correctly without preprocessing. In my opinion, this should "just work" when reading the file.

---
import codecs
import csv

f = open('foo.csv', 'wb')
f.write(codecs.BOM_UTF8 + b'a,b,c')
f.close()

expected = ['a', 'b', 'c']
f = open('foo.csv')
r = csv.reader(f)
row = next(r)

print(row)
print(row == expected)
---

Output
---
$ ./python ~/test.py
['\ufeffa', 'b', 'c']
False
---

----------
components: Library (Lib)
messages: 233549
nosy: jdufresne
priority: normal
severity: normal
status: open
title: csv.reader does not handle BOM
type: behavior
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23178>
_______________________________________

From report at bugs.python.org  Wed Jan  7 00:27:18 2015
From: report at bugs.python.org (Alessio Bogon)
Date: Tue, 06 Jan 2015 23:27:18 +0000
Subject: [New-bugs-announce] [issue23179] New function proposal:
	string.from_iterable(iterable [, map_function])
Message-ID: <1420586838.31.0.418706320174.issue23179@psf.upfronthosting.co.za>


New submission from Alessio Bogon:

I would like to suggest a new string function/constructor: 
string.from_iterable(iterable [,map_function])

I think that the behaviour is intuitive: given an iterable, it construct a string using its element by apply a `map_function`, if provided, to each one of them. After that the str() constructor will be applied to each element in any way, to ensure that effectively an iterable of strings is used.

Of course I do not expect that you will accept this patch, but I think this really is a missing piece of the python library.

You can argue that I could just use
"".join(iterable)
but in my opinion there are two problems:
1) if any of the elements of `iterable` is not a `str` instance, it will fail miserably;
2) this is not very pythonic.

This issue is meant to be an idea for the python maintainers, so I did not write the corresponding `Doc/libary/string.rst` documentation, but if you are interested I could do it.

Thank you people for your amazing work.

----------
components: Library (Lib)
files: string.from_iterable.patch
keywords: patch
messages: 233554
nosy: youtux
priority: normal
severity: normal
status: open
title: New function proposal: string.from_iterable(iterable [,map_function])
type: enhancement
versions: Python 3.4, Python 3.5
Added file: http://bugs.python.org/file37619/string.from_iterable.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23179>
_______________________________________

From report at bugs.python.org  Wed Jan  7 08:16:27 2015
From: report at bugs.python.org (Al Sweigart)
Date: Wed, 07 Jan 2015 07:16:27 +0000
Subject: [New-bugs-announce] [issue23180] Rename IDLE's "Windows" menu item
	to "Window"
Message-ID: <1420614987.43.0.350300091598.issue23180@psf.upfronthosting.co.za>


New submission from Al Sweigart:

Currently IDLE's top-level menu item is "Windows" on Windows & Linux but is specifically changed to "Window" on OS X to match mac's look and feel. See https://hg.python.org/cpython/rev/1b3b6b1982aa

The singular "Window" is the standard menu name on all platforms, and "Windows" should never be used. Note: Photoshop, Blender, Visual Studio, IntelliJ, WinMerge, HxD, and Notepad++ on Windows use the "Window" name in their menus. I can't find any that use "Windows".

----------
components: IDLE
files: patch.diff
keywords: patch
messages: 233566
nosy: Al.Sweigart
priority: normal
severity: normal
status: open
title: Rename IDLE's "Windows" menu item to "Window"
type: behavior
versions: Python 2.7, Python 3.2, Python 3.3, Python 3.4, Python 3.5, Python 3.6
Added file: http://bugs.python.org/file37622/patch.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23180>
_______________________________________

From report at bugs.python.org  Wed Jan  7 09:43:48 2015
From: report at bugs.python.org (Al Sweigart)
Date: Wed, 07 Jan 2015 08:43:48 +0000
Subject: [New-bugs-announce] [issue23181] Unicode "code point" should be two
	words in documentation
Message-ID: <1420620228.26.0.565920670024.issue23181@psf.upfronthosting.co.za>


New submission from Al Sweigart:

According to http://unicode.org/glossary/ "codepoint" is incorrect and should be changed to "code point".

----------
assignee: docs at python
components: Documentation
files: code_point_patch.diff
keywords: patch
messages: 233568
nosy: Al.Sweigart, docs at python
priority: normal
severity: normal
status: open
title: Unicode "code point" should be two words in documentation
type: behavior
versions: Python 3.4, Python 3.5
Added file: http://bugs.python.org/file37623/code_point_patch.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23181>
_______________________________________

From report at bugs.python.org  Wed Jan  7 09:49:05 2015
From: report at bugs.python.org (Ian Lee)
Date: Wed, 07 Jan 2015 08:49:05 +0000
Subject: [New-bugs-announce] [issue23182] Update grammar tests to use new
	style
Message-ID: <1420620545.16.0.862545867011.issue23182@psf.upfronthosting.co.za>


New submission from Ian Lee:

Per Guido's suggestion on the peps at python.org mailing list, I'm creating this issue to update the argument annotation tests at cpython/Lib/test/test_grammar.py to use the new style wording Guido requested on GitHub [1] that I proposed and was merged into PEP-8 [2] regarding annotated function definitions.

[1] https://github.com/jcrocholl/pep8/issues/357#issuecomment-67527455
[2] https://hg.python.org/peps/rev/7eb1ddc0291c

----------
components: Library (Lib), Tests
files: pep8-annotated-func-test-update.patch
keywords: patch
messages: 233569
nosy: IanLee1521, Rosuav, gvanrossum
priority: normal
severity: normal
status: open
title: Update grammar tests to use new style
type: enhancement
Added file: http://bugs.python.org/file37624/pep8-annotated-func-test-update.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23182>
_______________________________________

From report at bugs.python.org  Wed Jan  7 09:53:19 2015
From: report at bugs.python.org (Chathuranga)
Date: Wed, 07 Jan 2015 08:53:19 +0000
Subject: [New-bugs-announce] [issue23183] timeit CLI best of 3: undocumented
	output format
Message-ID: <1420620799.47.0.00516438922368.issue23183@psf.upfronthosting.co.za>


New submission from Chathuranga:

easy

Following command is executed in terminal to use timeit module to measure time.

python -m timeit -v -n 3 "time.sleep(1)"

Output of the command:
raw times: 3 3 3
3 loops, best of 3: 1 sec per loop

The interpretation of this output is unclear, and no details available in the documentation.
https://docs.python.org/2/library/timeit.html#command-line-interface

----------
assignee: docs at python
components: Documentation
messages: 233570
nosy: docs at python, hachat
priority: normal
severity: normal
status: open
title: timeit CLI best of 3: undocumented output format
type: enhancement
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23183>
_______________________________________

From report at bugs.python.org  Wed Jan  7 11:26:51 2015
From: report at bugs.python.org (Al Sweigart)
Date: Wed, 07 Jan 2015 10:26:51 +0000
Subject: [New-bugs-announce] [issue23184] Unused imports, variables,
	file in IDLE
Message-ID: <1420626411.51.0.528702156456.issue23184@psf.upfronthosting.co.za>


New submission from Al Sweigart:

The IDLE code base has several unused imports and local variables. The testcode.py file seems to have been accidentally checked in.

These changes were found through a lint program, not by hand. All idle unit tests pass after these changes.

----------
components: IDLE
files: idle_unused_patch.diff
keywords: patch
messages: 233573
nosy: Al.Sweigart
priority: normal
severity: normal
status: open
title: Unused imports, variables, file in IDLE
versions: Python 3.4, Python 3.5
Added file: http://bugs.python.org/file37626/idle_unused_patch.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23184>
_______________________________________

From report at bugs.python.org  Wed Jan  7 17:33:44 2015
From: report at bugs.python.org (Ethan Furman)
Date: Wed, 07 Jan 2015 16:33:44 +0000
Subject: [New-bugs-announce] [issue23185] add inf and nan to math module
Message-ID: <1420648424.37.0.39763409052.issue23185@psf.upfronthosting.co.za>


New submission from Ethan Furman:

Proposal:

  math.nan = float('nan')
  math.inf = float('inf')


Guido's approval:

  https://mail.python.org/pipermail/python-ideas/2015-January/030775.html


Followup question:

  Do we add a math.neginf, or somesuch, for float('-inf')?  Or just use -math.inf?

----------
keywords: easy
messages: 233580
nosy: ethan.furman
priority: normal
severity: normal
status: open
title: add inf and nan to math module
type: enhancement
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23185>
_______________________________________

From report at bugs.python.org  Wed Jan  7 18:01:02 2015
From: report at bugs.python.org (Benjamin Peterson)
Date: Wed, 07 Jan 2015 17:01:02 +0000
Subject: [New-bugs-announce] [issue23186] expose the client's list of
	ciphers during the handshake
Message-ID: <1420650062.65.0.672176536798.issue23186@psf.upfronthosting.co.za>


New submission from Benjamin Peterson:

This patch adds a shared_ciphers() method to SSLObject and SSLSocket, which allows servers to see the complete list of ciphers sent by the client during handshake.

----------
components: Library (Lib)
files: shared-ciphers.patch
keywords: patch
messages: 233584
nosy: alex, benjamin.peterson, christian.heimes, dstufft, giampaolo.rodola, janssen, pitrou
priority: normal
severity: normal
stage: patch review
status: open
title: expose the client's list of ciphers during the handshake
type: enhancement
versions: Python 3.5
Added file: http://bugs.python.org/file37630/shared-ciphers.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23186>
_______________________________________

From report at bugs.python.org  Thu Jan  8 00:41:35 2015
From: report at bugs.python.org (Ivailo Karamanolev)
Date: Wed, 07 Jan 2015 23:41:35 +0000
Subject: [New-bugs-announce] [issue23187] Segmentation fault,
	possibly asyncio related
Message-ID: <1420674095.59.0.174426734655.issue23187@psf.upfronthosting.co.za>


New submission from Ivailo Karamanolev:

I have a program that's running Tornado on top of asyncio and uses mysql-connector-python to monitor a set of HTTP servers and save the status to MySQL. Initially I ran it with the current version of Python in the Ubuntu 14.04 repos, that's 3.4.0 and started getting segmentation faults after keeping it running for a few days. I compiled and installed 3.4.2 and the same thing keeps happening, with the same stack trace (per gdb). Running it with and without gdb attached gives the same results - seg fault after 3-4 days of running.

The machine is running on a Xeon with ECC memory and doesn't show signs of illness otherwise, so it's probably not a hardware problem. Also, it's happening consistently when the programs runs for 3-4 days, with or without a system reboot.

Program received signal SIGSEGV, Segmentation fault.
0x0000000000471ed7 in subtype_dealloc (self=0x7ffff16a3860) at Objects/typeobject.c:1128
1128            _PyObject_GC_UNTRACK(self);

Stack trace:
#0  0x0000000000471ed7 in subtype_dealloc (self=0x7ffff16a3860) at Objects/typeobject.c:1128
#1  0x00000000004563f4 in dict_dealloc (mp=0x7fffe95444c8) at Objects/dictobject.c:1383
#2  0x0000000000472140 in subtype_dealloc (self=0x7fffe8e1fcf8) at Objects/typeobject.c:1157
#3  0x000000000058ad22 in frame_dealloc (f=0x1145258) at Objects/frameobject.c:429
#4  0x0000000000588fd7 in gen_send_ex (gen=gen at entry=0x7fffe8c5cea0, gen=gen at entry=0x7fffe8c5cea0, exc=0, arg=arg at entry=0x0) at Objects/genobject.c:145
#5  _PyGen_Send (gen=gen at entry=0x7fffe8c5cea0, arg=arg at entry=0x87fec0 <_Py_FalseStruct>) at Objects/genobject.c:158
#6  0x00000000004da329 in PyEval_EvalFrameEx (f=f at entry=0x11faaf8, throwflag=throwflag at entry=0) at Python/ceval.c:1906
#7  0x0000000000588e5c in gen_send_ex (gen=gen at entry=0x7fffe8c5ccf0, gen=gen at entry=0x7fffe8c5ccf0, exc=0, arg=arg at entry=0x7ffff1bb6420) at Objects/genobject.c:104
#8  _PyGen_Send (gen=gen at entry=0x7fffe8c5ccf0, arg=arg at entry=0x87fec0 <_Py_FalseStruct>) at Objects/genobject.c:158
#9  0x00000000004da329 in PyEval_EvalFrameEx (f=f at entry=0x10daa48, throwflag=throwflag at entry=0) at Python/ceval.c:1906
#10 0x0000000000588e5c in gen_send_ex (gen=gen at entry=0x7fffe8c5cc18, gen=gen at entry=0x7fffe8c5cc18, exc=0, arg=arg at entry=0x102aac8) at Objects/genobject.c:104
#11 _PyGen_Send (gen=gen at entry=0x7fffe8c5cc18, arg=arg at entry=0x87fec0 <_Py_FalseStruct>) at Objects/genobject.c:158
#12 0x00000000004da329 in PyEval_EvalFrameEx (f=f at entry=0x10da7f8, throwflag=throwflag at entry=0) at Python/ceval.c:1906
#13 0x0000000000588e5c in gen_send_ex (gen=0x7ffff16b65a0, gen=0x7ffff16b65a0, exc=0, arg=<optimized out>) at Objects/genobject.c:104
#14 _PyGen_Send (gen=0x7ffff16b65a0, arg=<optimized out>) at Objects/genobject.c:158
#15 0x00000000004d9d0b in call_function (oparg=<optimized out>, pp_stack=0x7fffffffd290) at Python/ceval.c:4221
#16 PyEval_EvalFrameEx (f=f at entry=0x10da568, throwflag=throwflag at entry=0) at Python/ceval.c:2836
#17 0x00000000004d28d4 in PyEval_EvalCodeEx (_co=0x7ffff1e0c030, globals=<optimized out>, locals=<optimized out>, args=<optimized out>, argcount=3, kws=0x12e38b8, kwcount=0,
    defs=0x7ffff1e0a8e0, defcount=2, kwdefs=0x0, closure=0x7ffff1e10588) at Python/ceval.c:3585
#18 0x00000000004d8e75 in fast_function (nk=<optimized out>, na=3, n=3, pp_stack=0x7fffffffd490, func=0x7ffff1e122f0) at Python/ceval.c:4341
#19 call_function (oparg=<optimized out>, pp_stack=0x7fffffffd490) at Python/ceval.c:4259
#20 PyEval_EvalFrameEx (f=f at entry=0x12e3708, throwflag=throwflag at entry=0) at Python/ceval.c:2836
#21 0x00000000004d28d4 in PyEval_EvalCodeEx (_co=0x7ffff1e0c0c0, globals=<optimized out>, locals=locals at entry=0x0, args=args at entry=0x7fffeae8a720, argcount=2, kws=kws at entry=0x0,
    kwcount=kwcount at entry=0, defs=defs at entry=0x0, defcount=defcount at entry=0, kwdefs=0x0, closure=0x0) at Python/ceval.c:3585
#22 0x000000000058c67c in function_call (func=0x7ffff1e12378, arg=0x7fffeae8a708, kw=0x0) at Objects/funcobject.c:632
#23 0x000000000042686a in PyObject_Call (func=func at entry=0x7ffff1e12378, arg=arg at entry=0x7fffeae8a708, kw=kw at entry=0x0) at Objects/abstract.c:2067
#24 0x00000000004d48a4 in ext_do_call (nk=<optimized out>, na=1, flags=<optimized out>, pp_stack=0x7fffffffd760, func=0x7ffff1e12378) at Python/ceval.c:4558
#25 PyEval_EvalFrameEx (f=f at entry=0x145ac48, throwflag=throwflag at entry=0) at Python/ceval.c:2876
#26 0x00000000004d9e82 in fast_function (nk=<optimized out>, na=1, n=1, pp_stack=0x7fffffffd8a0, func=0x7ffff1e2d488) at Python/ceval.c:4331
#27 call_function (oparg=<optimized out>, pp_stack=0x7fffffffd8a0) at Python/ceval.c:4259
#28 PyEval_EvalFrameEx (f=f at entry=0x11aee28, throwflag=throwflag at entry=0) at Python/ceval.c:2836
#29 0x00000000004d9e82 in fast_function (nk=<optimized out>, na=1, n=1, pp_stack=0x7fffffffd9f0, func=0x7ffff1bc1400) at Python/ceval.c:4331
#30 call_function (oparg=<optimized out>, pp_stack=0x7fffffffd9f0) at Python/ceval.c:4259
#31 PyEval_EvalFrameEx (f=f at entry=0x10d9c68, throwflag=throwflag at entry=0) at Python/ceval.c:2836
#32 0x00000000004d9e82 in fast_function (nk=<optimized out>, na=1, n=1, pp_stack=0x7fffffffdb40, func=0x7ffff1bc0268) at Python/ceval.c:4331
#33 call_function (oparg=<optimized out>, pp_stack=0x7fffffffdb40) at Python/ceval.c:4259
#34 PyEval_EvalFrameEx (f=f at entry=0xf9ed08, throwflag=throwflag at entry=0) at Python/ceval.c:2836
#35 0x00000000004d9e82 in fast_function (nk=<optimized out>, na=0, n=0, pp_stack=0x7fffffffdc90, func=0x7ffff16a4d08) at Python/ceval.c:4331
#36 call_function (oparg=<optimized out>, pp_stack=0x7fffffffdc90) at Python/ceval.c:4259
#37 PyEval_EvalFrameEx (f=f at entry=0x7ffff7f4b438, throwflag=throwflag at entry=0) at Python/ceval.c:2836
#38 0x00000000004dae2a in PyEval_EvalCodeEx (closure=0x0, kwdefs=0x0, defcount=0, defs=0x0, kwcount=0, kws=0x0, argcount=0, args=0x0, locals=locals at entry=0x7ffff7f95d50,
    globals=globals at entry=0x100000000, _co=0x7ffff7ee0150, _co at entry=0x8ac460) at Python/ceval.c:3585
#39 PyEval_EvalCode (co=co at entry=0x7ffff7ee0150, globals=globals at entry=0x7ffff7f48408, locals=locals at entry=0x7ffff7f48408) at Python/ceval.c:773
#40 0x000000000050a8f8 in run_mod (arena=0x95d9a0, flags=0x7fffffffdee0, locals=0x7ffff7f48408, globals=0x7ffff7f48408, filename=0x7ffff7e3f540, mod=0x8fb1e0)
    at Python/pythonrun.c:2180
#41 PyRun_FileExFlags (fp=0x8fa9c0, filename_str=<optimized out>, start=<optimized out>, globals=0x7ffff7f48408, locals=0x7ffff7f48408, closeit=1, flags=0x7fffffffdee0)
    at Python/pythonrun.c:2133
#42 0x000000000050c8f5 in PyRun_SimpleFileExFlags (fp=0x8fa9c0, filename=<optimized out>, closeit=1, flags=0x7fffffffdee0) at Python/pythonrun.c:1606
#43 0x000000000041edc4 in run_file (p_cf=0x7fffffffdee0, filename=0x8ac2a0 L"./torrent_manager.py", fp=0x8fa9c0) at Modules/main.c:319
#44 Py_Main (argc=argc at entry=2, argv=argv at entry=0x8ab010) at Modules/main.c:751
#45 0x000000000041b227 in main (argc=2, argv=<optimized out>) at ./Modules/python.c:69

----------
components: Interpreter Core, asyncio
messages: 233608
nosy: gvanrossum, haypo, karamanolev, yselivanov
priority: normal
severity: normal
status: open
title: Segmentation fault, possibly asyncio related
type: crash
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23187>
_______________________________________

From report at bugs.python.org  Thu Jan  8 04:17:21 2015
From: report at bugs.python.org (Nick Coghlan)
Date: Thu, 08 Jan 2015 03:17:21 +0000
Subject: [New-bugs-announce] [issue23188] Convert _PyErr_ChainExceptions to
	a public API
Message-ID: <1420687041.12.0.12728944273.issue23188@psf.upfronthosting.co.za>


New submission from Nick Coghlan:

There's currently an internal helper API for chaining exceptions from C code, _PyErr_ChainExceptions.

It would be good to have a public API that offered equivalent functionality for use in third party C extensions.

----------
messages: 233619
nosy: ncoghlan
priority: normal
severity: normal
stage: needs patch
status: open
title: Convert _PyErr_ChainExceptions to a public API
type: enhancement
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23188>
_______________________________________

From report at bugs.python.org  Thu Jan  8 10:41:45 2015
From: report at bugs.python.org (=?utf-8?q?Jarle_Selv=C3=A5g?=)
Date: Thu, 08 Jan 2015 09:41:45 +0000
Subject: [New-bugs-announce] [issue23189] Set docstrings to empty string
	when optimizing with -OO.
Message-ID: <1420710105.42.0.0455822244895.issue23189@psf.upfronthosting.co.za>


New submission from Jarle Selv?g:

Python code byte-compiled with -OO has doc-strings stripped out. 

This creates problems when compiling different packages which changes the doc-strings by doing something like this:
     __doc__ += "additional text"
(when the docstring is 'None', this will fail).

The packages "lmfit 0.8.1" and "Patsy 0.3.0" have this problem, and must be patched before compilation.

See related discussion on Stackoverflow:
http://stackoverflow.com/questions/22299532/unsupported-operand-types-for-nonetype-and-str-winappdbg-error-after-c

Proposal: Set the doc-strings to empty string ("") instead of removing them completely during optimization with -OO. The memory footprint would anyway be the same.

----------
components: Interpreter Core
messages: 233634
nosy: jvs
priority: normal
severity: normal
status: open
title: Set docstrings to empty string when optimizing with -OO.
type: enhancement
versions: Python 2.7, Python 3.2, Python 3.3, Python 3.4, Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23189>
_______________________________________

From report at bugs.python.org  Thu Jan  8 12:20:38 2015
From: report at bugs.python.org (Antoine Pitrou)
Date: Thu, 08 Jan 2015 11:20:38 +0000
Subject: [New-bugs-announce] [issue23190] OpenSSL fails building with
	build.bat
Message-ID: <1420716038.45.0.141553844503.issue23190@psf.upfronthosting.co.za>


New submission from Antoine Pitrou:

With "PCbuild\build.bat -d -e", OpenSSL fails building:

  nasm: fatal: unable to open input file `Z:\cpython\default\externals\openssl-
  1.0.1j\tmp32\aes-586.asm'

Detailed build log is:

"Z:\cpython\default\PCbuild\libeay.vcxproj" (default target) (32:3) ->
(BuildNasmFiles target) -> 
  Z:\cpython\default\PCbuild\openssl.props(61,5): error MSB3073: The command "setlocal [Z:\cpython\default\PCbuild\libeay.vcxproj]
Z:\cpython\default\PCbuild\openssl.props(61,5): error MSB3073: set PATH=%PATH%;Z:\cpython\default\externals\\nasm-2.11.06\ [Z:\cpython\default\PCbuild\libeay.vcxproj]
Z:\cpython\default\PCbuild\openssl.props(61,5): error MSB3073: nasm.exe -f win32 -o "Z:\cpython\default\externals\openssl-1.0.1j\tmp32\libeay\aes-586.obj" "Z:\cpython\default\externals\openssl-1.0.1j\tmp32\aes-586.asm"" exited with code 1. [Z:\cpython\default\PCbuild\libeay.vcxproj]

"Z:\cpython\default\PCbuild\pcbuild.proj" (Rebuild target) (1) ->
"Z:\cpython\default\PCbuild\_ssl.vcxproj" (Build target) (31:2) ->
"Z:\cpython\default\PCbuild\ssleay.vcxproj" (default target) (33:3) ->
(ClCompile target) -> 
  c1 : fatal error C1083: Cannot open source file: 'Z:\cpython\default\externals\openssl-1.0.1j\ssl\d1_both.c': No such file or directory [Z:\cpython\default\PCbuild\ssleay.vcxproj]

[etc.]


It seems there are problems with how the file paths are computed? Also, I have no file "aes-586.asm", how is it supposed to be generated?

----------
components: Build, Windows
messages: 233646
nosy: pitrou, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: OpenSSL fails building with build.bat
type: behavior
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23190>
_______________________________________

From report at bugs.python.org  Thu Jan  8 14:02:17 2015
From: report at bugs.python.org (M. Schmitzer)
Date: Thu, 08 Jan 2015 13:02:17 +0000
Subject: [New-bugs-announce] [issue23191] fnmatch regex cache use is not
	threadsafe
Message-ID: <1420722137.58.0.63717437035.issue23191@psf.upfronthosting.co.za>


New submission from M. Schmitzer:

The way the fnmatch module uses its regex cache is not threadsafe. When multiple threads use the module in parallel, a race condition between retrieving a - presumed present - item from the cache and clearing the cache (because the maximum size has been reached) can lead to KeyError being raised.

The attached script demonstrates the problem. Running it will (eventually) yield errors like the following.

Exception in thread Thread-10:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.7/threading.py", line 763, in run
    self.__target(*self.__args, **self.__kwargs)
  File "fnmatch_thread.py", line 12, in run
    fnmatch.fnmatchcase(name, pat)
  File "/home/marc/.venv/modern/lib/python2.7/fnmatch.py", line 79, in fnmatchcase
    return _cache[pat].match(name) is not None
KeyError: 'lYwrOCJtLU'

----------
components: Library (Lib)
files: fnmatch_thread.py
messages: 233650
nosy: mschmitzer
priority: normal
severity: normal
status: open
title: fnmatch regex cache use is not threadsafe
type: crash
versions: Python 2.7
Added file: http://bugs.python.org/file37642/fnmatch_thread.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23191>
_______________________________________

From report at bugs.python.org  Thu Jan  8 15:42:05 2015
From: report at bugs.python.org (Chris Angelico)
Date: Thu, 08 Jan 2015 14:42:05 +0000
Subject: [New-bugs-announce] [issue23192] Generator return value ignored in
	lambda function
Message-ID: <1420728125.93.0.910147391239.issue23192@psf.upfronthosting.co.za>


New submission from Chris Angelico:

As yield is an expression, it's legal in a lambda function, which then
means you have a generator function. But it's not quite the same as
the equivalent function made with def:

$ python3
Python 3.5.0a0 (default:1c51f1650c42+, Dec 29 2014, 02:29:06)
[GCC 4.7.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> f=lambda: (yield 5)
>>> x=f()
>>> next(x)
5
>>> x.send(123)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration
>>> def f(): return (yield 5)
...
>>> x=f()
>>> next(x)
5
>>> x.send(123)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration: 123
>>> x = (lambda: print((yield 1)) or 2)()
>>> next(x)
1
>>> x.send(3)
3
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration

The last example demonstrates that send() is working, but the return value is not getting propagated. Disassembly shows this:

>>> dis.dis(lambda: (yield 5))
  1           0 LOAD_CONST               1 (5)
              3 YIELD_VALUE
              4 POP_TOP
              5 LOAD_CONST               0 (None)
              8 RETURN_VALUE
>>> def f(): return (yield 5)
... 
>>> dis.dis(f)
  1           0 LOAD_CONST               1 (5)
              3 YIELD_VALUE
              4 RETURN_VALUE

I'm sure this is a bug that will affect very approximately zero people, but it's still a peculiar inconsistency!

Verified with 3.5 and 3.4.

----------
components: Interpreter Core
messages: 233662
nosy: Rosuav
priority: normal
severity: normal
status: open
title: Generator return value ignored in lambda function
versions: Python 3.4, Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23192>
_______________________________________

From report at bugs.python.org  Thu Jan  8 15:42:24 2015
From: report at bugs.python.org (Michael Vogt)
Date: Thu, 08 Jan 2015 14:42:24 +0000
Subject: [New-bugs-announce] [issue23193] Please support "numeric_owner" in
	tarfile
Message-ID: <1420728144.57.0.872420723687.issue23193@psf.upfronthosting.co.za>


New submission from Michael Vogt:

Please consider adding a option to extract a tarfile with the uid/gid instead of the lookup for uname/gname in the tarheader (i.e. what tar --numeric-owner provides).

One use-case is if you unpack a chroot tarfile that contains a /etc/{passwd,group} file with different uid/gid for user/groups like zope that may be present in both host and chroot but have different numbers. With the current approach files owned by this user will get the host uid/gid instead of the uid/gid of the chroot.

Attached is a patch to outline what I have in mind - if there is a chance that this patch goes in I'm happy to write the required test (mocking os.chown()) for this to go in.

Thanks for your consideration,
 Michael

----------
components: Library (Lib)
files: tarfile-numeric-owner.diff
keywords: patch
messages: 233663
nosy: mvo
priority: normal
severity: normal
status: open
title: Please support "numeric_owner" in tarfile
type: enhancement
Added file: http://bugs.python.org/file37645/tarfile-numeric-owner.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23193>
_______________________________________

From report at bugs.python.org  Thu Jan  8 19:52:03 2015
From: report at bugs.python.org (Jim Kubicek)
Date: Thu, 08 Jan 2015 18:52:03 +0000
Subject: [New-bugs-announce] [issue23194] Antigravity prints osascript
	errors in OS X Yosemite
Message-ID: <1420743123.77.0.0822630306948.issue23194@psf.upfronthosting.co.za>


New submission from Jim Kubicek:

Python 3.4.2 (default, Dec 29 2014, 14:03:16) 
    [GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.56)] on darwin
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import antigravity
    2015-01-08 10:45:03.771 osascript[47250:12135049] Error loading /Library/ScriptingAdditions/Adobe Unit Types.osax/Contents/MacOS/Adobe Unit Types:  dlopen(/Library/ScriptingAdditions/Adobe Unit Types.osax/Contents/MacOS/Adobe Unit Types, 262): no suitable image found.  Did find:
	/Library/ScriptingAdditions/Adobe Unit Types.osax/Contents/MacOS/Adobe Unit Types: no matching architecture in universal wrapper
    osascript: OpenScripting.framework - scripting addition "/Library/ScriptingAdditions/Adobe Unit Types.osax" declares no loadable handlers.

The import command appears to work correctly (the proper XKCD opens), so I think this issue is just cosmetic.

----------
components: Extension Modules
messages: 233676
nosy: jkubicek
priority: normal
severity: normal
status: open
title: Antigravity prints osascript errors in OS X Yosemite
type: behavior
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23194>
_______________________________________

From report at bugs.python.org  Thu Jan  8 21:30:56 2015
From: report at bugs.python.org (Pierre Nugues)
Date: Thu, 08 Jan 2015 20:30:56 +0000
Subject: [New-bugs-announce] [issue23195] Sorting with locale does not work
	properly with Python3	on Macos
Message-ID: <1420749056.81.0.378421650447.issue23195@psf.upfronthosting.co.za>


New submission from Pierre Nugues:

The sorted() function does not work properly with macosx.
Here is a script to reproduce the issue:

import locale
locale.setlocale(locale.LC_ALL, "fr_FR.UTF-8")
a = ["A", "E", "Z", "a", "e", "?", "z"]
sorted(a)
sorted(a, key=locale.strxfrm)


The execution on MacOsX produces:
pierre:Flaubert pierre$ sw_vers -productVersion
10.10.1
pierre:Flaubert pierre$ python3
Python 3.4.2 (v3.4.2:ab2c023a9432, Oct  5 2014, 20:42:22) 
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
import locale
locale.setlocale(locale.LC_ALL, "fr_FR.UTF-8")
'fr_FR.UTF-8'
a = ["A", "E", "Z", "a", "e", "?", "z"]
sorted(a)
['A', 'E', 'Z', 'a', 'e', 'z', '?']
sorted(a, key=locale.strxfrm)
['A', 'E', 'Z', 'a', 'e', 'z', '?']


while it produces this on your interactive shell (python.org):
In [10]: import locale
In [11]: locale.setlocale(locale.LC_ALL, "fr_FR.UTF-8")
Out[11]: 'fr_FR.UTF-8'
In [12]: a = ["A", "E", "Z", "a", "e", "?", "z"]
In [13]: sorted(a)
Out[13]: ['A', 'E', 'Z', 'a', 'e', 'z', '?']
In [14]: sorted(a, key=locale.strxfrm)
Out[14]: ['a', 'A', 'e', 'E', '?', 'z', 'Z']

which is correct.

----------
components: Unicode
messages: 233685
nosy: ezio.melotti, haypo, pnugues
priority: normal
severity: normal
status: open
title: Sorting with locale does not work properly with Python3 on Macos
type: behavior
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23195>
_______________________________________

From report at bugs.python.org  Thu Jan  8 21:54:42 2015
From: report at bugs.python.org (Pierre Nugues)
Date: Thu, 08 Jan 2015 20:54:42 +0000
Subject: [New-bugs-announce] [issue23196] Greek letters not sorted properly
Message-ID: <1420750482.34.0.750338196942.issue23196@psf.upfronthosting.co.za>


New submission from Pierre Nugues:

Greek letters are not properly sorted when a locale is set. I tested a French and a Greek locales. Here is an output obtained from the Python interactive shell available from the python.org home page:

In [22]: a
Out[22]: 
('?',
 '?',
 '?',
 '?',
 '?',
 '?',
 '?',
 '?',
 '?',
 '?',
 '?',
 '?',
 '?',
 '?',
 '?',
 '?',
 '?',
 '?',
 '?',
 '?',
 '?',
 '?',
 '?',
 '?',
 '?',
 '?',
 '?',
 '?')
In [26]: sorted(a, key=locale.strxfrm)
Out[26]: 
['?',
 '?',
 '?',
 '?',
 '?',
 '?',
 '?',
 '?',
 '?',
 '?',
 '?',
 '?',
 '?',
 '?',
 '?',
 '?',
 '?',
 '?',
 '?',
 '?',
 '?',
 '?',
 '?',
 '?',
 '?',
 '?',
 '?',
 '?']

The letter '?' is wrongly sorted. You can try to sort the same character list with the ICU demonstration to see the correct ordering here: 
http://demo.icu-project.org/icu-bin/locexp?_=el&d_=fr&x=col

----------
components: Unicode
messages: 233686
nosy: ezio.melotti, haypo, pnugues
priority: normal
severity: normal
status: open
title: Greek letters not sorted properly
type: behavior
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23196>
_______________________________________

From report at bugs.python.org  Fri Jan  9 01:11:52 2015
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 09 Jan 2015 00:11:52 +0000
Subject: [New-bugs-announce] [issue23197] asyncio: check if a future is
	cancelled before calling	set_result/set_exception
Message-ID: <1420762312.46.0.962527500609.issue23197@psf.upfronthosting.co.za>


New submission from STINNER Victor:

set_result/set_exception methods of an asyncio.Future raise an exception if the future is cancelled.

Attached patch adds the check in 3 remaining places.

----------
components: asyncio
files: asyncio.patch
keywords: patch
messages: 233699
nosy: gvanrossum, haypo, yselivanov
priority: normal
severity: normal
status: open
title: asyncio: check if a future is cancelled before calling set_result/set_exception
versions: Python 3.4, Python 3.5
Added file: http://bugs.python.org/file37648/asyncio.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23197>
_______________________________________

From report at bugs.python.org  Fri Jan  9 01:41:19 2015
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 09 Jan 2015 00:41:19 +0000
Subject: [New-bugs-announce] [issue23198] asyncio: refactor StreamReader
Message-ID: <1420764079.77.0.670547785997.issue23198@psf.upfronthosting.co.za>


New submission from STINNER Victor:

Attached patch refactors the asyncio.StreamReader class:

- use the value None instead of True or False to wake up the waiter
- add a new _wakeup_waiter() method
- replace _create_waiter() method with a _wait_for_data() coroutine function

The change adds a subgenerator (_wait_for_data), is it an issue in term of performances? (I don't think so.)

----------
components: asyncio
files: refactor_streamreader.patch
keywords: patch
messages: 233700
nosy: gvanrossum, haypo, yselivanov
priority: normal
severity: normal
status: open
title: asyncio: refactor StreamReader
versions: Python 3.4, Python 3.5
Added file: http://bugs.python.org/file37649/refactor_streamreader.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23198>
_______________________________________

From report at bugs.python.org  Fri Jan  9 01:45:37 2015
From: report at bugs.python.org (Zach Welch)
Date: Fri, 09 Jan 2015 00:45:37 +0000
Subject: [New-bugs-announce] [issue23199] libpython27.a in amd64 release is
	32-bit
Message-ID: <1420764337.42.0.255694319213.issue23199@psf.upfronthosting.co.za>


New submission from Zach Welch:

I tried to link a program against the libpython27.a provided by the latest 2.7.9 amd64 installer, only to discover that the provided library is a 32-bit version.  I had to go through the gendef/dlltool dance in order to produce a useable 64-bit library from the DLL.

----------
components: Library (Lib), Windows
messages: 233702
nosy: steve.dower, tim.golden, zach.ware, zwelch
priority: normal
severity: normal
status: open
title: libpython27.a in amd64 release is 32-bit
type: compile error
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23199>
_______________________________________

From report at bugs.python.org  Fri Jan  9 02:26:21 2015
From: report at bugs.python.org (Martin Panter)
Date: Fri, 09 Jan 2015 01:26:21 +0000
Subject: [New-bugs-announce] [issue23200] Clarify max_length and flush() for
	zlib decompression
Message-ID: <1420766781.53.0.179588350228.issue23200@psf.upfronthosting.co.za>


New submission from Martin Panter:

This simple patch documents that max_length has to be non-zero. The implementation actually uses zero as a special value to indicate max_length was not specified.

Also, I wonder what the point of the Decompressor.flush() method is. Reading the module code and zlib manual, it looks like it would return the same data as decompressor.decompress(decompressor.unconsumed_tail), except that it uses the Z_FINISH flag instead of Z_SYNC_FLUSH, so that zlib can optimize by assuming no more data is to be processed. Since unconsumed_tail is read-only and only relevant when using max_length, and flush does not support max_length, I wonder if the flush() method should be deprecated.

To further test this theory, I modified the test_zlib.py file so that all appropriate flush() calls are equivalent to this:

self.assertFalse(dco.flush())

and the tests all still pass.

BTW, it looks to me like zlib_Decompress_flush_impl() is setting avail_out too high (blessing unallocated memory) when the total length is more than half of UNIT_MAX, but I am not in a position to test this.

----------
assignee: docs at python
components: Documentation
files: max_length.patch
keywords: patch
messages: 233709
nosy: docs at python, vadmium
priority: normal
severity: normal
status: open
title: Clarify max_length and flush() for zlib decompression
versions: Python 3.4
Added file: http://bugs.python.org/file37650/max_length.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23200>
_______________________________________

From report at bugs.python.org  Fri Jan  9 04:13:06 2015
From: report at bugs.python.org (Devin Jeanpierre)
Date: Fri, 09 Jan 2015 03:13:06 +0000
Subject: [New-bugs-announce] [issue23201] Decimal(0)**0 is an error,
	0**0 is 1, but Decimal(0) == 0
Message-ID: <1420773186.22.0.229704657455.issue23201@psf.upfronthosting.co.za>


New submission from Devin Jeanpierre:

>>> import decimal
>>> x = 0
>>> y = float(x)
>>> z = decimal.Decimal(x)
>>> x == y == z == x
True
>>> x ** x
1
>>> y**y
1.0
>>> z**z
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/decimal.py", line 2216, in __pow__
    return context._raise_error(InvalidOperation, '0 ** 0')
  File "/usr/lib/python2.7/decimal.py", line 3872, in _raise_error
    raise error(explanation)
decimal.InvalidOperation: 0 ** 0

This is PHP-like and confusing, and maybe not justified just by standards compliance. If it is justified by standards compliance, this deserves to be spelled out in big red letters in the documentation for the decimal module, along with any other inconsistencies.

----------
components: Library (Lib)
messages: 233711
nosy: Devin Jeanpierre
priority: normal
severity: normal
status: open
title: Decimal(0)**0 is an error, 0**0 is 1, but Decimal(0) == 0
versions: Python 2.7, Python 3.2, Python 3.3, Python 3.4, Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23201>
_______________________________________

From report at bugs.python.org  Fri Jan  9 07:02:04 2015
From: report at bugs.python.org (Florian Bruhin)
Date: Fri, 09 Jan 2015 06:02:04 +0000
Subject: [New-bugs-announce] [issue23202] pyvenv does not fail like
	documented when a venv already	exists
Message-ID: <1420783324.13.0.763537521613.issue23202@psf.upfronthosting.co.za>


New submission from Florian Bruhin:

https://docs.python.org/3/library/venv.html says:

> If the target directory already exists an error will be raised, unless the --clear or --upgrade option was provided.

However, that doesn't seem to be the case:

[florian at ginny ~]$ python --version
Python 3.4.2
[florian at ginny ~]$ pyvenv foobar
[florian at ginny ~]$ ls foobar
bin  include  lib  lib64  pyvenv.cfg
[florian at ginny ~]$ pyvenv foobar
[florian at ginny ~]$

----------
components: Library (Lib)
messages: 233718
nosy: The Compiler, vinay.sajip
priority: normal
severity: normal
status: open
title: pyvenv does not fail like documented when a venv already exists
type: behavior
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23202>
_______________________________________

From report at bugs.python.org  Fri Jan  9 07:44:42 2015
From: report at bugs.python.org (Franck Michea)
Date: Fri, 09 Jan 2015 06:44:42 +0000
Subject: [New-bugs-announce] [issue23203] Aliasing import of sub-{module,
	package} from the package raises AttributeError on import.
Message-ID: <1420785882.78.0.914141516682.issue23203@psf.upfronthosting.co.za>


New submission from Franck Michea:

Hi, for those of you that prefer to read an example, you can read that commented demonstration of the bug[1].

Today I discovered what I think is a bug in the import system. Here is the basic setup:

We have three nested packages: foo -> bar -> baz. The bar package imports foo.bar.baz. We try to import foo.bar. This works well unless we try to alias the foo.bar.baz import in foo.bar with the "import ... as ..." syntax. In that case the file will be read and executed but when we get out of the import statement, then python throws:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "foo_alias_mod/bar/__init__.py", line 1, in <module>
    import foo_alias_mod.bar.baz as name_does_not_matter
AttributeError: 'module' object has no attribute 'bar'

This works whether baz is a package or a module actually. It does not matter if it's from the interpreter, or in a file, ... I've seen it trigger with 2.7.5, 2.7.9, 3.4.5, tip, so I guess this has been here for some time.

Please read the link below for a complete demo, and you can always download the tarball[2] to test yourself.

[1]: Commented demonstration: http://98810f8c06.net/wtf_python.html
[2]: Tarball for test: http://98810f8c06.net/wtf_python-demo.tar.bz2

----------
components: Interpreter Core
messages: 233719
nosy: franck
priority: normal
severity: normal
status: open
title: Aliasing import of sub-{module,package} from the package raises AttributeError on import.

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23203>
_______________________________________

From report at bugs.python.org  Fri Jan  9 08:25:48 2015
From: report at bugs.python.org (Devin Jeanpierre)
Date: Fri, 09 Jan 2015 07:25:48 +0000
Subject: [New-bugs-announce] [issue23204] list.index and rest of list
	methods disagree if a value	is in the list if it's mutated
	during the call
Message-ID: <1420788348.75.0.1692375665.issue23204@psf.upfronthosting.co.za>


New submission from Devin Jeanpierre:

>>> class AppendOnUnequal(object):
...     def __init__(self, append_to):
...         self.append_to = append_to
...     def __eq__(self, other):
...         if self is other:
...             return True
...         self.append_to.append(self)
...         return False
... 
>>> L = [1]; AppendOnUnequal(L) in L
True
>>> L = [1]; L.count(AppendOnUnequal(L))
1
>>> L = [1]; L.remove(AppendOnUnequal(L))
>>> L = [1]; L.index(AppendOnUnequal(L))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: <__main__.AppendOnUnequal object at 0x7f2562d071d0> is not in list


.index() is the only odd one out here. Looks like a bug to me.

----------
components: Interpreter Core
messages: 233721
nosy: Devin Jeanpierre
priority: normal
severity: normal
status: open
title: list.index and rest of list methods disagree if a value is in the list if it's mutated during the call
type: behavior
versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23204>
_______________________________________

From report at bugs.python.org  Fri Jan  9 09:16:00 2015
From: report at bugs.python.org (Al Sweigart)
Date: Fri, 09 Jan 2015 08:16:00 +0000
Subject: [New-bugs-announce] [issue23205] Unit test needed for IDLE's
	GrepDialog.py's findfiles()
Message-ID: <1420791360.73.0.110066531981.issue23205@psf.upfronthosting.co.za>


New submission from Al Sweigart:

GrepDialog.py's findfiles() method lacks a unit test.

The comments in the unit test stub in test_grep.py correctly notes that since findfiles() method does not rely on the GrepDialog class, it can be made into a function.

The attached patch does the following:

- Moves findfiles() to be a function in the GrepDialog.py module.
- Adds a unit test for findfiles()
- Adds sensible default arguments
- As suggested by the previous stub comments, findfiles() returns a generator instead of a full list
- Changes 'list' and 'dir' names since those are built-ins
- Uses os.walk() instead of reinventing it.

There were so many changes to make that I eventually just rewrote the entire findfiles function. I've checked that the new version returns the same strings as the old version.

----------
components: IDLE
files: idle_grep_findfiles_test.diff
keywords: patch
messages: 233727
nosy: Al.Sweigart
priority: normal
severity: normal
status: open
title: Unit test needed for IDLE's GrepDialog.py's findfiles()
versions: Python 3.5
Added file: http://bugs.python.org/file37652/idle_grep_findfiles_test.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23205>
_______________________________________

From report at bugs.python.org  Fri Jan  9 14:17:25 2015
From: report at bugs.python.org (INADA Naoki)
Date: Fri, 09 Jan 2015 13:17:25 +0000
Subject: [New-bugs-announce] [issue23206] json.dumps(ensure_ascii=False) is
	~10x slower than	json.dumps()
Message-ID: <1420809445.24.0.133647496806.issue23206@psf.upfronthosting.co.za>


New submission from INADA Naoki:

I prefer ensure_ascii=False because it's efficient.
But I notice it is very slower.

On Python 3.4.2:
In [3]: %timeit json.dumps([{'hello': 'world'}]*100)
10000 loops, best of 3: 74.8 ?s per loop

In [4]: %timeit json.dumps([{'hello': 'world'}]*100, ensure_ascii=False)
1000 loops, best of 3: 259 ?s per loop

On Python HEAD with attached patch:
In [2]: %timeit json.dumps([{'hello': 'world'}]*100)
10000 loops, best of 3: 80.8 ?s per loop

In [3]: %timeit json.dumps([{'hello': 'world'}]*100, ensure_ascii=False)
10000 loops, best of 3: 80.4 ?s per loop

----------
components: Library (Lib)
files: json-fast-unicode-encode.patch
keywords: patch
messages: 233752
nosy: naoki
priority: normal
severity: normal
status: open
title: json.dumps(ensure_ascii=False) is ~10x slower than json.dumps()
versions: Python 3.5
Added file: http://bugs.python.org/file37653/json-fast-unicode-encode.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23206>
_______________________________________

From report at bugs.python.org  Fri Jan  9 15:20:39 2015
From: report at bugs.python.org (Florian Bruhin)
Date: Fri, 09 Jan 2015 14:20:39 +0000
Subject: [New-bugs-announce] [issue23207] logging.basicConfig does not
	validate keyword arguments
Message-ID: <1420813239.75.0.476507944816.issue23207@psf.upfronthosting.co.za>


New submission from Florian Bruhin:

logging.basicConfig uses **kwargs and does not validate them.

This caused me to shoot myself in the foot multiple times by passing logLevel instead of level accidentally, and then trying to figure out why my messages don't get logged.

----------
messages: 233756
nosy: The Compiler, vinay.sajip
priority: normal
severity: normal
status: open
title: logging.basicConfig does not validate keyword arguments
type: behavior
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23207>
_______________________________________

From report at bugs.python.org  Fri Jan  9 16:34:55 2015
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 09 Jan 2015 15:34:55 +0000
Subject: [New-bugs-announce] [issue23208] asyncio: add
	BaseEventLoop._current_handle
Message-ID: <1420817695.78.0.447471935777.issue23208@psf.upfronthosting.co.za>


New submission from STINNER Victor:

One pain point of asynchronous programming is to understand bugs and rebuild the chain of callbacks / coroutines / tasks. I added the source traceback to Handle, Future (Task) and CoroWrapper to help debugging.

Here is a new enhancement to provide more context in debug mode: add BaseEventLoop._current_handle which is the handle currently executed.

The first usage is the call_exception_handler() which logs the source traceback of the current handle in debug mode.

Example:
---
import asyncio

def bug():
    loop.call_exception_handler({'message': 'bug!'})

def schedule_bug():
    bug()

loop = asyncio.get_event_loop()
loop.call_soon(schedule_bug)
loop.call_later(1, loop.stop)
loop.run_forever()
loop.close()
---

Output in debug mode, without the patch:
---
bug!
---

Output in debug mode, with the patch:
---
bug!
handle_traceback: Handle created at (most recent call last):
  File "x.py", line 10, in <module>
    loop.call_soon(schedule_bug)
---

Later, I plan to use the source traceback of the current handle in more places. For example, use it to log messages.

I would like to know "who" logged the "SSL handshake failed". At the beginning, I wanted to add a source traceback to all transports, but it looks simpler to get the source traceback of the current handler. Moreover, this traceback is more useful than the source traceback of the transport.

Previous try to add the source traceback to transports:
https://code.google.com/p/tulip/issues/detail?id=212

----------
components: asyncio
files: current_handle.patch
keywords: patch
messages: 233759
nosy: gvanrossum, haypo, yselivanov
priority: normal
severity: normal
status: open
title: asyncio: add BaseEventLoop._current_handle
versions: Python 3.4, Python 3.5
Added file: http://bugs.python.org/file37655/current_handle.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23208>
_______________________________________

From report at bugs.python.org  Fri Jan  9 18:16:15 2015
From: report at bugs.python.org (Martin Richard)
Date: Fri, 09 Jan 2015 17:16:15 +0000
Subject: [New-bugs-announce] [issue23209] asyncio: break some cycles
Message-ID: <1420823775.21.0.0191485458241.issue23209@psf.upfronthosting.co.za>


New submission from Martin Richard:

Hi,

I would like to submit 3 trivial modifications which break a cycle each. It is not much, but those three cycles caused a lot of objects to be garbage collected. They can now be freed using the reference counting mechanism, and therefore, reduce the latency that may be involved by the work of the garbage collector in a long living process.

In asyncio/base_subprocess.py:
WriteSubprocessPipeProto.proc is a reference to a BaseSubprocessTransport object, which holds a reference to the WriteSubprocessPipeProto in self._protocol.

I break the cycle in the protocol at the end of connection_lost().

In asyncio/futures.py:
wrap_future() defines a lambda which uses a variable defined in the function, therefore creating a closure, referencing the wrap_future() function and creating a cycle.

In the (really trivial) patch, the lambda uses the argument "future" instead of the "fut" variable defined in a closure. The closure is not needed anymore.

This single cycle is very common, because caused when one uses getaddrinfo().

In asyncio/selectors.py:
_BaseSelectorImpl._map keeps a reference to the _SelectorMapping object, which also references the selector with _SelectorMapping._selector.

The reference to the map in the selector is cleared once the selector is closed.

----------
files: break-some-cycles.diff
keywords: patch
messages: 233770
nosy: martius
priority: normal
severity: normal
status: open
title: asyncio: break some cycles
Added file: http://bugs.python.org/file37657/break-some-cycles.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23209>
_______________________________________

From report at bugs.python.org  Fri Jan  9 18:32:38 2015
From: report at bugs.python.org (Ethan Furman)
Date: Fri, 09 Jan 2015 17:32:38 +0000
Subject: [New-bugs-announce] [issue23210] clarify "virtual sequence" and
	range docs
Message-ID: <1420824758.29.0.716744408314.issue23210@psf.upfronthosting.co.za>


New submission from Ethan Furman:

The help() function explains range as being a "virtual sequence", but that phrase cannot be found anywhere in the docs.

Suggestion is to insert a phrase below:

>> The advantage of the range type over a regular list or tuple is
>> that a range object

 is a virtual sequence and so

>> will always take the same (small) amount of memory [...]

Bonus points for adding a glossary entry.  :)

----------
assignee: docs at python
components: Documentation
messages: 233771
nosy: docs at python, ethan.furman
priority: low
severity: normal
stage: needs patch
status: open
title: clarify "virtual sequence" and range docs
versions: Python 3.3, Python 3.4, Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23210>
_______________________________________

From report at bugs.python.org  Fri Jan  9 20:43:52 2015
From: report at bugs.python.org (Geoffrey Spear)
Date: Fri, 09 Jan 2015 19:43:52 +0000
Subject: [New-bugs-announce] [issue23211] test.test_logging.SMTPHandlerTest
	failing on Snow Leopard
Message-ID: <1420832632.4.0.00679469135671.issue23211@psf.upfronthosting.co.za>


New submission from Geoffrey Spear:

This seems to be related to issue20605 where _socket.getaddrinfo() mysteriously fails on some Snow Leopard systems but not others; I don't think the cause of that one was ever explained but this appears to be the same error:


======================================================================
ERROR: test_basic (test.test_logging.SMTPHandlerTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/geoff/Documents/programming/cpython/Lib/test/test_logging.py", line 930, in test_basic
    sockmap)
  File "/Users/geoff/Documents/programming/cpython/Lib/test/test_logging.py", line 687, in __init__
    decode_data=True)
  File "/Users/geoff/Documents/programming/cpython/Lib/smtpd.py", line 654, in __init__
    type=socket.SOCK_STREAM)
  File "/Users/geoff/Documents/programming/cpython/Lib/socket.py", line 730, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno 8] nodename nor servname provided, or not known

----------
components: Macintosh, Tests
messages: 233779
nosy: geoffreyspear, ned.deily, ronaldoussoren
priority: normal
severity: normal
status: open
title: test.test_logging.SMTPHandlerTest failing on Snow Leopard
type: behavior
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23211>
_______________________________________

From report at bugs.python.org  Fri Jan  9 21:05:10 2015
From: report at bugs.python.org (Ned Deily)
Date: Fri, 09 Jan 2015 20:05:10 +0000
Subject: [New-bugs-announce] [issue23212] Update Windows and OS X installer
	copies of OpenSSL to	1.0.1k
Message-ID: <1420833910.43.0.203396056169.issue23212@psf.upfronthosting.co.za>


New submission from Ned Deily:

https://www.openssl.org/news/secadv_20150108.txt

----------
components: Build, Macintosh, Windows
messages: 233780
nosy: ned.deily, ronaldoussoren, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
stage: needs patch
status: open
title: Update Windows and OS X installer copies of OpenSSL to 1.0.1k
versions: Python 2.7, Python 3.4, Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23212>
_______________________________________

From report at bugs.python.org  Sat Jan 10 01:02:11 2015
From: report at bugs.python.org (Thomas D.)
Date: Sat, 10 Jan 2015 00:02:11 +0000
Subject: [New-bugs-announce] [issue23213] subprocess communicate() hangs
	when stderr isn't closed
Message-ID: <1420848131.85.0.905282247977.issue23213@psf.upfronthosting.co.za>


New submission from Thomas D.:

Hi,

to demonstrate the problem you need >=systemd-217:

# python3.4
Python 3.4.2 (default, Oct 12 2014, 20:09:43)
[GCC 4.8.3] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import subprocess
>>> sp = subprocess.Popen(["/sbin/udevd", "--daemon"], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
>>> out, err = sp.communicate()

[hangs]

"ps" will show

root       9619  0.0  0.1  23340  5404 pts/5    Ss   Jan09   0:00  \_ -bash
root      13291  0.0  0.2  45352  9784 pts/5    S+   00:34   0:00      \_ python3.4
root      13311  0.0  0.0      0     0 pts/5    Z+   00:34   0:00          \_ [udevd] <defunct>

Calling "/sbin/udevd --daemon" from the shell works fine.

>>> errorlog = open("/tmp/stderr.log", "wb")
>>> sp = subprocess.Popen(["/sbin/udevd", "--daemon"], stdout=subprocess.PIPE, stderr=errorlog)

works, too.

The problem first appeared in systemd-217. I bisected systemd's source code and the commit since when Python's subprocess module is unable to start udevd is

https://github.com/systemd/systemd/commit/5c67cf2774a8b964f4d7cd92a4c447da81ac6087


This is not a systemd/udev only problem. The problem was first seen with the php-fpm daemon from PHP (but only when using "error_log = syslog").

Please see the original bug report at https://github.com/saltstack/salt/issues/14957 for more details.

Because Salt is still at Python 2.7, the problem can be reproduced with Python 2.7, too.


Is it a bug in subprocess? In systemd/PHP? Are we (salt) using subprocess the wrong way?

Thanks!


PS:
On your system, "/sbin/udevd" will be probably "/lib/systemd/systemd-udevd"

Not sure if this is related to http://bugs.python.org/issue12786 in some ways.

----------
components: Library (Lib)
messages: 233788
nosy: whissi
priority: normal
severity: normal
status: open
title: subprocess communicate() hangs when stderr isn't closed
type: behavior
versions: Python 2.7, Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23213>
_______________________________________

From report at bugs.python.org  Sat Jan 10 02:46:22 2015
From: report at bugs.python.org (Martin Panter)
Date: Sat, 10 Jan 2015 01:46:22 +0000
Subject: [New-bugs-announce] [issue23214] BufferedReader.read1(size)
	signature incompatible with	BufferedIOBase.read1(size=-1)
Message-ID: <1420854382.8.0.44417983014.issue23214@psf.upfronthosting.co.za>


New submission from Martin Panter:

I am trying to make LZMAFile (which implements BufferedIOBase) use a BufferedReader in read mode. However this broke test_lzma.FileTestCase.test_read1_multistream(), which calls read1() with the default size argument. This is because BufferedReader.read1() does not accept size=-1:

>>> stdin = open(0, "rb", closefd=False)
>>> stdin
<_io.BufferedReader name=0>
>>> stdin.read1()  # Parameter is mandatory
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: read1() takes exactly 1 argument (0 given)
>>> stdin.read1(-1)  # Does not accept the BufferedIOBase default
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: read length must be positive
>>> stdin.read1(0)  # Technically not positive
b''

Also, the BufferedIOBase documentation does not say what the size=-1 value means, only that it reads and returns up to -1 bytes.

----------
components: IO
messages: 233794
nosy: vadmium
priority: normal
severity: normal
status: open
title: BufferedReader.read1(size) signature incompatible with BufferedIOBase.read1(size=-1)
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23214>
_______________________________________

From report at bugs.python.org  Sat Jan 10 04:33:03 2015
From: report at bugs.python.org (Aleksi Torhamo)
Date: Sat, 10 Jan 2015 03:33:03 +0000
Subject: [New-bugs-announce] [issue23215] MemoryError with custom error
	handlers and multibyte	codecs
Message-ID: <1420860783.67.0.50434618074.issue23215@psf.upfronthosting.co.za>


New submission from Aleksi Torhamo:

Using a multibyte codec and a custom error handler that ignores errors to encode a string that contains characters not representable in said encoding causes exponential growth of the output buffer, raising MemoryError.

The problem is in multibytecodec_encerror() and REQUIRE_ENCODEBUFFER() in Modules/cjkcodecs/multibytecodec.c. multibytecodec_encerror() always uses REQUIRE_ENCODEBUFFER() to ensure there's enough space for the replacement string, and if more space is needed, REQUIRE_ENCODEBUFFER() calls expand_encodebuffer(), which in turn always grows the buffer by at least 50%. However, if size < 1, REQUIRE_ENCODEBUFFER() doesn't check if more space is actually needed. (It's used with negative values in other places)

I have no idea why the condition was originally size < 1 instead of size < 0, but changing it seems to fix this. The replacement string case is also the only use of the macro that may use 0 as the argument. 

In the patch, I've instead wrapped the REQUIRE_ENCODEBUFFER() (and memcpy) in a if(size > 0), since that's what the corresponding part in multibytecodec_decerror() did in the past:
https://hg.python.org/cpython/file/1c3f8d044589/Modules/cjkcodecs/multibytecodec.c#l438

Not sure which one makes more sense.

As for the tests, I'm not sure if 1) all of the affected encodings should be tested or only one (or even all encodings, affected or not?) and 2) whether it should be a new test or if I should just add it to test_longstrings in Lib/test/test_codeccallbacks.py. (Structurally it's a perfect fit, but it really isn't a "long string" test as it can happen with <50 characters) At the moment, the patch is testing affected encodings in a separate test.

Is the test philosophy "as thorough as possible" or "as fast as possible"?

----------
components: Interpreter Core
files: python_codec_crasher.py
messages: 233800
nosy: alexer
priority: normal
severity: normal
status: open
title: MemoryError with custom error handlers and multibyte codecs
type: resource usage
versions: Python 2.7, Python 3.2, Python 3.3, Python 3.4, Python 3.5
Added file: http://bugs.python.org/file37659/python_codec_crasher.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23215>
_______________________________________

From report at bugs.python.org  Sat Jan 10 10:15:06 2015
From: report at bugs.python.org (Al Sweigart)
Date: Sat, 10 Jan 2015 09:15:06 +0000
Subject: [New-bugs-announce] [issue23216] IDLE grep/find/replace source code
	needs docstrings
Message-ID: <1420881306.22.0.52075806524.issue23216@psf.upfronthosting.co.za>


New submission from Al Sweigart:

The following IDLE files need docstrings for their methods:

GrepDialog.py
SearchEngine.py
SearchDialogBase.py
SearchDialog.py
ReplaceDialog.py

----------
components: IDLE
files: idle_docstrings.diff
keywords: patch
messages: 233807
nosy: Al.Sweigart
priority: normal
severity: normal
status: open
title: IDLE grep/find/replace source code needs docstrings
type: enhancement
versions: Python 3.5
Added file: http://bugs.python.org/file37663/idle_docstrings.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23216>
_______________________________________

From report at bugs.python.org  Sat Jan 10 10:33:16 2015
From: report at bugs.python.org (Raymond Hettinger)
Date: Sat, 10 Jan 2015 09:33:16 +0000
Subject: [New-bugs-announce] [issue23217] help() function incorrectly
	captures comment preceding a	nested function
Message-ID: <1420882396.37.0.573730184021.issue23217@psf.upfronthosting.co.za>


New submission from Raymond Hettinger:

The help() function mysteriously captures a comment on the line preceding an inner function definition in a nested scope.

Given this code:
----------------

    def outer(func):
        #comment
        def inner():
            return
        return inner

    @outer
    def f():
        return


Calling help(f) produces:
-------------------------

    Help on function inner in module __main__:

    inner()
        #comment

----------
components: Library (Lib)
messages: 233811
nosy: rhettinger
priority: normal
severity: normal
status: open
title: help() function incorrectly captures comment preceding a nested function
type: behavior
versions: Python 3.2, Python 3.3, Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23217>
_______________________________________

From report at bugs.python.org  Sat Jan 10 11:00:51 2015
From: report at bugs.python.org (Al Sweigart)
Date: Sat, 10 Jan 2015 10:00:51 +0000
Subject: [New-bugs-announce] [issue23218] Modernize the IDLE
	Find/Replace/Find in Files dialogs
Message-ID: <1420884051.8.0.416866319984.issue23218@psf.upfronthosting.co.za>


New submission from Al Sweigart:

Various changes to modernize the user interface of the Find, Replace, and Find in Files dialogs:

- Got rid of the "close" button, since it is redundant.
- Moved the buttons below the text fields, to make better use of screen real estate.
- Shorten the titles by getting rid of the "Dialog" part.
- Renamed "Search" title to "Find" to match the menu item.
- Renamed "Regular expression" label to the shorter "Regex"
- Added slightly more padding.
- Removed the "Replace+Find" and made "Replace" have this functionality. (all modern editors use Replace to mean "Replace and Find")

The attached png shows the before/after differences in the design of these dialogs.

----------
components: IDLE
files: idle_grep_compare.png
messages: 233812
nosy: Al.Sweigart
priority: normal
severity: normal
status: open
title: Modernize the IDLE Find/Replace/Find in Files dialogs
type: enhancement
versions: Python 3.5
Added file: http://bugs.python.org/file37664/idle_grep_compare.png

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23218>
_______________________________________

From report at bugs.python.org  Sat Jan 10 11:02:13 2015
From: report at bugs.python.org (STINNER Victor)
Date: Sat, 10 Jan 2015 10:02:13 +0000
Subject: [New-bugs-announce] [issue23219] asyncio: cancelling wait_for(task,
	timeout) must also cancel the task
Message-ID: <1420884133.75.0.926989519431.issue23219@psf.upfronthosting.co.za>


New submission from STINNER Victor:

Cancelling wait_for(task, timeout) must also cancel the waited task. Attached patch fixes this issue.

Original issue:
https://code.google.com/p/tulip/issues/detail?id=211

----------
components: asyncio
files: cancel_wait_for.patch
keywords: patch
messages: 233813
nosy: gvanrossum, haypo, yselivanov
priority: normal
severity: normal
status: open
title: asyncio: cancelling wait_for(task, timeout) must also cancel the task
versions: Python 3.4, Python 3.5
Added file: http://bugs.python.org/file37665/cancel_wait_for.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23219>
_______________________________________

From report at bugs.python.org  Sun Jan 11 01:54:47 2015
From: report at bugs.python.org (Al Sweigart)
Date: Sun, 11 Jan 2015 00:54:47 +0000
Subject: [New-bugs-announce] [issue23220] IDLE does not display \b backspace
	correctly.
Message-ID: <1420937687.17.0.885670059147.issue23220@psf.upfronthosting.co.za>


New submission from Al Sweigart:

IDLE cannot display the \b backspace character correctly. From IDLE's interactive shell:

>>> print('hello\b\b\b\b\bHELLO')
hello?????HELLO

Whereas from cmd:

>>> print('hello\b\b\b\b\bHELLO')
HELLO

----------
messages: 233828
nosy: Al.Sweigart
priority: normal
severity: normal
status: open
title: IDLE does not display \b backspace correctly.
versions: Python 3.4, Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23220>
_______________________________________

From report at bugs.python.org  Sun Jan 11 02:39:20 2015
From: report at bugs.python.org (Arfrever Frehtes Taifersar Arahesis)
Date: Sun, 11 Jan 2015 01:39:20 +0000
Subject: [New-bugs-announce] [issue23221] "a(n) the", "the a(n)" typos
Message-ID: <1420940360.26.0.834732182943.issue23221@psf.upfronthosting.co.za>


New submission from Arfrever Frehtes Taifersar Arahesis:

In default branch:

$ grep -Er '(^|[[:space:]])an? the($|[[:space:]])' *
Doc/c-api/structures.rst::c:type:`PyObject\*`, it is common that the method implementation uses a the
Include/dynamic_annotations.h:     is about to be reused, or when a the locking discipline for a variable
Include/unicodeobject.h:/* Initializes the canonical string representation from a the deprecated
Lib/lib2to3/fixes/fix_exitfunc.py:        # First, find a the sys import. We'll just hope it's global scope.
Lib/socket.py:        Create a socket object from a the bytes object returned by
Lib/http/cookiejar.py:    """Return string representation of Cookie in an the LWP cookie file format.
Modules/_ctypes/_ctypes.c:       A Pointer instance must keep a the value it points to alive.  So, a

$ grep -Er '(^|[[:space:]])the an?($|[[:space:]])' *
Doc/library/unittest.mock.rst:being looked up on the a module and so we have to patch ``a.SomeClass`` instead::
Doc/c-api/init.rst:   Return the a pointer to the first :c:type:`PyThreadState` object in the list of
Doc/c-api/exceptions.rst:      case of a class exception, or it may the a subclass of the expected exception.)
Doc/distutils/apiref.rst:   *base_dir* is just the a name of a directory which doesn't necessarily exist
Lib/test/test_argparse.py:    """Tests the an optional action that is required"""
Lib/ssl.py:        """Return the a list of ciphers shared by the client during the
Lib/distutils/dir_util.py:    'base_dir' is just the a name of a directory which doesn't necessarily

'the a module' in Doc/library/unittest.mock.rst probably should be 'the ``a`` module'.
'may the a' in Doc/c-api/exceptions.rst should be 'may be a'.

----------
keywords: easy
messages: 233829
nosy: Arfrever
priority: normal
severity: normal
status: open
title: "a(n) the", "the a(n)" typos
versions: Python 2.7, Python 3.4, Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23221>
_______________________________________

From report at bugs.python.org  Sun Jan 11 13:03:34 2015
From: report at bugs.python.org (Ievgen Aleinikov)
Date: Sun, 11 Jan 2015 12:03:34 +0000
Subject: [New-bugs-announce] [issue23222] open doesn't provide traceback for
	int argument
Message-ID: <1420977814.07.0.566784617319.issue23222@psf.upfronthosting.co.za>


New submission from Ievgen Aleinikov:

HI,

I was able to gat such behaviour on python version 3.4.2 and 3.2.3
>>> for i in range(10):
...   f = open(i, "w")
...   f.close()
... 
This will close interactive session without any output.


On python 2 (2.7.9) it's not happening:
>>> for i in range(10):
...  f = open(i, "w")
...  f.close()
... 
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
TypeError: coercing to Unicode: need string or buffer, int found
>>>

And interpreter is not closed.

If run attached 1.py we'll see following:

python2.7 1.py 
Traceback (most recent call last):
  File "1.py", line 2, in <module>
    f = open(i, "w")
TypeError: coercing to Unicode: need string or buffer, int found


python3.4 1.py -> nothing happens (echo $? -> 1)



Kind regards,
Ievgen.

----------
components: IO
files: 1.py
messages: 233844
nosy: slivabox
priority: normal
severity: normal
status: open
title: open doesn't provide traceback for int argument
versions: Python 3.2, Python 3.3, Python 3.4
Added file: http://bugs.python.org/file37676/1.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23222>
_______________________________________

From report at bugs.python.org  Sun Jan 11 15:58:55 2015
From: report at bugs.python.org (gajdig)
Date: Sun, 11 Jan 2015 14:58:55 +0000
Subject: [New-bugs-announce] [issue23223] subprocess32 unable to be
	installed via pip
Message-ID: <1420988335.84.0.852324840032.issue23223@psf.upfronthosting.co.za>


New submission from gajdig:

The latest subprocess32, 3.2.6, is unable to be installed in Windows 7 via pip. 

The error messages:

_posixsubprocess.c(10) : fatal error C1083: Cannot open include file: 'unistd.h': No such file or directory

error: command 'C:\\Users\\user1\\AppData\\Local\\Programs\\Common\\Microsoft\\Visual C++ for Python\\9.0\\VC\\Bin\\cl.exe' failed with exit status 2

Read that unistd.h is not a Windows file.

----------
components: Installation, Windows
messages: 233857
nosy: gajdig, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: subprocess32 unable to be installed via pip
type: compile error
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23223>
_______________________________________

From report at bugs.python.org  Mon Jan 12 01:58:28 2015
From: report at bugs.python.org (Martin Panter)
Date: Mon, 12 Jan 2015 00:58:28 +0000
Subject: [New-bugs-announce] [issue23224] LZMADecompressor object is only
	initialized in __init__
Message-ID: <1421024308.33.0.868171678499.issue23224@psf.upfronthosting.co.za>


New submission from Martin Panter:

Noticed in a patch review around LZMModules/_lzmamodule.c:1055 that the C-level LZMADecompressor object is being initialized in an __init__() method. It crashes if you create the object with __new__() and never call __init__():

>>> from lzma import LZMADecompressor
>>> LZMADecompressor.__new__(LZMADecompressor).decompress(bytes())
Segmentation fault (core dumped)
[Exit 139]

----------
components: Extension Modules
messages: 233866
nosy: vadmium
priority: normal
severity: normal
status: open
title: LZMADecompressor object is only initialized in __init__
type: crash
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23224>
_______________________________________

From report at bugs.python.org  Mon Jan 12 12:06:01 2015
From: report at bugs.python.org (STINNER Victor)
Date: Mon, 12 Jan 2015 11:06:01 +0000
Subject: [New-bugs-announce] [issue23225] selectors: raise an exception if
	the selector is closed
Message-ID: <1421060761.71.0.41386603707.issue23225@psf.upfronthosting.co.za>


New submission from STINNER Victor:

I propose to raise a RuntimeError exception on operations of a selector when the selector is closed.

I'm not sure that RuntimeError is the most common exception:

- io and gzip raise ValueError
- asyncio raises RuntimeError (and selectors is "linked" to asyncio)
- multiprocessing raises OSError
- dbm.dumb raises dbm.dumb.error

This issue is related to the issue #23209 which propopses to set the _map attribute to None when in the close() method.

----------
components: asyncio
files: close_selector.patch
keywords: patch
messages: 233881
nosy: gvanrossum, haypo, martius, neologix, yselivanov
priority: normal
severity: normal
status: open
title: selectors: raise an exception if the selector is closed
versions: Python 3.4, Python 3.5
Added file: http://bugs.python.org/file37682/close_selector.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23225>
_______________________________________

From report at bugs.python.org  Mon Jan 12 23:40:48 2015
From: report at bugs.python.org (Andrew Barnert)
Date: Mon, 12 Jan 2015 22:40:48 +0000
Subject: [New-bugs-announce] [issue23226] Add float linspace recipe to docs
Message-ID: <1421102448.57.0.289351503451.issue23226@psf.upfronthosting.co.za>


New submission from Andrew Barnert:

In a recent thread on python-ideas (https://mail.python.org/pipermail/python-ideas/2015-January/030817.html), it was concluded that Python should not have a range-like type for floats in the stdlib, but there should be some simple discussion of the alternatives in the docs.

The attached diff describes the issues, demonstrates the simplest reasonable algorithm, and links to a recipe on ActiveState (which builds the same algorithm into a lazy sequence, and links to other recipes with different alternatives).

----------
assignee: docs at python
components: Documentation
files: stdtypes.rst.diff
keywords: patch
messages: 233893
nosy: abarnert, docs at python
priority: normal
severity: normal
status: open
title: Add float linspace recipe to docs
type: enhancement
versions: Python 3.5
Added file: http://bugs.python.org/file37684/stdtypes.rst.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23226>
_______________________________________

From report at bugs.python.org  Tue Jan 13 08:35:46 2015
From: report at bugs.python.org (Stephen Drake)
Date: Tue, 13 Jan 2015 07:35:46 +0000
Subject: [New-bugs-announce] [issue23227] Generator's finally block not run
	if close() called	before first iteration
Message-ID: <1421134546.13.0.154248882029.issue23227@psf.upfronthosting.co.za>


New submission from Stephen Drake:

If a generator has its close() method called before any items are requested from it, a finally block in the generator function will not be executed.

I encountered this when wrapping an open file to alter the result of iterating over it.  Using a generator function with a try/finally block seemed like a simple way of acheiving this.  Here's an example that logs each line as it's read:

def logged_lines(f):
    try:
        for line in f:
            logging.warning(line.strip())
            yield line
    finally:
        logging.warning('closing')
        f.close()

If the generator is created and closed immediately, the underlying file-like object is left open:
>>> f = urlopen('https://docs.python.org/')
>>> lines = logged_lines(f)
>>> lines.close()
>>> f.closed
False

But once the first item is requested from the generator, close() will trigger cleanup:
>>> lines = logged_lines(f)
>>> next(lines)
WARNING:root:<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
'<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"\n'
>>> lines.close()
WARNING:root:closing
>>> f.closed
True

Having read the documentation for yield expressions, I don't believe this behaviour to be non-conformant - but it still seems like a bit of a gotcha to me.  Should this usage be warned against?

----------
assignee: docs at python
components: Documentation, Interpreter Core
messages: 233903
nosy: docs at python, sjdrake
priority: normal
severity: normal
status: open
title: Generator's finally block not run if close() called before first iteration
type: behavior
versions: Python 2.7, Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23227>
_______________________________________

From report at bugs.python.org  Tue Jan 13 09:55:02 2015
From: report at bugs.python.org (Michael Vogt)
Date: Tue, 13 Jan 2015 08:55:02 +0000
Subject: [New-bugs-announce] [issue23228] Crashes when tarfile contains a
	symlink and unpack	directory contain it too
Message-ID: <1421139302.16.0.710158118765.issue23228@psf.upfronthosting.co.za>


New submission from Michael Vogt:

The tarfile.makelink() code crashes with a maximum recursion error when it unpacks a tarfile that contains a symlink into a directory that already contains this symlink.

Attached is a standalone testcase (that probably better explains whats going on :) and a possible fix.

----------
components: Library (Lib)
files: tarfile-extract-crash.py
messages: 233907
nosy: mvo
priority: normal
severity: normal
status: open
title: Crashes when tarfile contains a symlink and unpack directory contain it too
type: crash
Added file: http://bugs.python.org/file37688/tarfile-extract-crash.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23228>
_______________________________________

From report at bugs.python.org  Tue Jan 13 10:37:07 2015
From: report at bugs.python.org (Mark Dickinson)
Date: Tue, 13 Jan 2015 09:37:07 +0000
Subject: [New-bugs-announce] [issue23229] add inf and nan to cmath module
Message-ID: <1421141827.81.0.668754742456.issue23229@psf.upfronthosting.co.za>


New submission from Mark Dickinson:

As pointed out by Guido in issue 23185, the constants `inf` and `nan` should be added to the cmath module, too.

----------
assignee: mark.dickinson
components: Extension Modules
messages: 233919
nosy: mark.dickinson
priority: normal
severity: normal
status: open
title: add inf and nan to cmath module
type: enhancement
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23229>
_______________________________________

From report at bugs.python.org  Tue Jan 13 13:18:29 2015
From: report at bugs.python.org (Luis G.F)
Date: Tue, 13 Jan 2015 12:18:29 +0000
Subject: [New-bugs-announce] [issue23230] Bug parsing integers with zero
	padding
Message-ID: <1421151509.05.0.199234609043.issue23230@psf.upfronthosting.co.za>


New submission from Luis G.F:

Python 3.4 interpreter fail to parse a integer that has zero padding, whereas python 2.7 works properly.

Python 2.7.6 (default, Mar 22 2014, 22:59:56) 
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> int(001)
1
>>> 

Python 3.4.0 (default, Apr 11 2014, 13:05:11) 
[GCC 4.8.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> int(001)
  File "<stdin>", line 1
    int(001)
          ^
SyntaxError: invalid token
>>>

----------
components: Interpreter Core
messages: 233928
nosy: luisgf
priority: normal
severity: normal
status: open
title: Bug parsing integers with zero padding
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23230>
_______________________________________

From report at bugs.python.org  Tue Jan 13 13:48:19 2015
From: report at bugs.python.org (Martin Panter)
Date: Tue, 13 Jan 2015 12:48:19 +0000
Subject: [New-bugs-announce] [issue23231] Fix codecs.iterencode/decode() by
	allowing data	parameter to be omitted
Message-ID: <1421153299.35.0.701583632763.issue23231@psf.upfronthosting.co.za>


New submission from Martin Panter:

As mentioned in Issue 20132, iterencode() and iterdecode() only work on text-to-byte codecs, because they assume particular data types when finalizing the incremental codecs. This patch changes the signature of the IncrementalEncoder and IncrementalDecoder methods from

IncrementalEncoder.encode(object[, final])
IncrementalEncoder.decode(object[, final])

to

IncrementalEncoder.encode([object,] [final])
IncrementalEncoder.decode([object,] [final])

so that iteren/decode(), and perhaps in the future, StreamWriter/Reader, can operate the incremental codec without knowing what kind of data should be processed.

----------
components: Library (Lib), Unicode
files: final-no-object.patch
keywords: patch
messages: 233932
nosy: ezio.melotti, haypo, vadmium
priority: normal
severity: normal
status: open
title: Fix codecs.iterencode/decode() by allowing data parameter to be omitted
type: behavior
versions: Python 3.5
Added file: http://bugs.python.org/file37691/final-no-object.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23231>
_______________________________________

From report at bugs.python.org  Tue Jan 13 16:04:27 2015
From: report at bugs.python.org (Jan Kaliszewski)
Date: Tue, 13 Jan 2015 15:04:27 +0000
Subject: [New-bugs-announce] [issue23232] 'codecs' module functionality +
	its docs -- concerning	custom codecs, especially non-string ones
Message-ID: <1421161467.08.0.0207328104239.issue23232@psf.upfronthosting.co.za>


New submission from Jan Kaliszewski:

To some extent, this issue is a follow-up of Issue 20132. It concerns some parts of functionality + documentation of the 'codecs' module related to registering custom codecs, especially non-string ones (i.e., codecs that encode/decode between arbitrary types, not necessarily the str and bytes types).

A few fragments of documented behaviour and/or documentation itself bother me:


0. Ad "7.2.1. Codec Base Classes"

"Each codec has to define four interfaces to make it usable as codec in Python: stateless encoder, stateless decoder, stream reader and stream writer. The stream reader and writers typically reuse the stateless encoder/decoder to implement the file protocols. Codec authors also need to define how the codec will handle encoding and decoding errors."

IMHO it is still unclear:

a) what is the relation between codecs in this meaning and CodecInfo objects? (especially: CodecInfo contains information about six interfaces, not four)

b) How codec authors define "how the codec will handle encoding and decoding errors"? What is relation between this and error handling schemes (defined as generic, not per-codec ones) documented below? 


1. Ad "7.2.1.1. Error Handlers" and "codecs.strict_errors(exception)"

"'strict' 	Raise UnicodeError (or a subclass); this is the default. Implemented in strict_errors()."

"codecs.strict_errors(exception)
Implements the 'strict' error handling: each encoding or decoding error raises a UnicodeError."

Is it true that always it is a UnicodeError or its subclass and not just ValueError or its subclass? (as it is described in other fragments of the module documentation).

Please note, that 'strict' is documented as a universal (and not e.g. text-encoding-only) error handling scheme. So, what about non-string codecs?


2. Ad "codecs.register_error(name, error_handler)"

"For encoding, error_handler will be called with a UnicodeEncodeError instance..." "Decoding and translating works similarly, except UnicodeDecodeError or UnicodeTranslateError will be passed..."

Again: what about non-string codecs? UnicodeError subclasses do not seem to be appropriate for them.


3. It would be nice to address the Zoinkity's concerns from the Issue 20132 (partially related to the above points):

"""
One glaring omission is any information about multibyte codecs--the class, its methods, and how to even define one.  

Also, the primary use for codecs.register would be to append a single codec to the lookup registry.  Simple usage of the method only provides lookup for the provided codecs and will not include regularly-accessible ones such as "utf-8".  It would be enormously helpful to provide an example of proper, safe usage.
"""

----------
assignee: docs at python
components: Documentation, Library (Lib)
messages: 233940
nosy: docs at python, zuo
priority: normal
severity: normal
status: open
title: 'codecs' module functionality + its docs -- concerning custom codecs, especially non-string ones
versions: Python 3.4, Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23232>
_______________________________________

From report at bugs.python.org  Tue Jan 13 21:18:15 2015
From: report at bugs.python.org (Serhiy Storchaka)
Date: Tue, 13 Jan 2015 20:18:15 +0000
Subject: [New-bugs-announce] [issue23233] TypeError in ./setup.py
Message-ID: <1421180295.13.0.16794585619.issue23233@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka:

After updating outdated workspace to current sources I got following error:

$ make
Traceback (most recent call last):
  File "./setup.py", line 26, in <module>
    sysconfig.get_config_vars()['CFLAGS'] = cflags + ' ' + py_cflags_nodist
TypeError: Can't convert 'NoneType' object to str implicitly
make: *** [sharedmods] Error 1

`make distclean` doesn't help.

Here is a patch which fixes symptoms but may be right solution should change something other.

----------
components: Build
files: setup_py_cflags_nodist.diff
keywords: patch
messages: 233966
nosy: serhiy.storchaka
priority: normal
severity: normal
status: open
title: TypeError in ./setup.py
type: compile error
versions: Python 3.5
Added file: http://bugs.python.org/file37695/setup_py_cflags_nodist.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23233>
_______________________________________

From report at bugs.python.org  Wed Jan 14 00:41:01 2015
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 13 Jan 2015 23:41:01 +0000
Subject: [New-bugs-announce] [issue23234] refactor subprocess: use new
	OSError exceptions, factorize stdin.write() code
Message-ID: <1421192461.64.0.911516234193.issue23234@psf.upfronthosting.co.za>


New submission from STINNER Victor:

Attached patch refactors subprocess code:

- use new OSError exceptions
- factorize stdin.write() code

----------
components: Extension Modules
files: subprocess.patch
keywords: patch
messages: 233974
nosy: haypo, serhiy.storchaka
priority: normal
severity: normal
status: open
title: refactor subprocess: use new OSError exceptions, factorize stdin.write() code
type: enhancement
versions: Python 3.4, Python 3.5
Added file: http://bugs.python.org/file37696/subprocess.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23234>
_______________________________________

From report at bugs.python.org  Wed Jan 14 01:04:35 2015
From: report at bugs.python.org (Berker Peksag)
Date: Wed, 14 Jan 2015 00:04:35 +0000
Subject: [New-bugs-announce] [issue23235] run_tests.py doesn't set
	test.support.verbose correctly
Message-ID: <1421193875.73.0.411637461646.issue23235@psf.upfronthosting.co.za>


New submission from Berker Peksag:

"make buildbottest" uses Tools/scripts/run_tests.py and it doesn't set test.support.verbose to 0.

Example output from a buildbot:

    [ 60/389] test_codecmaps_hk
    	fetching http://www.pythontest.net/unicode/BIG5HKSCS-2004.TXT ...

http://buildbot.python.org/all/builders/System%20Z%20Linux%203.4/builds/683/steps/test/logs/stdio

It works as expected if you use regrtest directly with empty Lib/test/data/ directory:

    $ ./python -m test test_codecmaps_hk
    [1/1] test_codecmaps_hk
    1 test OK.

----------
components: Demos and Tools
messages: 233976
nosy: berker.peksag
priority: normal
severity: normal
stage: needs patch
status: open
title: run_tests.py doesn't set test.support.verbose correctly
type: behavior
versions: Python 3.4, Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23235>
_______________________________________

From report at bugs.python.org  Wed Jan 14 01:48:28 2015
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 14 Jan 2015 00:48:28 +0000
Subject: [New-bugs-announce] [issue23236] asyncio: add timeout to
	StreamReader read methods
Message-ID: <1421196508.1.0.44699876054.issue23236@psf.upfronthosting.co.za>


New submission from STINNER Victor:

Attached patch adds an optional timeout parameter to the read(), read_exactly() and readline() methods of StreamReader.

If a single read takes longer than timeout seconds, a asyncio.TimeoutError is raised. The timeout is reset each time new data is received.

Read data is pushed back to the StreamReader buffer if the wait raises an exception (timeout, cancelled, etc.), so it's possible to retry the read later.

----------
components: asyncio
files: streamreader_timeout.patch
keywords: patch
messages: 233978
nosy: gvanrossum, haypo, yselivanov
priority: normal
severity: normal
status: open
title: asyncio: add timeout to StreamReader read methods
versions: Python 3.4, Python 3.5
Added file: http://bugs.python.org/file37697/streamreader_timeout.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23236>
_______________________________________

From report at bugs.python.org  Wed Jan 14 03:13:03 2015
From: report at bugs.python.org (Michiel de Hoon)
Date: Wed, 14 Jan 2015 02:13:03 +0000
Subject: [New-bugs-announce] [issue23237] Interrupts are lost during
	readline PyOS_InputHook	processing (reopening)
Message-ID: <1421201583.23.0.507661432161.issue23237@psf.upfronthosting.co.za>


New submission from Michiel de Hoon:

This bug was previously reported in
http://bugs.python.org/issue3180
but was closed after seven years for being out of date.
Still, the bug remains: Interrupts are lost during readline PyOS_InputHook processing.

To reproduce the bug, try

>>> from Tkinter import *
>>> Tk()

You will notice that Ctrl-C now does not generate a KeybordInterrupt.

This bug appears in all Python versions I tested (Python 2.7, 3.3, 3.4) and, judging from the source code, is still present in the current Python source code in Mercurial. The bug appears both on Mac and on Linux; I do not have access to Windows.

The suggested patch at http://bugs.python.org/issue3180 makes multiple changes to the behavior of PyOS_InputHook. In particular, it allows for waiting for a file descriptor other than stdin. I think that this is unrelated to the current bug, so it should not be included in the patch, in particular since that would change the API.

----------
components: Extension Modules
messages: 233997
nosy: Michiel.de.Hoon
priority: normal
severity: normal
status: open
title: Interrupts are lost during readline PyOS_InputHook processing (reopening)
type: behavior
versions: Python 2.7, Python 3.2, Python 3.3, Python 3.4, Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23237>
_______________________________________

From report at bugs.python.org  Wed Jan 14 09:59:16 2015
From: report at bugs.python.org (memkmemk)
Date: Wed, 14 Jan 2015 08:59:16 +0000
Subject: [New-bugs-announce] [issue23238] http.server failed to decode '+'
	to ' ' when calling cgi	script/executable in window
Message-ID: <1421225956.24.0.0678998380513.issue23238@psf.upfronthosting.co.za>


New submission from memkmemk:

The actual commandline generated from http request does not decode '+' back to ' '

----------
messages: 234016
nosy: memkmemk
priority: normal
severity: normal
status: open
title: http.server failed to decode '+' to ' ' when calling cgi script/executable in window
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23238>
_______________________________________

From report at bugs.python.org  Wed Jan 14 10:04:52 2015
From: report at bugs.python.org (=?utf-8?b?w4Fkw6FtIFpzaWdtb25k?=)
Date: Wed, 14 Jan 2015 09:04:52 +0000
Subject: [New-bugs-announce] [issue23239] SSL match_hostname does not accept
	IP Address
Message-ID: <1421226292.77.0.451398063166.issue23239@psf.upfronthosting.co.za>


New submission from ?d?m Zsigmond:

ssl.match_hostname does not accept the ca certificate if the hostname matches the ip address. 

I am trying to connect to a servert with a cacert by IP address but I get an error message like: '42.42.42.42' hostname does not match '<hostname_in_cacert>'

The IP Address is in the ca certificate, so it should be accepted.

----------
components: Extension Modules
messages: 234017
nosy: ?d?m.Zsigmond
priority: normal
severity: normal
status: open
title: SSL match_hostname does not accept IP Address
type: security
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23239>
_______________________________________

From report at bugs.python.org  Wed Jan 14 13:04:03 2015
From: report at bugs.python.org (arnaud gaboury)
Date: Wed, 14 Jan 2015 12:04:03 +0000
Subject: [New-bugs-announce] [issue23240] pip 6.0.6- pip install command is
	broken
Message-ID: <1421237043.78.0.734461978939.issue23240@psf.upfronthosting.co.za>


New submission from arnaud gaboury:

gabx at hortensia ?? ~ % pip3.4 install websocket-client
Exception:
Traceback (most recent call last):
  File "/usr/lib/python3.4/site-packages/pip/basecommand.py", line 232, in main
    status = self.run(options, args)
  File "/usr/lib/python3.4/site-packages/pip/commands/install.py", line 339, in run
    requirement_set.prepare_files(finder)
  File "/usr/lib/python3.4/site-packages/pip/req/req_set.py", line 229, in prepare_files
    req_to_install.check_if_exists()
  File "/usr/lib/python3.4/site-packages/pip/req/req_install.py", line 928, in check_if_exists
    self.satisfied_by = pkg_resources.get_distribution(self.req)
  File "/usr/lib/python3.4/site-packages/pip/_vendor/pkg_resources/__init__.py", line 461, in get_distribution
    dist = get_provider(dist)
  File "/usr/lib/python3.4/site-packages/pip/_vendor/pkg_resources/__init__.py", line 341, in get_provider
    return working_set.find(moduleOrReq) or require(str(moduleOrReq))[0]
  File "/usr/lib/python3.4/site-packages/pip/_vendor/pkg_resources/__init__.py", line 870, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/usr/lib/python3.4/site-packages/pip/_vendor/pkg_resources/__init__.py", line 740, in resolve
    env = Environment(self.entries)
  File "/usr/lib/python3.4/site-packages/pip/_vendor/pkg_resources/__init__.py", line 927, in __init__
    self.scan(search_path)
  File "/usr/lib/python3.4/site-packages/pip/_vendor/pkg_resources/__init__.py", line 957, in scan
    self.add(dist)
  File "/usr/lib/python3.4/site-packages/pip/_vendor/pkg_resources/__init__.py", line 977, in add
    dists.sort(key=operator.attrgetter('hashcmp'), reverse=True)
TypeError: unorderable types: NoneType() < str()
---------------------------------------------------------
gabx at hortensia ?? ~ % pip list   
aiohttp (0.9.1)
apparmor (2.9.1)
appdirs (1.3.0)
.................
--------------------------------------------------
gabx at hortensia ?? ~ % pip show six
---
Name: six
Version: 1.9.0
Location: /usr/lib/python3.4/site-packages
Requires: 
------------------------------------------
gabx at hortensia ?? ~ % pip install ranger         
Requirement already satisfied (use --upgrade to upgrade): ranger in /usr/lib/python3.4/site-packages
------------------------------------------

pip install seems broken when installing a new package.

----------
components: Distutils
messages: 234021
nosy: dstufft, eric.araujo, gabx
priority: normal
severity: normal
status: open
title: pip 6.0.6- pip install command is broken
type: crash
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23240>
_______________________________________

From report at bugs.python.org  Wed Jan 14 14:02:56 2015
From: report at bugs.python.org (Michael Kesper)
Date: Wed, 14 Jan 2015 13:02:56 +0000
Subject: [New-bugs-announce] [issue23241] shutil should accept pathlib types
Message-ID: <1421240576.75.0.719151648499.issue23241@psf.upfronthosting.co.za>


New submission from Michael Kesper:

source_path = Path('../data')
destination_path = Path('//file_server/work/whatever')
for file_name in source_path.glob('xyz-*'):
    shutil.copyfile(source_path / file_name, destination_path / file_name)

leads to:

Traceback (most recent call last):
  File "copy_shares_2_work.py", line 9, in <module>
    shutil.copyfile(source_path / file_name, destination_path / file_name)
  File "C:\Python34\lib\shutil.py", line 90, in copyfile
    if _samefile(src, dst):
  File "C:\Python34\lib\shutil.py", line 75, in _samefile
    return os.path.samefile(src, dst)
  File "C:\Python34\lib\genericpath.py", line 90, in samefile
    s1 = os.stat(f1)
TypeError: argument should be string, bytes or integer, not WindowsPath

Converting to str() works but is unexpected.

----------
messages: 234025
nosy: mkesper
priority: normal
severity: normal
status: open
title: shutil should accept pathlib types
type: behavior
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23241>
_______________________________________

From report at bugs.python.org  Thu Jan 15 00:11:18 2015
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 14 Jan 2015 23:11:18 +0000
Subject: [New-bugs-announce] [issue23242] asyncio: Process must close the
	transport when the	process exit
Message-ID: <1421277078.82.0.431621848437.issue23242@psf.upfronthosting.co.za>


New submission from STINNER Victor:

The asyncio.subprocess.Process class never closes the subprocess transport, whereas this transport is private.

I propose to close the transport as soon as possible: when transport.get_returncode() is called and its result is not None.

I'm not sure that it's the most efficient way to close the transport. It may be better to close the transport in the connection_lost() method of the protocol.

The patch also checks in Process.communicate() that the process is alive, and is clears the reference to the transport.

----------
components: asyncio
files: subprocess_close.patch
keywords: patch
messages: 234042
nosy: gvanrossum, haypo, yselivanov
priority: normal
severity: normal
status: open
title: asyncio: Process must close the transport when the process exit
versions: Python 3.4, Python 3.5
Added file: http://bugs.python.org/file37707/subprocess_close.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23242>
_______________________________________

From report at bugs.python.org  Thu Jan 15 00:18:04 2015
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 14 Jan 2015 23:18:04 +0000
Subject: [New-bugs-announce] [issue23243] asyncio: emit ResourceWarning
	warnings if	transports/event loops are not explicitly closed
Message-ID: <1421277484.12.0.751563251482.issue23243@psf.upfronthosting.co.za>


New submission from STINNER Victor:

I propose to add destructors to transports and event loops which emit a ResourceWarning if they are not closed.

The change should help to detect resource leaks and bugs.

Attached patch implements this issue. It only adds destructors on Python 3.4 and later, because older Python versions don't implement the PEP 442 (Safe object finalization) and so objects part of reference cycle would never be deleted.

The patch adds a new _closed attribute to BaseSubprocessTransport and _SSLProtocolTransport classes, to track if the transport was closed or not.

The patch should help to find bugs like this one:
https://code.google.com/p/tulip/issues/detail?id=218

----------
components: asyncio
files: resource_warnings.patch
keywords: patch
messages: 234043
nosy: gvanrossum, haypo, yselivanov
priority: normal
severity: normal
status: open
title: asyncio: emit ResourceWarning warnings if transports/event loops are not explicitly closed
versions: Python 3.4, Python 3.5
Added file: http://bugs.python.org/file37708/resource_warnings.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23243>
_______________________________________

From report at bugs.python.org  Thu Jan 15 05:40:01 2015
From: report at bugs.python.org (N0v1ce Llve)
Date: Thu, 15 Jan 2015 04:40:01 +0000
Subject: [New-bugs-announce] [issue23244] doc typo in 'Glossary': 'generator'
Message-ID: <1421296801.34.0.273749037351.issue23244@psf.upfronthosting.co.za>


New submission from N0v1ce Llve:

'a series a values' in the description of 'generator' in 'Glossary' should be 'a series of values'. 

Take the online documentation for example, the whole sentence is - 
 
> It looks like a normal function except that it contains yield statements for producing **a series a values** usable in a for-loop or that can be retrieved one at a time with the next() function.

:)

----------
assignee: docs at python
components: Documentation
messages: 234047
nosy: docs at python, n0vicelive
priority: normal
severity: normal
status: open
title: doc typo in 'Glossary': 'generator'
versions: Python 2.7, Python 3.2, Python 3.3, Python 3.4, Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23244>
_______________________________________

From report at bugs.python.org  Thu Jan 15 07:01:34 2015
From: report at bugs.python.org (Douman)
Date: Thu, 15 Jan 2015 06:01:34 +0000
Subject: [New-bugs-announce] [issue23245] urllib2: urlopen() gets
	exception(kwargs bug?)
Message-ID: <1421301694.48.0.880647525512.issue23245@psf.upfronthosting.co.za>


New submission from Douman:

I get strange callstack from urllib2

It seems that python thinks that HTTPSConnection doesn't have context argument.
Which is entirely incorrect. 
I have suspicions that this is related to the way how context argument is passed(kwargs)
This happens starting from python 2.7.9

----------
components: Extension Modules
files: callstack_urllib2
messages: 234055
nosy: Douman
priority: normal
severity: normal
status: open
title: urllib2: urlopen() gets exception(kwargs bug?)
type: compile error
versions: Python 2.7
Added file: http://bugs.python.org/file37713/callstack_urllib2

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23245>
_______________________________________

From report at bugs.python.org  Fri Jan 16 03:59:53 2015
From: report at bugs.python.org (Gregory Szorc)
Date: Fri, 16 Jan 2015 02:59:53 +0000
Subject: [New-bugs-announce] [issue23246] distutils fails to locate
	vcvarsall with Visual C++	Compiler for Python
Message-ID: <1421377193.75.0.720322416964.issue23246@psf.upfronthosting.co.za>


New submission from Gregory Szorc:

distutils as of Python 2.7.9 is unable to locate vcvarsall.bat if Visual C++ Compiler for Python is the only Visual Studio "distribution" installed.

STR:

1) Do a fresh install of Windows + all updates
2) Install Microsoft Visual C++ Compiler for Python from http://www.microsoft.com/en-us/download/details.aspx?id=44266
3) Enter a Visual C++ 2008 command prompt via start menu 
4) Attempt to run any |python.exe setup.py| that contains C extensions
5) "Unable to find vcvarsall.bat"

Examining the behavior of MSVC for Python's bat scripts and filesystem layout, it is different enough from "full" "distributions" that it confused distutils.

First, MSVC for Python doesn't appear to set any meaningful registry entries. So, the registry checking in msvc9compiler fails to find anything.

Second, the command environment for MSVC for Python doesn't export VS90COMNTOOLS, so msvc9compiler has no clue where to go looking for files.

Third, even if VS90COMNTOOLS is set, msvc9compiler isn't able to find vcvarsall.bat because it is installed in %installdir%/vcvarsall.bat and not %installdir%/VC/vcvarsall.bat, unlike every other Visual Studio AFAICT.

Another concern is that distutils.msvc9compiler.find_vcvarsall() first attempts to read from the registry, not the environment. If you are in a MSVC for Python command shell and you also have Visual Studio 2008 installed, distutils will use vcvarsall.bat from the full Visual Studio installation instead of the Python one. I think this is wrong.

The MSVC for Python command prompt does have an environment variable that can be used: VCINSTALLDIR. It is set to %installdir%\VC\, which is equivalent to ~/AppData/Local/Programs/Common/Microsoft/Visual C++ for Python/9.0/VC/ if you launch the installer with default options. distutils could be patched to find vcvarsall.bat in %VCINSTALLDIR%\..\vcvarsall.bat

Fortunately, a workaround is available:

1) Enter MSVC for Python command prompt
2) SET DISTUTILS_USE_SDK=1
3) SET MSSdk=1
4) python.exe setup.py ...

----------
components: Distutils
messages: 234110
nosy: Gregory.Szorc, dstufft, eric.araujo
priority: normal
severity: normal
status: open
title: distutils fails to locate vcvarsall with Visual C++ Compiler for Python
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23246>
_______________________________________

From report at bugs.python.org  Fri Jan 16 07:17:50 2015
From: report at bugs.python.org (Martin Panter)
Date: Fri, 16 Jan 2015 06:17:50 +0000
Subject: [New-bugs-announce] [issue23247] Multibyte codec
	StreamWriter.reset() crashes
Message-ID: <1421389070.93.0.922548564015.issue23247@psf.upfronthosting.co.za>


New submission from Martin Panter:

$ python3 -c 'import codecs; from io import BytesIO; codecs.getwriter("big5")(BytesIO()).reset()'
Segmentation fault (core dumped)
[Exit 139]

Happens for all the multibyte codecs:
broken_stream_codecs = {
    "big5", "big5hkscs", "cp932", "cp949", "cp950",
    "euc_jp", "euc_jis_2004", "euc_jisx0213", "euc_kr",
    "gb2312", "gbk", "gb18030", "hz",
    "iso2022_jp", "iso2022_jp_1", "iso2022_jp_2", "iso2022_jp_2004",
    "iso2022_jp_3", "iso2022_jp_ext", "iso2022_kr",
    "johab", "shift_jis", "shift_jis_2004", "shift_jisx0213",
}

These codecs also share the property that their StreamReader.read() methods do not accept the second ?chars? parameter:

>>> codecs.getreader("big5")(BytesIO()).read(1, 1)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: read expected at most 1 arguments, got 2

----------
components: Unicode
messages: 234112
nosy: ezio.melotti, haypo, vadmium
priority: normal
severity: normal
status: open
title: Multibyte codec StreamWriter.reset() crashes
type: crash
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23247>
_______________________________________

From report at bugs.python.org  Fri Jan 16 09:17:01 2015
From: report at bugs.python.org (Antoine Pitrou)
Date: Fri, 16 Jan 2015 08:17:01 +0000
Subject: [New-bugs-announce] [issue23248] Update ssl data
Message-ID: <1421396221.93.0.559881277875.issue23248@psf.upfronthosting.co.za>


New submission from Antoine Pitrou:

Amusingly OpenSSL has removed some error codes, so perhaps we should only apply this to the default branch.

----------
components: Library (Lib)
files: update_ssl_data.patch
keywords: patch
messages: 234114
nosy: alex, christian.heimes, dstufft, giampaolo.rodola, janssen, pitrou
priority: normal
severity: normal
stage: patch review
status: open
title: Update ssl data
type: behavior
versions: Python 3.5
Added file: http://bugs.python.org/file37722/update_ssl_data.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23248>
_______________________________________

From report at bugs.python.org  Fri Jan 16 10:41:23 2015
From: report at bugs.python.org (Robert Kuska)
Date: Fri, 16 Jan 2015 09:41:23 +0000
Subject: [New-bugs-announce] [issue23249] test_win32 fails on aarch64
Message-ID: <1421401283.38.0.406623425658.issue23249@psf.upfronthosting.co.za>


New submission from Robert Kuska:

Original bug report at https://bugzilla.redhat.com/show_bug.cgi?id=1174037
Additional informations at Issue #20160

Note that this was reproduced not only with separate libffi package but also with libffi bundled in python.
Reproduced with Python 2.7.9 but same issue should exists also in 3.4.x.

Richard Henderson provided fix in original bug report at bugzilla (attached).

Summary:
======================================================================
FAIL: test_struct_by_value (ctypes.test.test_win32.Structures)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builddir/build/BUILD/Python-2.7.9/Lib/ctypes/test/test_win32.py", line 112, in test_struct_by_value
    self.assertEqual(ret.left, left.value)
AssertionError: -200 != 10
----------------------------------------------------------------------

(gdb) b ReturnRect
Function "ReturnRect" not defined.
Make breakpoint pending on future shared library load? (y or [n]) y
Breakpoint 1 (ReturnRect) pending.
(gdb) run test_win32.py
Starting program: /usr/bin/python test_win32.py
Missing separate debuginfos, use: debuginfo-install glibc-2.20.90-12.fc22.aarch64
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".

Breakpoint 1, ReturnRect (i=0, ar=..., br=0x59b750, cp=..., dr=..., er=0x59b750, fp=..., 
    gr=<error reading variable: Cannot access memory at address 0xffff000000000000>)
    at /usr/src/debug/Python-2.7.9/Modules/_ctypes/_ctypes_test.c:552
552	    if (ar.left + br->left + dr.left + er->left + gr.left != left * 5)
(gdb) p fp
$1 = {x = 4396722194992, y = 5879632}
(gdb) p cp
$2 = {x = 15, y = 25}
(gdb)

Consider to apply a patch or update bundled libffi.

----------
components: ctypes
files: libffi-henderson
messages: 234119
nosy: rkuska
priority: normal
severity: normal
status: open
title: test_win32 fails on aarch64
type: crash
versions: Python 2.7, Python 3.4
Added file: http://bugs.python.org/file37723/libffi-henderson

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23249>
_______________________________________

From report at bugs.python.org  Fri Jan 16 15:25:00 2015
From: report at bugs.python.org (Jon Dufresne)
Date: Fri, 16 Jan 2015 14:25:00 +0000
Subject: [New-bugs-announce] [issue23250] http.cookies HttpOnly attribute
	does not use suggested	case-style of HTTP standard
Message-ID: <1421418300.49.0.235293790837.issue23250@psf.upfronthosting.co.za>


New submission from Jon Dufresne:

See http://tools.ietf.org/html/rfc6265#section-5.2.6

Relevant section:

---

5.2.6. The HttpOnly Attribute

If the attribute-name case-insensitively matches the string HttpOnly", the user agent MUST append an attribute to the cookie-attribute-list with an attribute-name of HttpOnly and an empty attribute-value.

...

If the cookie-attribute-list contains an attribute with an attribute-name of "HttpOnly", set the cookie's http-only-flag to true. Otherwise, set the cookie's http-only-flag to false.

---

http.cookies creates this attribute as `httponly` not `HttpOnly`.

It is true, when interpreted by the user agent, this attribute is case insensitive, but it seems odd that Python would go out of its way to purposely use a different case then stated in the standard. When looking at other web technologies, the case used in the standard is most typical. The examples in the standard also use the `HttpOnly` style.

(Same applies to the Secure flag.)

----------
components: Library (Lib)
messages: 234132
nosy: jdufresne
priority: normal
severity: normal
status: open
title: http.cookies HttpOnly attribute does not use suggested case-style of HTTP standard
type: behavior
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23250>
_______________________________________

From report at bugs.python.org  Fri Jan 16 18:01:06 2015
From: report at bugs.python.org (Akira Li)
Date: Fri, 16 Jan 2015 17:01:06 +0000
Subject: [New-bugs-announce] [issue23251] mention in time.sleep() docs that
	it does not block	other Python threads
Message-ID: <1421427666.28.0.597926970646.issue23251@psf.upfronthosting.co.za>


New submission from Akira Li:

There is the corresponding StackOverflow question with 60K view 
"time.sleep ? sleeps thread or process?" [1]

The documentation patch is attached.

[1] http://stackoverflow.com/questions/92928/time-sleep-sleeps-thread-or-process

----------
assignee: docs at python
components: Documentation
files: docs-time.sleep-other-threads-are-not-blocked.diff
keywords: patch
messages: 234135
nosy: akira, docs at python
priority: normal
severity: normal
status: open
title: mention in time.sleep() docs that it does not block other Python threads
type: enhancement
versions: Python 3.5
Added file: http://bugs.python.org/file37730/docs-time.sleep-other-threads-are-not-blocked.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23251>
_______________________________________

From report at bugs.python.org  Fri Jan 16 20:55:48 2015
From: report at bugs.python.org (Serhiy Storchaka)
Date: Fri, 16 Jan 2015 19:55:48 +0000
Subject: [New-bugs-announce] [issue23252] Add support of writing to
	unseekable file in zipfile
Message-ID: <1421438148.04.0.307260333821.issue23252@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka:

ZIP files can be created to transfer it via unseekable streams (pipes, sockets). Mercurial uses a workaround to write ZIP files right to wsgirequest, but this is possible only with writestr(). write() needs seek() to updated file size, compressed sized and CRC. However ZIP file format supports streamed data without writing sizes and CRC before file data. It is possible and desirable to add full support of unseekable output files in zipfile.

----------
assignee: serhiy.storchaka
components: Library (Lib)
messages: 234145
nosy: serhiy.storchaka
priority: normal
severity: normal
stage: needs patch
status: open
title: Add support of writing to unseekable file in zipfile
type: enhancement
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23252>
_______________________________________

From report at bugs.python.org  Sat Jan 17 00:11:50 2015
From: report at bugs.python.org (Steve Dower)
Date: Fri, 16 Jan 2015 23:11:50 +0000
Subject: [New-bugs-announce] [issue23253] Delay-load ShellExecute[AW] in
	os.startfile
Message-ID: <1421449910.62.0.491687570058.issue23253@psf.upfronthosting.co.za>


New submission from Steve Dower:

Currently, pythonXY.dll has a dependency on shell32.dll solely for the os.startfile (Modules/posixmodule.c) function. This is quite a heavy dependency that many would rather not have to load (e.g. lightweight server configurations).

It would be nice to delay load the DLL and fail the operation if it is not available.

(This is as much a reminder for myself as anything else, but if someone wants to do it then feel free.)

----------
components: Windows
messages: 234151
nosy: steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Delay-load ShellExecute[AW] in os.startfile
type: enhancement
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23253>
_______________________________________

From report at bugs.python.org  Sat Jan 17 05:32:55 2015
From: report at bugs.python.org (Martin Panter)
Date: Sat, 17 Jan 2015 04:32:55 +0000
Subject: [New-bugs-announce] [issue23254] Document how to close the
	TCPServer listening socket
Message-ID: <1421469175.0.0.628748486385.issue23254@psf.upfronthosting.co.za>


New submission from Martin Panter:

Running the example from the Asynchronous Mixins section of the ?socketserver? documentation generates a ResourceWarning:

$ ./python -btWall ThreadedTCPServer.py
Server loop running in thread: Thread-1
Received: Thread-2: Hello World 1
Received: Thread-3: Hello World 2
Received: Thread-4: Hello World 3
sys:1: ResourceWarning: unclosed <socket.socket fd=3, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('127.0.0.1', 43804)>

There is a server_close() method mentioned in the doc string of the BaseServer class, so I assume it is meant to be part of the API. But there is no mention of it in the reference documentation.

I think server.server_close() should be documented, and called after server.shutdown() in the example. A further enhancement might be to turn BaseServer into a context manager, but I would be happy with using the existing server_close() method.

----------
assignee: docs at python
components: Documentation
messages: 234161
nosy: docs at python, vadmium
priority: normal
severity: normal
status: open
title: Document how to close the TCPServer listening socket
type: enhancement

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23254>
_______________________________________

From report at bugs.python.org  Sat Jan 17 14:51:58 2015
From: report at bugs.python.org (Ent)
Date: Sat, 17 Jan 2015 13:51:58 +0000
Subject: [New-bugs-announce] [issue23255] SimpleHTTPRequestHandler refactor
	for more extensible	usage.
Message-ID: <1421502718.52.0.546532418197.issue23255@psf.upfronthosting.co.za>


New submission from Ent:

Use of http.server.BaseHTTPRequestHandler for exploratory or simple usage, SimpleHTTPRequestHandler provides a good platform to start on. However, the current code in SimpleHTTPRequestHandler's send_head is tightly coupled together as a single unit.

This patch aims to break send_head down into composable parts so that any developer can subclass SimpleHTTPRequestHandler to create one's own HTTPRequestHandler class with their own custom behaviour without having to rewrite a lot of repeated code.

For example consider a developer wanting to address two usecases
* Use SimpleHTTPRequestHandler, but use index.html from a different directory.
* For certain GET urls, call on a specific function to response.
* Use custom html page instead of index.html

class CustomHTTPRequestHandler(SimpleHTTPRequestHandler):
    def do_GET(self):
        if self.path =='/':
            f = self.home_head()
        elif self.path in self.custom_paths:
            f = self.special_head()
        else:
            f = self.send_head()
        # ...
        # Standard Code

As a result of the patch, in above scenario instead of copy-pasting logic for browser redirection, getting object for file or directory and, applying headers upon success; Developer can use methods redirect_browser, get_dir_or_html_dir_path and, apply_headers to reduce the code.

Basically, applying DRY principle.

Note: Since this is but refactoring of existing code without any new functionality being added, no test cases are provided.

----------
components: Library (Lib)
files: simplehttp.patch
keywords: patch
messages: 234165
nosy: last-ent
priority: normal
severity: normal
status: open
title: SimpleHTTPRequestHandler refactor for more extensible usage.
type: enhancement
versions: Python 3.4
Added file: http://bugs.python.org/file37736/simplehttp.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23255>
_______________________________________

From report at bugs.python.org  Sat Jan 17 15:34:14 2015
From: report at bugs.python.org (Claudiu Popa)
Date: Sat, 17 Jan 2015 14:34:14 +0000
Subject: [New-bugs-announce] [issue23256] test_ctypes crashes on Windows on
	debug builds
Message-ID: <1421505254.13.0.888402709711.issue23256@psf.upfronthosting.co.za>


New submission from Claudiu Popa:

Hi,

Doing the following on Windows leads to a nasty crash "python_d -m test test_ctypes":


[1/1] test_ctypes
Fatal Python error: Segmentation fault

Current thread 0x000016fc (most recent call first):
  File "E:\Projects\repos\cpython\lib\unittest\case.py", line 162 in handle
  File "E:\Projects\repos\cpython\lib\unittest\case.py", line 704 in assertRaises
  File "E:\Projects\repos\cpython\lib\ctypes\test\test_win32.py", line 47 in test_SEH
  File "E:\Projects\repos\cpython\lib\unittest\case.py", line 577 in run
  File "E:\Projects\repos\cpython\lib\unittest\case.py", line 625 in __call__
  File "E:\Projects\repos\cpython\lib\unittest\suite.py", line 125 in run
  File "E:\Projects\repos\cpython\lib\unittest\suite.py", line 87 in __call__
  File "E:\Projects\repos\cpython\lib\unittest\suite.py", line 125 in run
  File "E:\Projects\repos\cpython\lib\unittest\suite.py", line 87 in __call__
  File "E:\Projects\repos\cpython\lib\unittest\suite.py", line 125 in run
  File "E:\Projects\repos\cpython\lib\unittest\suite.py", line 87 in __call__
  File "E:\Projects\repos\cpython\lib\unittest\suite.py", line 125 in run
  File "E:\Projects\repos\cpython\lib\unittest\suite.py", line 87 in __call__
  File "E:\Projects\repos\cpython\lib\unittest\suite.py", line 125 in run
  File "E:\Projects\repos\cpython\lib\unittest\suite.py", line 87 in __call__
  File "E:\Projects\repos\cpython\lib\test\support\__init__.py", line 1669 in run
  File "E:\Projects\repos\cpython\lib\test\support\__init__.py", line 1770 in _run_suite
  File "E:\Projects\repos\cpython\lib\test\support\__init__.py", line 1804 in run_unittest
  File "E:\Projects\repos\cpython\lib\test\regrtest.py", line 1283 in test_runner
  File "E:\Projects\repos\cpython\lib\test\regrtest.py", line 1284 in runtest_inner
  File "E:\Projects\repos\cpython\lib\test\regrtest.py", line 978 in runtest
  File "E:\Projects\repos\cpython\lib\test\regrtest.py", line 763 in main
  File "E:\Projects\repos\cpython\lib\test\regrtest.py", line 1568 in main_in_temp_cwd
  File "E:\Projects\repos\cpython\lib\test\__main__.py", line 3 in <module>
  File "E:\Projects\repos\cpython\lib\runpy.py", line 85 in _run_code
  File "E:\Projects\repos\cpython\lib\runpy.py", line 170 in _run_module_as_main


It seems that test_SEH is not properly skipped, since on my machine, sys.executable is in fact cpython\PCbuild\win32\python_d.EXE. Not sure from where the .EXE part comes, but the following patch makes the test pass.



diff -r d3671e6ba106 Lib/ctypes/test/test_win32.py
--- a/Lib/ctypes/test/test_win32.py	Fri Jan 16 20:46:37 2015 -0500
+++ b/Lib/ctypes/test/test_win32.py	Sat Jan 17 16:32:23 2015 +0200
@@ -38,7 +38,7 @@
 @unittest.skipUnless(sys.platform == "win32", 'Windows-specific test')
 class FunctionCallTestCase(unittest.TestCase):
     @unittest.skipUnless('MSC' in sys.version, "SEH only supported by MSC")
-    @unittest.skipIf(sys.executable.endswith('_d.exe'),
+    @unittest.skipIf(sys.executable.lower().endswith('_d.exe'),
                      "SEH not enabled in debug builds")
     def test_SEH(self):
         # Call functions with invalid arguments, and make sure

----------
components: ctypes
messages: 234168
nosy: Claudiu.Popa, zach.ware
priority: low
severity: normal
stage: patch review
status: open
title: test_ctypes crashes on Windows on debug builds
type: crash
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23256>
_______________________________________

From report at bugs.python.org  Sat Jan 17 18:13:52 2015
From: report at bugs.python.org (Steve Dower)
Date: Sat, 17 Jan 2015 17:13:52 +0000
Subject: [New-bugs-announce] [issue23257] Update Windows build/setup
	instructions in devguide
Message-ID: <1421514832.21.0.130284458039.issue23257@psf.upfronthosting.co.za>


New submission from Steve Dower:

This seems like the best (only?) way to get a devguide patch reviewed... I don't have any concerns, but other people may have some more suggestions for this section that I'm happy to add.

----------
assignee: steve.dower
components: Devguide, Windows
files: winbuild.patch
keywords: patch
messages: 234177
nosy: eric.araujo, ezio.melotti, ncoghlan, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
stage: patch review
status: open
title: Update Windows build/setup instructions in devguide
versions: Python 3.5
Added file: http://bugs.python.org/file37740/winbuild.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23257>
_______________________________________

From report at bugs.python.org  Sat Jan 17 20:26:53 2015
From: report at bugs.python.org (steve)
Date: Sat, 17 Jan 2015 19:26:53 +0000
Subject: [New-bugs-announce] [issue23258] Cannot Install Python 3.4.2 on
	Windows 7 64 bit / screen	print attached
Message-ID: <1421522813.3.0.624329571132.issue23258@psf.upfronthosting.co.za>


New submission from steve:

I down loaded and tried to install version 3.4.2 on a Windows 7 64 bit system.

2 error messages came up saying that I had to stop two Windows systems tasks to allow the install to complete.  Please see the attached screen print for details. 

What can I do to resolve this install problem ???

----------
components: Windows
files: Python_install-error-mgg.png
messages: 234193
nosy: 20scanman, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Cannot Install Python 3.4.2  on Windows 7 64 bit / screen print attached
versions: Python 3.4
Added file: http://bugs.python.org/file37747/Python_install-error-mgg.png

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23258>
_______________________________________

From report at bugs.python.org  Sat Jan 17 21:46:06 2015
From: report at bugs.python.org (Raymond Hettinger)
Date: Sat, 17 Jan 2015 20:46:06 +0000
Subject: [New-bugs-announce] [issue23259] Remove dummy reuse optimization
	from sets
Message-ID: <1421527566.61.0.766748710579.issue23259@psf.upfronthosting.co.za>


New submission from Raymond Hettinger:

The lookkey routines in Object/setobject.c have logic to track the first open "freeslot" in a search.  

The benefit is that new keys can reuse previously deleted slots.  The benefit only occurs in cases where keys are added, then some removed, and then more added with no intervening resizes (which clear all dummy entries) and only when the newly added keys happen to land on previously deleted entries.

The cost of the optimization is the extra logic in the inner search loop, an extra freeslot stackframe field that needs to be saved and restored, and extra logic on the loop exit.  It also causes dummies to "leak" out of the lookkey routines so that the code in contains(), discard(), and insert() all have to check for the dummy case.

This patch removes the freeslot tracking.  It saves one field on the stackframe, simplifies the lookkey inner-loop logic, simplifies the lookkey "found_null" logic, it confines knowledge of dummies to just the lookkey functions, and it simplifies the code in the insert(), contains(), and discard() functions.

In short, it looks like the freeslot idea was a net negative -- it optimized an uncommon case at the cost of slowing and complicating the common cases.

----------
assignee: rhettinger
components: Interpreter Core
files: late8.diff
keywords: patch
messages: 234198
nosy: rhettinger
priority: low
severity: normal
stage: patch review
status: open
title: Remove dummy reuse optimization from sets
type: performance
versions: Python 3.5
Added file: http://bugs.python.org/file37750/late8.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23259>
_______________________________________

From report at bugs.python.org  Sun Jan 18 00:12:41 2015
From: report at bugs.python.org (Steve Dower)
Date: Sat, 17 Jan 2015 23:12:41 +0000
Subject: [New-bugs-announce] [issue23260] Update Windows installer
Message-ID: <1421536361.51.0.300522765173.issue23260@psf.upfronthosting.co.za>


New submission from Steve Dower:

Updating the installer for better security and robustness. Large patch coming soon (just getting an issue number to put in NEWS).

----------
assignee: steve.dower
components: Windows
messages: 234203
nosy: steve.dower, tim.golden, zach.ware
priority: release blocker
severity: normal
status: open
title: Update Windows installer
type: enhancement
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23260>
_______________________________________

From report at bugs.python.org  Sun Jan 18 04:08:31 2015
From: report at bugs.python.org (Raymond Hettinger)
Date: Sun, 18 Jan 2015 03:08:31 +0000
Subject: [New-bugs-announce] [issue23261] Clean-up set.pop() search finger
	logic
Message-ID: <1421550511.96.0.63875506681.issue23261@psf.upfronthosting.co.za>


New submission from Raymond Hettinger:

The existing search finger is stored in a hackish way (using the hash field of entry zero in the hash table).  Replace this with normal coding techniques (saving the field in the set object).

Cost one extra field in the set object.  Benefit, remove an arcane hack and simplify the set.pop() code just a little bit.

----------
assignee: rhettinger
components: Interpreter Core
files: finger.diff
keywords: patch
messages: 234216
nosy: rhettinger
priority: low
severity: normal
status: open
title: Clean-up set.pop() search finger logic
versions: Python 3.5
Added file: http://bugs.python.org/file37753/finger.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23261>
_______________________________________

From report at bugs.python.org  Sun Jan 18 05:16:56 2015
From: report at bugs.python.org (Stephan Sokolow)
Date: Sun, 18 Jan 2015 04:16:56 +0000
Subject: [New-bugs-announce] [issue23262] webbrowser module broken with
	Firefox 36+
Message-ID: <1421554616.48.0.911036047072.issue23262@psf.upfronthosting.co.za>


New submission from Stephan Sokolow:

As of Firefox 36 (currently in beta channel), the -remote option has been removed.

https://www.mozilla.org/en-US/firefox/36.0a2/auroranotes/
https://www.mozilla.org/en-US/firefox/36.0beta/releasenotes/

As such, attempting to open http://www.example.com/ using webbrowser.open() or webbrowser.open_new_tab() results in "File not Found" tabs pointing to these two URLs, respectively:

file:///home/ssokolow/openURL%28http://www.example.com/%29
file:///home/ssokolow/openURL%28http://www.example.com/,new-tab%29

As I happen to have the Dead Snakes PPA set up on Lubuntu 14.04 for use with tox, I was able to confirm this as an issue in Python 2.7, 3.1, 3.2, 3.3, and 3.4.

----------
components: Library (Lib)
messages: 234218
nosy: ssokolow
priority: normal
severity: normal
status: open
title: webbrowser module broken with Firefox 36+
type: behavior
versions: Python 2.7, Python 3.2, Python 3.3, Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23262>
_______________________________________

From report at bugs.python.org  Sun Jan 18 06:44:41 2015
From: report at bugs.python.org (Matt Bachmann)
Date: Sun, 18 Jan 2015 05:44:41 +0000
Subject: [New-bugs-announce] [issue23263] Python 3 gives misleading errors
	when validating unicode	identifiers
Message-ID: <1421559881.85.0.938769960217.issue23263@psf.upfronthosting.co.za>


New submission from Matt Bachmann:

PEP 3131 changed the definition of valid identifiers to match this pattern

<XID_Start> <XID_Continue>* .

Currently if you have an invalid character in an identifier you get this error

? = 4
SyntaxError: invalid character in identifier


This is fine in most cases. But in some cases the problem is not the character is invalid so much as the character may not be used to START the identifier. One example of this is the "combining grave accent" which is an XID_CONTINUE character but not an XID_START

So ?e is an invalid identifier but e? is a valid identifier. So the ? character is not invalid in all cases.

The attached patch attempts to clarify this by providing a different error when the start character is invalid.

>>> ?e = 4
  File "<stdin>", line 1
    ?e = 4
     ^
SyntaxError: invalid start character in identifier

However, if the character is simply not allowed (as it is neither an XID_START or an XID_CONTINUE character) the original error is used.
>>> ?smile = 4
  File "<stdin>", line 1
    ?smile = 4
         ^
SyntaxError: invalid character in identifier

----------
components: Unicode
files: clarify_unicode_identifier_errors.patch
keywords: patch
messages: 234222
nosy: Matt.Bachmann, ezio.melotti, haypo
priority: normal
severity: normal
status: open
title: Python 3 gives misleading errors when validating unicode identifiers
type: enhancement
versions: Python 3.2, Python 3.3, Python 3.4, Python 3.5, Python 3.6
Added file: http://bugs.python.org/file37755/clarify_unicode_identifier_errors.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23263>
_______________________________________

From report at bugs.python.org  Sun Jan 18 09:11:56 2015
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sun, 18 Jan 2015 08:11:56 +0000
Subject: [New-bugs-announce] [issue23264] Add pickle support of dict views
Message-ID: <1421568716.9.0.947546089999.issue23264@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka:

For now dict views are not pickleable in Python 3 (unlike to Python 2 and unlike to dict view iterators). Proposed patch adds pickle support of dict views.

----------
components: Interpreter Core
files: pickle_dictviews.patch
keywords: patch
messages: 234225
nosy: alexandre.vassalotti, pitrou, rhettinger, serhiy.storchaka
priority: normal
severity: normal
stage: patch review
status: open
title: Add pickle support of dict views
type: enhancement
versions: Python 3.5
Added file: http://bugs.python.org/file37758/pickle_dictviews.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23264>
_______________________________________

From report at bugs.python.org  Sun Jan 18 11:17:04 2015
From: report at bugs.python.org (i01100101)
Date: Sun, 18 Jan 2015 10:17:04 +0000
Subject: [New-bugs-announce] [issue23265] tar.xz support for sdist
Message-ID: <1421576224.46.0.598158761162.issue23265@psf.upfronthosting.co.za>


New submission from i01100101:

The Python standard library includes lzma and support for *tar.xz archives since Python 3.3 but the distutils' 'sdist' command cannot output source distributions as such tarball.
I suggest the changed version of distutils.archive_util module attached to this issue in order to tell distutils that tarfile now supports tar.xz
(The changes are to the defenition of 'ARCHIVE_FORMATS' and 'make_tarball'

----------
components: Distutils
files: archive_util.py
messages: 234237
nosy: dstufft, eric.araujo, i01100101
priority: normal
severity: normal
status: open
title: tar.xz support for sdist
type: enhancement
versions: Python 3.4
Added file: http://bugs.python.org/file37761/archive_util.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23265>
_______________________________________

From report at bugs.python.org  Sun Jan 18 13:32:34 2015
From: report at bugs.python.org (Markus)
Date: Sun, 18 Jan 2015 12:32:34 +0000
Subject: [New-bugs-announce] [issue23266] Faster implementation to collapse
	non-consecutive	ip-addresses
Message-ID: <1421584354.17.0.398119389429.issue23266@psf.upfronthosting.co.za>


New submission from Markus:

I found the code used to collapse addresses to be very slow on a large number (64k) of island addresses which are not collapseable.

The code at
https://github.com/python/cpython/blob/0f164ccc85ff055a32d11ad00017eff768a79625/Lib/ipaddress.py#L349
was found to be guilty, especially the index lookup.
The patch changes the code to discard the index lookup and have _find_address_range return the number of items consumed.
That way the set operation to dedup the addresses can be dropped as well.

Numbers from the testrig I adapted from http://bugs.python.org/issue20826 with 8k non-consecutive addresses:

Execution time: 0.6893927365541458 seconds
vs.
Execution time: 12.116527611762285 seconds


MfG
Markus K?tter

----------
components: Library (Lib)
files: ipaddress_collapse_non_consecutive_performance.diff
keywords: patch
messages: 234239
nosy: cmn
priority: normal
severity: normal
status: open
title: Faster implementation to collapse non-consecutive ip-addresses
type: performance
versions: Python 3.5
Added file: http://bugs.python.org/file37762/ipaddress_collapse_non_consecutive_performance.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23266>
_______________________________________

From report at bugs.python.org  Sun Jan 18 14:33:39 2015
From: report at bugs.python.org (shani)
Date: Sun, 18 Jan 2015 13:33:39 +0000
Subject: [New-bugs-announce] [issue23267] multiprocessing pool.py doesn't
	close inqueue and	outqueue pipes on termination
Message-ID: <1421588019.18.0.707676170756.issue23267@psf.upfronthosting.co.za>


New submission from shani:

Multiprocessing pool.py gets SimpleQueue objects as inqueue and outqueue.
when it terminates, it doesn't call the close() method of the queues' readers and writers.
As a results, 4 file pipes leak in one pool termination.

Expected: The pool closes reader and writer pipes of the inqueue and outqueue when it terminates.
What did happen: the pool doesn't close the pipes. 4 pipes leak.

----------
components: Library (Lib)
messages: 234244
nosy: shanip
priority: normal
severity: normal
status: open
title: multiprocessing pool.py doesn't close inqueue and outqueue pipes on termination
type: resource usage

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23267>
_______________________________________

From report at bugs.python.org  Sun Jan 18 21:02:38 2015
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sun, 18 Jan 2015 20:02:38 +0000
Subject: [New-bugs-announce] [issue23268] Fix comparison of ipaddress classes
Message-ID: <1421611358.75.0.717594852846.issue23268@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka:

Here is a patch which fixes following errors in comparison of ipaddress classes.

1. Ordering comparison raised AttributeError when compared with foreign classes.

2. Ordering comparison didn't return NotImplemented when compared with foreign classes. This prevented fallback to foreign class comparison method.

3. There was a bug in _TotalOrderingMixin.__le__(). It could return False instead of NotImplemented if compared network and address of different versions.

4. There was a typo in ComparisonTests.test_incompatible_versions().

As far as functools.total_ordering now is fixed and more correct and tested than _TotalOrderingMixin, _TotalOrderingMixin is dropped away.

----------
components: Library (Lib)
files: ipaddress_comparison.patch
keywords: patch
messages: 234267
nosy: ncoghlan, pmoody, serhiy.storchaka
priority: normal
severity: normal
stage: patch review
status: open
title: Fix comparison of ipaddress classes
type: behavior
versions: Python 3.4, Python 3.5
Added file: http://bugs.python.org/file37767/ipaddress_comparison.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23268>
_______________________________________

From report at bugs.python.org  Mon Jan 19 10:15:42 2015
From: report at bugs.python.org (Raymond Hettinger)
Date: Mon, 19 Jan 2015 09:15:42 +0000
Subject: [New-bugs-announce] [issue23269] Tighten-up search loops in sets
Message-ID: <1421658942.27.0.429398283891.issue23269@psf.upfronthosting.co.za>


New submission from Raymond Hettinger:

First draft of patch to switch from a table[(i+j)&mask] style of entry calculation to an entry++ style.  The entry computation simplifies from add/shl4/and/lea to a single add16.  To do this, the linear probes are limited to the length of table rather than wrapping-around.

I haven't timed this one yet (have just looked at the assembly) or worked through the implications.  The new approach is a little less attractive but may be easier to reason about than the mask wrap-around.  Also, it disadvantages small sets where the wrap-around property assured that the linear probes would always fine a hit without any entry being checked more than once.  There is a extra setup time before the linear probes to compute the limit.  Also, since the loop size is now variable instead of fixed, the inner loop for set_insert_clean() no longer gets unrolled by either GCC or CLANG.

With the change, the inner loop of set_insert_clean() is very tight:

L436:
        addq    $16, %rax       #, entry
entry < limit    
        cmpq    %rdx, %rax      # limit, entry
        ja      L399    #,
entry->key == NULL
        cmpq    $0, (%rax)      #,* entry
        jne     L436 

The code for lookkey() is also tight (though I can't tell why the entry->key gets loaded from memory twice):

L102:
        cmpq    %r15, %r13      # tmp170, D.12706      entry->key == dummy
        jne     L103    #,
        testq   %r12, %r12      # freeslot
        cmove   %r14, %r12      # entry -> freeslot
L103:
        addq    $16, %r14       #, entry            entry++ 
        cmpq    %r14, %rbx      # entry, limit
        jb      L146    #,
        movq    (%r14), %r13    # MEM[base: entry_65, offset: 0B], D.12706
        testq   %r13, %r13      # D.12706           entry->key == NULL
        je      L147    #,
        cmpq    %rbp, 8(%r14)   # hash, MEM[base: entry_104, offset: 8B]
        je      L148    #,                          entry->hash == hash
?       movq    (%r14), %r13    # MEM[base: entry_104, offset: 0B], D.12706
        jmp     L102    #

----------
assignee: rhettinger
components: Interpreter Core
files: limit.diff
keywords: patch
messages: 234308
nosy: rhettinger
priority: low
severity: normal
stage: patch review
status: open
title: Tighten-up search loops in sets
type: performance
versions: Python 3.5
Added file: http://bugs.python.org/file37773/limit.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23269>
_______________________________________

From report at bugs.python.org  Mon Jan 19 10:50:10 2015
From: report at bugs.python.org (STINNER Victor)
Date: Mon, 19 Jan 2015 09:50:10 +0000
Subject: [New-bugs-announce] [issue23270] Use the new
	__builtin_mul_overflow() of Clang and GCC 5	to check for
	integer overflow
Message-ID: <1421661010.06.0.656712864154.issue23270@psf.upfronthosting.co.za>


New submission from STINNER Victor:

In CPython, almost all memory allocations are protected against integer overflow with code looking like that:

    if (length > ((PY_SSIZE_T_MAX - struct_size) / char_size - 1)) {
        PyErr_NoMemory();
        return NULL;
    }
    new_size = (struct_size + (length + 1) * char_size);

For performances, GCC 5 introduces __builtin_mul_overflow() which is an integer multiplication with overflow check. On x86/x86_64, it is implemented in hardware (assembler instruction JO, jump if overflow, if I remember correctly).

The function already exists in Clang: "... which existed in Clang/LLVM for a while" says http://lwn.net/Articles/623368/ According to this mail sent to the Linux kernel mailing list, the Linux kernel has functions like "check_mul_overflow(X, Y, C)".

For other compilers, it should be easy to reimplement it, but I don't know what is the most efficient implementation (Py_LOCAL_INLINE function in an header?)

GCC 5 changelog:
https://gcc.gnu.org/gcc-5/changes.html

Note: GCC 5 is not released yet.

----------
messages: 234310
nosy: haypo
priority: normal
severity: normal
status: open
title: Use the new __builtin_mul_overflow() of Clang and GCC 5 to check for integer overflow
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23270>
_______________________________________

From report at bugs.python.org  Mon Jan 19 14:56:59 2015
From: report at bugs.python.org (Dionysis Zindros)
Date: Mon, 19 Jan 2015 13:56:59 +0000
Subject: [New-bugs-announce] [issue23271] Unicode HOWTO documentation error
Message-ID: <1421675819.3.0.538078693406.issue23271@psf.upfronthosting.co.za>


New submission from Dionysis Zindros:

In the Unicode HOTWO documentation for Python 2.x [0], there's an error in the fourth code sample under the section "The Unicode Type".

The code states:

```
>>> s = u'Was ever feather so lightly blown to and fro as this multitude?'
>>> s.count('e')
5
>>> s.find('feather')
9
>>> s.find('bird')
-1
>>> s.replace('feather', 'sand')
u'Was ever sand so lightly blown to and fro as this multitude?'
>>> s.upper()
u'WAS EVER FEATHER SO LIGHTLY BLOWN TO AND FRO AS THIS MULTITUDE?'
```

Notice that in the last line, "sand" was turned back into "feather". The correct last line should have been:

```
u'WAS EVER SAND SO LIGHTLY BLOWN TO AND FRO AS THIS MULTITUDE?'
```

[0] https://docs.python.org/2/howto/unicode.html

----------
assignee: docs at python
components: Documentation
messages: 234312
nosy: Dionysis.Zindros, docs at python
priority: normal
severity: normal
status: open
title: Unicode HOWTO documentation error
type: enhancement
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23271>
_______________________________________

From report at bugs.python.org  Mon Jan 19 16:05:49 2015
From: report at bugs.python.org (=?utf-8?b?THVrw6HFoSBOxJttZWM=?=)
Date: Mon, 19 Jan 2015 15:05:49 +0000
Subject: [New-bugs-announce] [issue23272] Python built-in comparison problem
Message-ID: <1421679949.19.0.258601138429.issue23272@psf.upfronthosting.co.za>


Changes by Luk?? N?mec <lu.nemec at gmail.com>:


----------
nosy: Luk??.N?mec
priority: normal
severity: normal
status: open
title: Python built-in comparison problem

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23272>
_______________________________________

From report at bugs.python.org  Mon Jan 19 18:55:08 2015
From: report at bugs.python.org (Robert Collins)
Date: Mon, 19 Jan 2015 17:55:08 +0000
Subject: [New-bugs-announce] [issue23273] traceback: formatting a traceback
	stats the filesystem
Message-ID: <1421690108.46.0.51510686328.issue23273@psf.upfronthosting.co.za>


New submission from Robert Collins:

Discovered in issue 17911, all the traceback calls that render a stack trace end up calling linecache.checkcache, which stats files on disk, making getting a traceback rather more expensive than folk may expect. For "oops, it crashed" situations thats fine, but when used to gather diagnostic details like tulip does, it becomes substantially more wasteful. Since we know when we've reloaded a module, there should be no need to go out to disk at any other time - particularly since if we *haven't* reloaded the module, doing so will just end up showing the wrong source.

Further, PEP 302 source code isn't refreshed when the cache is checked, which can lead to stale code in modules that *have* been reloaded.

----------
messages: 234318
nosy: rbcollins
priority: normal
severity: normal
status: open
title: traceback: formatting a traceback stats the filesystem

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23273>
_______________________________________

From report at bugs.python.org  Mon Jan 19 18:59:47 2015
From: report at bugs.python.org (Michael Schlenker)
Date: Mon, 19 Jan 2015 17:59:47 +0000
Subject: [New-bugs-announce] [issue23274] make_ssl_data.py in Python 2.7.9
	needs Python 3 to run
Message-ID: <1421690387.01.0.101283593365.issue23274@psf.upfronthosting.co.za>


New submission from Michael Schlenker:

The make_ssl_data.py script in Tools/ssl/ needs a python3 to run due to the usage of open(..., encoding='latin1').

This makes usage on a host without python3 installed more complex than needed. It should use io.open(...) to run on both python3 and python2.

----------
components: Demos and Tools
messages: 234320
nosy: schlenk
priority: normal
severity: normal
status: open
title: make_ssl_data.py in Python 2.7.9 needs Python 3 to run
type: enhancement

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23274>
_______________________________________

From report at bugs.python.org  Mon Jan 19 20:05:47 2015
From: report at bugs.python.org (Devin Jeanpierre)
Date: Mon, 19 Jan 2015 19:05:47 +0000
Subject: [New-bugs-announce] [issue23275] Can assign [] = (), but not () = []
Message-ID: <1421694347.87.0.147201508367.issue23275@psf.upfronthosting.co.za>


New submission from Devin Jeanpierre:

>>> [] = ()
>>> () = []
  File "<stdin>", line 1
SyntaxError: can't assign to ()

This contradicts the assignment grammar, which would make both illegal: https://docs.python.org/3/reference/simple_stmts.html#assignment-statements

----------
components: Interpreter Core
messages: 234324
nosy: Devin Jeanpierre
priority: normal
severity: normal
status: open
title: Can assign [] = (), but not () = []
type: behavior

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23275>
_______________________________________

From report at bugs.python.org  Mon Jan 19 23:22:19 2015
From: report at bugs.python.org (Alfred Krohmer)
Date: Mon, 19 Jan 2015 22:22:19 +0000
Subject: [New-bugs-announce] [issue23276] hackcheck is broken in association
	with __setattr__
Message-ID: <1421706139.14.0.239687578391.issue23276@psf.upfronthosting.co.za>


New submission from Alfred Krohmer:

The following code:


import traceback
import sys

from PyQt5.QtCore import Qt

class MetaA(type):
    pass

class A(metaclass=MetaA):
    pass

class MetaB(type):
    pass

class B(metaclass=MetaB):
    pass

for ClassB in B, Qt:

    print("Trying class %s" % (ClassB.__name__, ))

    class MyMeta(type(A), type(ClassB)):
        def __setattr__(cls, key, value):
            print(cls)
            super(type, cls).__setattr__(key, value)

    class MyClass(A, ClassB, metaclass=MyMeta):
        pass

    try:
        setattr(MyClass, 'abc', 123)
    except:
        traceback.print_exc(file=sys.stdout)

    try:
        type.__setattr__(MyClass, 'test', 42)
    except:
        traceback.print_exc(file=sys.stdout)


Fails with the following output:


Trying class B
<class '__main__.MyClass'>
Traceback (most recent call last):
  File "test3.py", line 31, in <module>
    setattr(MyClass, 'abc', 123)
  File "test3.py", line 25, in __setattr__
    super(type, cls).__setattr__(key, value)
TypeError: can't apply this __setattr__ to type object
Trying class Qt
<class '__main__.MyClass'>
Traceback (most recent call last):
  File "test3.py", line 31, in <module>
    setattr(MyClass, 'abc', 123)
  File "test3.py", line 25, in __setattr__
    super(type, cls).__setattr__(key, value)
TypeError: can't apply this __setattr__ to sip.wrappertype object
Traceback (most recent call last):
  File "test3.py", line 36, in <module>
    type.__setattr__(MyClass, 'test', 42)
TypeError: can't apply this __setattr__ to sip.wrappertype object


The metaclass of a class should be able to update its class' __dict__ my calling super(type, cls).__setattr__ (there is no other way known to me to do this). Furthermore, if subclassing an external class, like Qt, it should be possible to use type.__setattr__(MyClass, ...) externally to change the class' attributes.

The error is caused by the hackcheck function in objects/typeobject.c.

----------
components: Interpreter Core, Library (Lib)
messages: 234329
nosy: devkid
priority: normal
severity: normal
status: open
title: hackcheck is broken in association with __setattr__
type: behavior
versions: Python 2.7, Python 3.2, Python 3.3, Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23276>
_______________________________________

From report at bugs.python.org  Tue Jan 20 02:17:43 2015
From: report at bugs.python.org (Jon Dufresne)
Date: Tue, 20 Jan 2015 01:17:43 +0000
Subject: [New-bugs-announce] [issue23277] Cleanup unused and duplicate
	imports in tests
Message-ID: <1421716663.28.0.404138725628.issue23277@psf.upfronthosting.co.za>


New submission from Jon Dufresne:

Ran variations of the command:

$ find . -wholename '*/test/*.py' | xargs flake8 --select=F401,F811

To look for unused or duplicate imports. The attached patch removes them.

----------
components: Tests
files: cleanup-unused-imports.patch
keywords: patch
messages: 234334
nosy: jdufresne
priority: normal
severity: normal
status: open
title: Cleanup unused and duplicate imports in tests
versions: Python 3.6
Added file: http://bugs.python.org/file37780/cleanup-unused-imports.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23277>
_______________________________________

From report at bugs.python.org  Tue Jan 20 03:02:52 2015
From: report at bugs.python.org (Nelson Minar)
Date: Tue, 20 Jan 2015 02:02:52 +0000
Subject: [New-bugs-announce] [issue23278] multiprocessing maxtasksperchild=1
	+ logging = task loss
Message-ID: <1421719372.42.0.719053884187.issue23278@psf.upfronthosting.co.za>


New submission from Nelson Minar:

I have a demonstration of a problem where the combination of multiprocessing with maxtasksperchild=1 and the Python logging library causes tasks to occasionally get lost. The bug might be related to issue 22393 or issue 6721, but I'm not certain. issue 10037 and issue 9205 also might be relevant.  I've attached sample code, it can also be found at https://gist.github.com/NelsonMinar/022794b6a709ea5b7682

My program uses Pool.imap_unordered() to execute 200 tasks. Each worker task writes a log message and sleeps a short time. The master process uses a timeout on next() to log a status message occasionally.

When it works, 200 jobs are completed quickly. When it breaks, roughly 195 of 200 jobs will have completed and next() never raises StopIteration.

If everything logs to logging.getLogger() and maxtasksperchild=1, it usually breaks. It appears that sometimes jobs just get lost and don't complete. We've observed that with maxtasksperchild=1 sometimes a new worker process gets created but no work assigned to it. When that happens the task queue never runs to completion.

If we log straight to stderr or don't set maxtasksperchild, the run completes.

The bug has been observed in Python 2.7.6 and Python 3.4.0 on Ubuntu 14.04

This is a distillation of much more complex application-specific code. Discussion of the bug and original code can be found at

https://github.com/openaddresses/machine/issues/51
https://github.com/openaddresses/machine/blob/7c3d0fba8ba0915af2101ace45dfaf5519d5ad85/openaddr/jobs.py

Thank you, Nelson

----------
components: Library (Lib)
files: bug-demo.py
messages: 234336
nosy: nelson
priority: normal
severity: normal
status: open
title: multiprocessing maxtasksperchild=1 + logging = task loss
type: behavior
versions: Python 2.7, Python 3.4
Added file: http://bugs.python.org/file37781/bug-demo.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23278>
_______________________________________

From report at bugs.python.org  Tue Jan 20 05:48:28 2015
From: report at bugs.python.org (Douglas Rudd)
Date: Tue, 20 Jan 2015 04:48:28 +0000
Subject: [New-bugs-announce] [issue23279] test_site/test_startup_imports
	fails when mpl_toolkit or	logilab based modules installed
Message-ID: <1421729308.95.0.570059573722.issue23279@psf.upfronthosting.co.za>


New submission from Douglas Rudd:

pth files for logilab (e.g. logilab_common, logilab_astng) and mpl_toolkit (e.g. basemap, matplotlib) contain code like the following (taken from basemap 1.0.7):

import sys, types, os;p = os.path.join(sys._getframe(1).f_locals['sitedir'], *('mpl_toolkits',));ie = os.path.exists(os.path.join(p,'__init__.py'));m = not ie and sys.modules.setdefault('mpl_toolkits', types.ModuleType('mpl_toolkits'));mp = (m or []) and m.__dict__.setdefault('__path__',[]);(p not in mp) and mp.append(p)

This leads to the types module being loaded on import, which then causes test_site/test_startup_imports to fail since types is in collection_mods:

        collection_mods = {'_collections', 'collections', 'functools',
                           'heapq', 'itertools', 'keyword', 'operator',
                           'reprlib', 'types', 'weakref'
                          }.difference(sys.builtin_module_names)
        self.assertFalse(modules.intersection(collection_mods), stderr)

$ python3.4 -c 'import sys; print(set(sys.modules))'
{'builtins', '_codecs', 'io', 'errno', 'abc', '_weakref', '_collections_abc', '_bootlocale', 'stat', 'os', '_frozen_importlib', 'encodings.utf_8', '_sitebuiltins', '_sysconfigdata', 'sysconfig', 'posixpath', 'sys', 'mpl_toolkits', '_weakrefset', 'encodings', 'encodings.aliases', 'signal', '__main__', '_stat', 'zipimport', 'genericpath', 'site', '_io', 'posix', 'codecs', 'types', '_imp', 'os.path', '_warnings', 'marshal', '_locale', '_thread', 'encodings.latin_1'}


This is a similar bug to http://bugs.python.org/issue20986 

I don't know the purpose of this test well enough to propose a fix, but given the number of exceptions already there it doesn't seem to be a robust test.

----------
components: Tests
messages: 234345
nosy: drudd
priority: normal
severity: normal
status: open
title: test_site/test_startup_imports fails when mpl_toolkit or logilab based modules installed
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23279>
_______________________________________

From report at bugs.python.org  Tue Jan 20 07:07:31 2015
From: report at bugs.python.org (Zachary Ware)
Date: Tue, 20 Jan 2015 06:07:31 +0000
Subject: [New-bugs-announce] [issue23280] Convert binascii.{un}hexlify to
	Argument Clinic (fix	docstrings)
Message-ID: <1421734051.68.0.00270883538381.issue23280@psf.upfronthosting.co.za>


New submission from Zachary Ware:

The Argument Clinic conversion of the binascii module left hexlify and unhexlify with bad docstrings:

    hexlify(...)
        b2a_hex($module, data, /)
        --
        
        Hexadecimal representation of binary data.
        
        The return value is a bytes object.  This function is also
        available as "hexlify()".

    unhexlify(...)
        a2b_hex($module, hexstr, /)
        --
        
        Binary data of hexadecimal representation.
        
        hexstr must contain an even number of hex digits (upper or lower case).
        This function is also available as "unhexlify()".

Attached patch fixes it, removes the note that the function is also available as itself (leaving the note on {a2b,b2a}_hex), and tests that the functions are in fact aliases.

----------
components: Extension Modules
files: binascii_clinic_fix.diff
keywords: patch
messages: 234353
nosy: serhiy.storchaka, zach.ware
priority: normal
severity: normal
stage: patch review
status: open
title: Convert binascii.{un}hexlify to Argument Clinic (fix docstrings)
type: behavior
versions: Python 3.4, Python 3.5
Added file: http://bugs.python.org/file37784/binascii_clinic_fix.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23280>
_______________________________________

From report at bugs.python.org  Tue Jan 20 08:35:29 2015
From: report at bugs.python.org (=?utf-8?q?Pawe=C5=82_Zduniak?=)
Date: Tue, 20 Jan 2015 07:35:29 +0000
Subject: [New-bugs-announce] [issue23281] Access violation - pyc file
Message-ID: <1421739329.05.0.520185996533.issue23281@psf.upfronthosting.co.za>


New submission from Pawe? Zduniak:

(950.e58): Access violation - code c0000005 (first chance)
First chance exceptions are reported before any exception handling.
This exception may be expected and handled.
*** ERROR: Symbol file could not be found.  Defaulted to export symbols for C:\Windows\SysWOW64\python27.dll - 
python27!PyEval_EvalFrameEx+0x1895:
1e0bcb45 8b74b00c        mov     esi,dword ptr [eax+esi*4+0Ch] ds:002b:0224207c=????????

----------
components: Windows
files: test.pyc
messages: 234357
nosy: Pawe?.Zduniak, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Access violation - pyc file
type: crash
versions: Python 2.7
Added file: http://bugs.python.org/file37785/test.pyc

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23281>
_______________________________________

From report at bugs.python.org  Tue Jan 20 11:39:16 2015
From: report at bugs.python.org (Serhiy Storchaka)
Date: Tue, 20 Jan 2015 10:39:16 +0000
Subject: [New-bugs-announce] [issue23282] Slightly faster set lookup
Message-ID: <1421750356.2.0.166287277194.issue23282@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka:

Currently set_lookkey() first tests entry->key == NULL, then entry->hash == hash and entry->key != dummy, and only after that entry->key == key. Proposed patch optimizes the order of comparisons. entry->key == key is tested first as for dicts. And no need to test entry->key != dummy after entry->hash == hash if entry->hash of dummy key is set to -1.

Microbenchmark which demonstrates the best case (a lot of lookups of keys existing in the set).

$ ./python -m timeit -s "a = list(range(10**6)); s1 = set(a); s2 = set(a)" -- "s1 <= s2"

Unpatched: 10 loops, best of 3: 39.4 msec per loop
Patched: 10 loops, best of 3: 35.3 msec per loop

----------
components: Interpreter Core
files: set_faster_lookup.patch
keywords: patch
messages: 234367
nosy: pitrou, rhettinger, serhiy.storchaka
priority: normal
severity: normal
stage: patch review
status: open
title: Slightly faster set lookup
type: performance
versions: Python 3.5
Added file: http://bugs.python.org/file37789/set_faster_lookup.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23282>
_______________________________________

From report at bugs.python.org  Tue Jan 20 21:30:22 2015
From: report at bugs.python.org (Zachary Ware)
Date: Tue, 20 Jan 2015 20:30:22 +0000
Subject: [New-bugs-announce] [issue23283] Backport Tools/clinic to 3.4
Message-ID: <1421785822.4.0.222715524471.issue23283@psf.upfronthosting.co.za>


New submission from Zachary Ware:

Larry, in #22120 msg224817, you said:
"Since IIUC there's no code in 3.4 that uses an unsigned integer return converter, I'm not backporting the fix."

Modules/binascii.c does have one use of an unsigned integer return, resulting in the only not-something-new difference between 3.4 and 3.5's Modules/clinic dir.  Is there any reason not to do a quick "hg revert -Cr default Tools/clinic && make clinic && hg commit" on 3.4 to update binascii.c.h?

----------
messages: 234391
nosy: larry, zach.ware
priority: normal
severity: normal
status: open
title: Backport Tools/clinic to 3.4
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23283>
_______________________________________

From report at bugs.python.org  Tue Jan 20 23:51:23 2015
From: report at bugs.python.org (Poor Yorick)
Date: Tue, 20 Jan 2015 22:51:23 +0000
Subject: [New-bugs-announce] [issue23284] curses, readline, tinfo,
	and also --prefix, dbm, CPPFLAGS
Message-ID: <1421794283.67.0.365311015753.issue23284@psf.upfronthosting.co.za>


New submission from Poor Yorick:

Building Python-2.7.9 using --prefix, with an ncurses that's linked to libtinfo
and a readline that isn't linked to any termcap library, I ran into the trouble
that the curses module wan't buing build with the needed -L and -l flags for
the libtinfo shared object.  I dug into setup.py, and ended up overhauling the
readline, curses, and also dbm handling (more on that in another ticket).  I
also made changes to allow setup.py to pick up options like -isystem from
$CPPFLAGS, replacing in the process the "optparse" module with the "argparse"
module.  Since argparse requires collections.OrderedDict, which isn't yet built
at this stage, I added Lib_boot/argparse_boot.py, which uses the pure-Python
OrderedDict from

    https://pypi.python.org/pypi/ordereddict

and had setup.py use "argparse_boot" module instead.

The build also ran into trouble with system directories that setup.py
explicitly adds to inc_dirs and lib_dirs ahead of those of the alternate
prefix.

The attached files fixed all these issues in my scenario, allowing a succesful
build and install of Python-2.7.9.

----------
components: Build
hgrepos: 290
messages: 234399
nosy: pooryorick
priority: normal
severity: normal
status: open
title: curses, readline, tinfo, and also --prefix, dbm, CPPFLAGS
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23284>
_______________________________________

From report at bugs.python.org  Wed Jan 21 00:03:28 2015
From: report at bugs.python.org (=?utf-8?q?Charles-Fran=C3=A7ois_Natali?=)
Date: Tue, 20 Jan 2015 23:03:28 +0000
Subject: [New-bugs-announce] [issue23285] PEP 475 - EINTR hanndling
Message-ID: <1421795008.21.0.948909551681.issue23285@psf.upfronthosting.co.za>


Changes by Charles-Fran?ois Natali <cf.natali at gmail.com>:


----------
components: Library (Lib)
hgrepos: 293
nosy: haypo, neologix, pitrou
priority: normal
severity: normal
status: open
title: PEP 475 - EINTR hanndling
type: enhancement

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23285>
_______________________________________

From report at bugs.python.org  Wed Jan 21 00:09:22 2015
From: report at bugs.python.org (aruseni)
Date: Tue, 20 Jan 2015 23:09:22 +0000
Subject: [New-bugs-announce] [issue23286] A typo in the tutorial
Message-ID: <1421795362.32.0.373442891805.issue23286@psf.upfronthosting.co.za>


New submission from aruseni:

https://docs.python.org/3/tutorial/introduction.html

> Lists also supports operations like concatenation

----------
assignee: docs at python
components: Documentation
messages: 234401
nosy: aruseni, docs at python
priority: normal
severity: normal
status: open
title: A typo in the tutorial

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23286>
_______________________________________

From report at bugs.python.org  Wed Jan 21 01:33:02 2015
From: report at bugs.python.org (John Beck)
Date: Wed, 21 Jan 2015 00:33:02 +0000
Subject: [New-bugs-announce] [issue23287] ctypes.util.find_library
	needlessly call crle on Solaris
Message-ID: <1421800382.95.0.158394542432.issue23287@psf.upfronthosting.co.za>


New submission from John Beck:

On Solaris, in Lib/ctypes/util.py, we have code that looks for
/usr/bin/crle and calls it to parse its output to try to determine
the Default Library Path.  This code broke recently (Solaris 12 build
65), as it expects to find a line starting with
    "Default Library Path (ELF):"
but the " (ELF)" part of that line was removed because it was no longer
needed.  So we need a change here regardless.  But it turns out that
calling crle is not needed at all because the default library path is
a constant on Solaris: "/lib/64:/usr/lib/64" in 64-bit mode and
"/lib:/usr/lib" in 32-bit mode.  Thus I offer the attached patch
for both 2.7 and 3.4.

----------
components: ctypes
files: crle-fix.patch
keywords: patch
messages: 234417
nosy: jbeck
priority: normal
severity: normal
status: open
title: ctypes.util.find_library needlessly call crle on Solaris
type: resource usage
versions: Python 2.7, Python 3.4
Added file: http://bugs.python.org/file37800/crle-fix.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23287>
_______________________________________

From report at bugs.python.org  Wed Jan 21 11:37:56 2015
From: report at bugs.python.org (Mike Sampson)
Date: Wed, 21 Jan 2015 10:37:56 +0000
Subject: [New-bugs-announce] [issue23288] subprocess.Popen close_fds
	behaviour differs between 3.2	and 3.4
Message-ID: <1421836676.03.0.70112175637.issue23288@psf.upfronthosting.co.za>


New submission from Mike Sampson:

I'm seeing differing behaviour with subprocess.Popen(..., close_fds = False) between 3.2 and 3.4. The docs don't say this is meant to be the case as far as I can see.

Python 3.2.3 on Debian Wheezy
=============================

>>> import subprocess
>>> import os
>>> r,w = os.pipe()
>>> p = subprocess.Popen('ls /dev/fd/*', shell = True, close_fds = False)
>>> ls: cannot access /dev/fd/5: No such file or directory
/dev/fd/0  /dev/fd/1  /dev/fd/2  /dev/fd/3  /dev/fd/4

Python 3.4.2 on Arch Linux
==========================

>>> import subprocess
>>> import os
>>> r,w = os.pipe()
>>> p = subprocess.Popen('ls /dev/fd/*', shell = True, close_fds = False)                                                                                                            
>>> ls: cannot access /dev/fd/3: No such file or directory
/dev/fd/0  /dev/fd/1  /dev/fd/2

In 3.4 even though close_fds is False the fds are closed in the child. Using pass_fds works around this though I would like to know if this is a bug, documentation issue, or am I missing something here?

----------
assignee: docs at python
components: Documentation
messages: 234428
nosy: docs at python, mfs
priority: normal
severity: normal
status: open
title: subprocess.Popen close_fds behaviour differs between 3.2 and 3.4
type: behavior
versions: Python 3.2, Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23288>
_______________________________________

From report at bugs.python.org  Wed Jan 21 13:43:00 2015
From: report at bugs.python.org (Piotr Majkrzak)
Date: Wed, 21 Jan 2015 12:43:00 +0000
Subject: [New-bugs-announce] [issue23289] concurrent.futures.Executor.map is
	not equivalent to map.
Message-ID: <1421844180.03.0.988670092885.issue23289@psf.upfronthosting.co.za>


New submission from Piotr Majkrzak:

In documentation https://docs.python.org/3/library/concurrent.futures.html#concurrent.futures.Executor.map is writen that this fucntion is equivalent to the builtin map. But it is not true due to the fact that it is not lazy evalueded. The reason is in https://hg.python.org/cpython/file/0893b9ee44ea/Lib/concurrent/futures/_base.py#l548 where the full list of features is created.

I don't find any reasonable solutions, but in my case following code was suitable.
https://gist.github.com/06bbd83eccd4083c68d0

----------
components: Library (Lib)
messages: 234433
nosy: majkrzak
priority: normal
severity: normal
status: open
title: concurrent.futures.Executor.map is not equivalent to map.
type: behavior
versions: Python 3.2, Python 3.3, Python 3.4, Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23289>
_______________________________________

From report at bugs.python.org  Wed Jan 21 14:50:08 2015
From: report at bugs.python.org (Serhiy Storchaka)
Date: Wed, 21 Jan 2015 13:50:08 +0000
Subject: [New-bugs-announce] [issue23290] Faster set copying
Message-ID: <1421848208.43.0.905402107396.issue23290@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka:

Proposed patch makes faster creating new set from other set. The benefit is only few percents when there are no hash conflicts, but can be significant if there are hash duplicates.

$ ./python -m timeit -s "s = set(range(10**4))" -- "frozenset(s)"
Unpatched: 1000 loops, best of 3: 658 usec per loop
Patched: 1000 loops, best of 3: 620 usec per loop

$ ./python -m timeit -s "s = {i+(j<<64) for i in range(10**3) for j in range(10)}" -- "frozenset(s)"
Unpatched: 100 loops, best of 3: 6.72 msec per loop
Patched: 100 loops, best of 3: 2.05 msec per loop

$ ./python -m timeit -s "s = {i+(j<<64) for i in range(10**2) for j in range(10**2)}" -- "frozenset(s)"
Unpatched: 100 loops, best of 3: 14 msec per loop
Patched: 100 loops, best of 3: 2.67 msec per loop

It should also affect set.copy and operations which makes implicit copy (most set operators). The effect should be larger for objects with slow equality operator.

set_find_free_slot() can be used to prevent a catastrophic linear pile-up in issue23259.

----------
components: Interpreter Core
messages: 234437
nosy: serhiy.storchaka
priority: normal
severity: normal
stage: patch review
status: open
title: Faster set copying
type: performance
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23290>
_______________________________________

From report at bugs.python.org  Wed Jan 21 18:53:44 2015
From: report at bugs.python.org (Albert Zeyer)
Date: Wed, 21 Jan 2015 17:53:44 +0000
Subject: [New-bugs-announce] [issue23291] Documentation about Py_Finalize():
	Freeing objects
Message-ID: <1421862824.96.0.769285888679.issue23291@psf.upfronthosting.co.za>


New submission from Albert Zeyer:

The documentation about Py_Finalize() about freeing objects is not exactly clear.

Esp., when I have called Py_INCREF somewhere, those objects will always have ob_refcnt > 0 unless I call Py_DECREF somewhere, what about these objects? Will they be freed in Py_Finalize() or not? If not, what can I do? I guess calling Py_DECREF after Py_Finalize() is not allowed.

I studied the Py_Finalize() code but I'm not exactly sure. It looks like such objects would not get freed (unless some implicit magic is happening somewhere). The comment about the last commented-out PyGC_Collect() call is also insightful and somewhat unsatisfying.

Maybe PyObject_Free() still works, although that would not free any further referenced objects, I guess. Also, I'm not exactly sure about the pymalloc arenas.

With pymalloc, maybe we could just free all pymalloc arenas and it would free all memory allocated by Python objects? Of course, that would not call any advanced tp_dealloc functions which might be important to be called, but we would never ever call those anyway anymore, or would we (how?)?
So, maybe it would be good to do that?

----------
components: Interpreter Core
messages: 234441
nosy: Albert.Zeyer
priority: normal
severity: normal
status: open
title: Documentation about Py_Finalize(): Freeing objects
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23291>
_______________________________________

From report at bugs.python.org  Wed Jan 21 21:00:29 2015
From: report at bugs.python.org (Mark Summerfield)
Date: Wed, 21 Jan 2015 20:00:29 +0000
Subject: [New-bugs-announce] [issue23292] Enum doc suggestion
Message-ID: <1421870429.3.0.139624304143.issue23292@psf.upfronthosting.co.za>


New submission from Mark Summerfield:

I think it would be worth documenting
globals().update(MyEnumeration.__members__) in the "Interesting
Examples" section of the enum docs.

I suspect that most people will find that importing enums is annoying
because they'll get

import A
print(A.MyEnumeration.MAX)

when they're more used to

import A
print(A.MAX)

Of course the latter is easily achieved using the globals().update()
trick (as used in signals.py in 3.5).

Georg suggested I add this to the tracker.

----------
assignee: docs at python
components: Documentation
files: py-enum.tar.gz
messages: 234442
nosy: docs at python, mark
priority: normal
severity: normal
status: open
title: Enum doc suggestion
type: enhancement
versions: Python 3.5, Python 3.6
Added file: http://bugs.python.org/file37809/py-enum.tar.gz

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23292>
_______________________________________

From report at bugs.python.org  Wed Jan 21 23:32:53 2015
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 21 Jan 2015 22:32:53 +0000
Subject: [New-bugs-announce] [issue23293] [Windows] asyncio: race condition
	related in	IocpProactor.connect_pipe()
Message-ID: <1421879573.47.0.0852926503812.issue23293@psf.upfronthosting.co.za>


New submission from STINNER Victor:

Currently, IocpProactor.connect_pipe() is implemented with QueueUserWorkItem() which starts a thread that cannot be interrupted. Because of that, this function requires special cases in _register() and close() methods of IocpProactor.

While fixing the issue #23095, I saw that IocpProactor.connect_pipe() causes "GetQueuedCompletionStatus() returned an unexpected event" messages to be logged, but also to hang the test suite.

I propose a solution to reimplement IocpProactor.connect_pipe() without a thread:
https://code.google.com/p/tulip/issues/detail?id=197

It should fix this issue.

----------
components: Windows, asyncio
messages: 234448
nosy: gvanrossum, haypo, steve.dower, tim.golden, yselivanov, zach.ware
priority: normal
severity: normal
status: open
title: [Windows] asyncio: race condition related in IocpProactor.connect_pipe()
versions: Python 3.4, Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23293>
_______________________________________

From report at bugs.python.org  Wed Jan 21 23:34:04 2015
From: report at bugs.python.org (aruseni)
Date: Wed, 21 Jan 2015 22:34:04 +0000
Subject: [New-bugs-announce] [issue23294] A typo in the tutorial
Message-ID: <1421879644.03.0.144985173362.issue23294@psf.upfronthosting.co.za>


New submission from aruseni:

https://docs.python.org/3/tutorial/controlflow.html

> In many ways the object returned by range() behaves as if it is a list, but in fact it isn?t.

----------
messages: 234449
nosy: aruseni
priority: normal
severity: normal
status: open
title: A typo in the tutorial

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23294>
_______________________________________

From report at bugs.python.org  Thu Jan 22 00:40:13 2015
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 21 Jan 2015 23:40:13 +0000
Subject: [New-bugs-announce] [issue23295] [Windows] asyncio: add UDP support
	to ProactorEventLoop
Message-ID: <1421883613.58.0.0395481360088.issue23295@psf.upfronthosting.co.za>


New submission from STINNER Victor:

ProactorEventLoop lacks UDP support: create_datagram_endpoint() is not supported.

New functions should be added to the _overlapped modul. Example: add maybe WSARecvFrom()?

----------
components: Windows, asyncio
messages: 234456
nosy: gvanrossum, haypo, steve.dower, tim.golden, yselivanov, zach.ware
priority: normal
severity: normal
status: open
title: [Windows] asyncio: add UDP support to ProactorEventLoop
versions: Python 3.4, Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23295>
_______________________________________

From report at bugs.python.org  Thu Jan 22 05:40:12 2015
From: report at bugs.python.org (Ben Finney)
Date: Thu, 22 Jan 2015 04:40:12 +0000
Subject: [New-bugs-announce] =?utf-8?b?W2lzc3VlMjMyOTZdIOKAmHRva2VuaXpl?=
 =?utf-8?q?=2Edetect=5Fencoding=E2=80=99_is_confused_between_text_and_byte?=
 =?utf-8?q?s=3A_no_=E2=80=98startswith=E2=80=99_method_on_a_byte_string?=
Message-ID: <1421901612.51.0.543377948137.issue23296@psf.upfronthosting.co.za>


New submission from Ben Finney:

In `tokenize.detect_encoding` is the following code::

    first = read_or_stop()
    if first.startswith(BOM_UTF8):
        # ?

The `read_or_stop` function is defined as::

    def read_or_stop():
        try:
            return readline()
        except StopIteration:
            return b''

So, on catching ``StopIteration``, the return value will be a byte string. The `detect_encoding` code then immediately calls `sartswith`, which fails::

    File "/usr/lib/python3.4/tokenize.py", line 409, in detect_encoding
      if first.startswith(BOM_UTF8):
  TypeError: startswith first arg must be str or a tuple of str, not bytes

One or both of those locations in the code is wrong. Either `read_or_stop` should never return a byte string; or `detect_encoding` should not assume it can call `startswith` on the result.

----------
components: Library (Lib)
messages: 234470
nosy: bignose
priority: normal
severity: normal
status: open
title: ?tokenize.detect_encoding? is confused between text and bytes: no ?startswith? method on a byte string
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23296>
_______________________________________

From report at bugs.python.org  Thu Jan 22 05:40:26 2015
From: report at bugs.python.org (Ben Finney)
Date: Thu, 22 Jan 2015 04:40:26 +0000
Subject: [New-bugs-announce] =?utf-8?b?W2lzc3VlMjMyOTddIOKAmHRva2VuaXpl?=
 =?utf-8?q?=2Edetect=5Fencoding=E2=80=99_is_confused_between_text_and_byte?=
 =?utf-8?q?s=3A_no_=E2=80=98startswith=E2=80=99_method_on_a_byte_string?=
Message-ID: <1421901626.89.0.392776519963.issue23297@psf.upfronthosting.co.za>


New submission from Ben Finney:

In `tokenize.detect_encoding` is the following code::

    first = read_or_stop()
    if first.startswith(BOM_UTF8):
        # ?

The `read_or_stop` function is defined as::

    def read_or_stop():
        try:
            return readline()
        except StopIteration:
            return b''

So, on catching ``StopIteration``, the return value will be a byte string. The `detect_encoding` code then immediately calls `sartswith`, which fails::

    File "/usr/lib/python3.4/tokenize.py", line 409, in detect_encoding
      if first.startswith(BOM_UTF8):
  TypeError: startswith first arg must be str or a tuple of str, not bytes

One or both of those locations in the code is wrong. Either `read_or_stop` should never return a byte string; or `detect_encoding` should not assume it can call `startswith` on the result.

----------
components: Library (Lib)
messages: 234471
nosy: bignose
priority: normal
severity: normal
status: open
title: ?tokenize.detect_encoding? is confused between text and bytes: no ?startswith? method on a byte string
type: crash
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23297>
_______________________________________

From report at bugs.python.org  Thu Jan 22 06:41:52 2015
From: report at bugs.python.org (dongwm)
Date: Thu, 22 Jan 2015 05:41:52 +0000
Subject: [New-bugs-announce] [issue23298] Add
	ArgumentParser.add_mutually_dependence_group
Message-ID: <1421905312.49.0.407798450784.issue23298@psf.upfronthosting.co.za>


New submission from dongwm:

Sometimes I need to use argparse like this:


>>> parser = argparse.ArgumentParser(prog='PROG')
>>> group = parser.add_mutually_dependence_group()
>>> group.add_argument('--foo')
>>> group.add_argument('--bar')
>>> parser.parse_args(['--foo', 'f', '--bar', 'b'])
Namespace(bar='b', foo='f')
>>> parser.parse_args(['--foo', 'f'])
PROG: error: --foo dependence on --bar
>>> parser.parse_args(['--bar', 'b'])
PROG: error: --bar dependence on --foo

I have some optional argument. but if any argument in a group was present on the command line. i need the others must also was present on. so i think ``add_mutually_dependence_group`` does make sense.

----------
components: Library (Lib)
files: argparse_lib.patch
keywords: patch
messages: 234475
nosy: bethard, dongwm
priority: normal
severity: normal
status: open
title: Add ArgumentParser.add_mutually_dependence_group
type: enhancement
versions: Python 2.7, Python 3.2, Python 3.3, Python 3.4, Python 3.5
Added file: http://bugs.python.org/file37814/argparse_lib.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23298>
_______________________________________

From report at bugs.python.org  Thu Jan 22 09:53:41 2015
From: report at bugs.python.org (Liang Zhang)
Date: Thu, 22 Jan 2015 08:53:41 +0000
Subject: [New-bugs-announce] [issue23299] Documentation correction - 5.1.4.
	List Comprehensions
Message-ID: <1421916821.61.0.582688428376.issue23299@psf.upfronthosting.co.za>


New submission from Liang Zhang:

This was copied from Chapter 5 Data Structure in Python language tutorial.

The link I was looking for:  https://docs.python.org/2.7/tutorial/datastructures.html

Some thing might be incorrect and need you to update them in
(5.1.4. List Comprehensions)

It should be like this:
------------------start------------------
>>> vec = [-4, -2, 0, 2, 4]
>>> # create a new list with the values doubled
>>> [x*2 for x in vec]
[-8, -4, 0, 4, 8]
>>> # filter the list to exclude negative numbers
>>> [x for x in vec if x >= 0]
[0, 2, 4]
>>> # apply a function to all the elements
>>> [abs(x) for x in vec]
[4, 2, 0, 2, 4]                     >>>>>>>>>>>It should be [0, 2, 4]<<<<<<<<<<<
------------------end------------------

----------
assignee: docs at python
components: Documentation
messages: 234485
nosy: Liang.Zhang, docs at python
priority: normal
severity: normal
status: open
title: Documentation correction - 5.1.4. List Comprehensions
type: resource usage
versions: Python 2.7, Python 3.2, Python 3.3, Python 3.4, Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23299>
_______________________________________

From report at bugs.python.org  Thu Jan 22 12:50:08 2015
From: report at bugs.python.org (Guohua Ouyang)
Date: Thu, 22 Jan 2015 11:50:08 +0000
Subject: [New-bugs-announce] [issue23300] An improper change in httplib.py
Message-ID: <1421927408.72.0.18242677241.issue23300@psf.upfronthosting.co.za>


New submission from Guohua Ouyang:

Following the issue 7776, there is a patch for 2.7 version.

Which changes the method of class HTTPConnection from "_set_hostport" to "_get_hostport"[1], it seems introduce in some incompatibility issues.

On my system, the file "/usr/lib64/python2.7/site-packages/mercurial/url.py" from package mercurial-3.0-2.fc21 still use the old method "_set_hostport". I met an error "AttributeError: httpsconnection instance has no attribute '_set_hostport'" when use this package. I only see this incompatibility issue so far.

And in the file httplib.py itself, [2] still use "self._conn._set_hostport(host, port)", which should be "self._conn._get_hostport(host, port)" if it's settled to rename "_set_hostport" to "_get_hostport".

[1] https://github.com/python/cpython/blob/2.7/Lib/httplib.py#L743
[2] https://github.com/python/cpython/blob/2.7/Lib/httplib.py#L1132

----------
components: Library (Lib)
messages: 234488
nosy: guohua
priority: normal
severity: normal
status: open
title: An improper change in httplib.py
type: enhancement
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23300>
_______________________________________

From report at bugs.python.org  Thu Jan 22 16:58:05 2015
From: report at bugs.python.org (Sebastian Bank)
Date: Thu, 22 Jan 2015 15:58:05 +0000
Subject: [New-bugs-announce] [issue23301] ConfigParser does not handle
	square brackets in section	name
Message-ID: <1421942285.34.0.084703922586.issue23301@psf.upfronthosting.co.za>


New submission from Sebastian Bank:

ConfigParser parses section lines containing square brackets like '[spam [eggs] spam]' up to the first instead of the last occurrence of ']' preventing roundtrips:

>>> s = StringIO()
>>> c1 = ConfigParser()
>>> c1.add_section('spam [eggs]')
>>> c1.write(s)
>>> s.seek(0)
>>> c2 = ConfigParser()
>>> c2.readfp(s)
>>> assert c1.sections() == c2.sections()  # fails

Potential fix: change the second line of SECTCRE from r'(?P<header>[^]]+)' to r'(?P<header>.+?)'.

If the parsing behaviour cannot be changed, the user should at least be warned about supplying data that breaks the roundtrip.

----------
components: Library (Lib)
messages: 234497
nosy: xflr6
priority: normal
severity: normal
status: open
title: ConfigParser does not handle square brackets in section name
type: behavior
versions: Python 2.7, Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23301>
_______________________________________

From report at bugs.python.org  Thu Jan 22 18:00:35 2015
From: report at bugs.python.org (Demian Brecht)
Date: Thu, 22 Jan 2015 17:00:35 +0000
Subject: [New-bugs-announce] [issue23302] Small fixes around the use of TCP
	MSS in http.client
Message-ID: <1421946035.45.0.334946806325.issue23302@psf.upfronthosting.co.za>


New submission from Demian Brecht:

There are a couple of small issues with the determination of whether or not a request can fit in a single TCP/IP packet in http.client.

1. The MSS is hardcoded
2. The TCP data size is calculated as only the message body. This is incorrect as the size of the HTTP headers should also be accounted for.

I suggest two changes be made to fix this:

1. The MSS is retrieved (on platforms that support it) using getsockopt(socket.IPPROTO_TCP, socket.TCP_MAXSEG). If the platform doesn't support it (i.e. Windows), it should default to the currently hardcoded value. This does add a requirement for a connection to be established prior to this calculation (currently the connection is only established after).
2. The HTTP headers are added to the full payload size calculation when compared against the connection's MSS value.

This is still a best guess based on minimal TCP/IP header sizes, but if options or extension headers are used, the packet may still be split at the lower levels. This is fine because what we're trying to avoid is multiple send()s where the payload is less than the MSS.

----------
messages: 234500
nosy: demian.brecht
priority: normal
severity: normal
status: open
title: Small fixes around the use of TCP MSS in http.client

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23302>
_______________________________________

From report at bugs.python.org  Thu Jan 22 22:30:03 2015
From: report at bugs.python.org (STINNER Victor)
Date: Thu, 22 Jan 2015 21:30:03 +0000
Subject: [New-bugs-announce] [issue23303] test_license_exists_at_url() of
	test_site fails on "x86	XP-4 3.4" buildbot
Message-ID: <1421962203.05.0.381123323956.issue23303@psf.upfronthosting.co.za>


New submission from STINNER Victor:

test_license_exists_at_url() of test_site fails on "x86 XP-4 3.4" buildbot. I don't understand why the test pass on "x86 XP-4 3.x", is it the same server?

http://buildbot.python.org/all/builders/x86%20XP-4%203.4/builds/747/steps/test/logs/stdio

======================================================================
ERROR: test_license_exists_at_url (test.test_site.ImportSideEffectTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\cygwin\home\db3l\buildarea\3.4.bolen-windows\build\lib\urllib\request.py", line 1182, in do_open
    h.request(req.get_method(), req.selector, req.data, headers)
  File "D:\cygwin\home\db3l\buildarea\3.4.bolen-windows\build\lib\http\client.py", line 1088, in request
    self._send_request(method, url, body, headers)
  File "D:\cygwin\home\db3l\buildarea\3.4.bolen-windows\build\lib\http\client.py", line 1126, in _send_request
    self.endheaders(body)
  File "D:\cygwin\home\db3l\buildarea\3.4.bolen-windows\build\lib\http\client.py", line 1084, in endheaders
    self._send_output(message_body)
  File "D:\cygwin\home\db3l\buildarea\3.4.bolen-windows\build\lib\http\client.py", line 922, in _send_output
    self.send(msg)
  File "D:\cygwin\home\db3l\buildarea\3.4.bolen-windows\build\lib\http\client.py", line 857, in send
    self.connect()
  File "D:\cygwin\home\db3l\buildarea\3.4.bolen-windows\build\lib\http\client.py", line 1231, in connect
    server_hostname=server_hostname)
  File "D:\cygwin\home\db3l\buildarea\3.4.bolen-windows\build\lib\ssl.py", line 365, in wrap_socket
    _context=self)
  File "D:\cygwin\home\db3l\buildarea\3.4.bolen-windows\build\lib\ssl.py", line 583, in __init__
    self.do_handshake()
  File "D:\cygwin\home\db3l\buildarea\3.4.bolen-windows\build\lib\ssl.py", line 810, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:600)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:\cygwin\home\db3l\buildarea\3.4.bolen-windows\build\lib\test\test_site.py", line 426, in test_license_exists_at_url
    with urllib.request.urlopen(req) as data:
  File "D:\cygwin\home\db3l\buildarea\3.4.bolen-windows\build\lib\urllib\request.py", line 161, in urlopen
    return opener.open(url, data, timeout)
  File "D:\cygwin\home\db3l\buildarea\3.4.bolen-windows\build\lib\urllib\request.py", line 469, in open
    response = meth(req, response)
  File "D:\cygwin\home\db3l\buildarea\3.4.bolen-windows\build\lib\urllib\request.py", line 579, in http_response
    'http', request, response, code, msg, hdrs)
  File "D:\cygwin\home\db3l\buildarea\3.4.bolen-windows\build\lib\urllib\request.py", line 501, in error
    result = self._call_chain(*args)
  File "D:\cygwin\home\db3l\buildarea\3.4.bolen-windows\build\lib\urllib\request.py", line 441, in _call_chain
    result = func(*args)
  File "D:\cygwin\home\db3l\buildarea\3.4.bolen-windows\build\lib\urllib\request.py", line 684, in http_error_302
    return self.parent.open(new, timeout=req.timeout)
  File "D:\cygwin\home\db3l\buildarea\3.4.bolen-windows\build\lib\urllib\request.py", line 463, in open
    response = self._open(req, data)
  File "D:\cygwin\home\db3l\buildarea\3.4.bolen-windows\build\lib\urllib\request.py", line 481, in _open
    '_open', req)
  File "D:\cygwin\home\db3l\buildarea\3.4.bolen-windows\build\lib\urllib\request.py", line 441, in _call_chain
    result = func(*args)
  File "D:\cygwin\home\db3l\buildarea\3.4.bolen-windows\build\lib\urllib\request.py", line 1225, in https_open
    context=self._context, check_hostname=self._check_hostname)
  File "D:\cygwin\home\db3l\buildarea\3.4.bolen-windows\build\lib\urllib\request.py", line 1184, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:600)>

----------
components: Tests
keywords: buildbot
messages: 234510
nosy: alex, haypo, pitrou
priority: normal
severity: normal
status: open
title: test_license_exists_at_url() of test_site fails on "x86 XP-4 3.4" buildbot
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23303>
_______________________________________

From report at bugs.python.org  Thu Jan 22 22:46:50 2015
From: report at bugs.python.org (CliffM)
Date: Thu, 22 Jan 2015 21:46:50 +0000
Subject: [New-bugs-announce] [issue23304] Unused Superclass in calendar.py
Message-ID: <1421963210.26.0.700938779083.issue23304@psf.upfronthosting.co.za>


New submission from CliffM:

calendar.py ( lines 17,18 )  is not used :

# Exception raised for bad input (with string parameter for details)
error = ValueError

This could either be deleted OR used as the superclass in the next two class declarations as the comment suggests.

----------
components: Library (Lib), Tests
messages: 234513
nosy: CliffM
priority: normal
severity: normal
status: open
title: Unused Superclass in calendar.py
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23304>
_______________________________________

From report at bugs.python.org  Fri Jan 23 11:37:03 2015
From: report at bugs.python.org (Juha Lemmetti)
Date: Fri, 23 Jan 2015 10:37:03 +0000
Subject: [New-bugs-announce] [issue23305] RotatingFileHandler does not
	rotate if backupCount is 0
Message-ID: <1422009423.63.0.642413518943.issue23305@psf.upfronthosting.co.za>


New submission from Juha Lemmetti:

If RotatingFileHandler is initialized with backupCount 0, the file does not rotate but grows indefinitely. This is not self-evident from the documentation.

Suggestion: either rotate (discard) the log if backupCount==0 or mention the current operation the documentation. The former leaves the user with only one log message at the worst case. 

In the latter case, the documentation can be modified to explicitly mention what the result will be if backupCount is left to its default value of 0, i.e. that the RotatingFileHandler will grow the file indefinitely regardless of the value of maxBytes value. The default value for the backupCount could also be modified, as the default value does nothing.

Tested with Windows-Python2.7, Linux-Python 3.3.2+ (Ubuntu-14.04). Can be reproduced with a simple logging example with backupCount == 0 and maxBytes > 0

----------
components: Library (Lib)
messages: 234552
nosy: Juha.Lemmetti, vinay.sajip
priority: normal
severity: normal
status: open
title: RotatingFileHandler does not rotate if backupCount is 0
type: enhancement
versions: Python 2.7, Python 3.3, Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23305>
_______________________________________

From report at bugs.python.org  Sat Jan 24 00:49:26 2015
From: report at bugs.python.org (Danny Yoo)
Date: Fri, 23 Jan 2015 23:49:26 +0000
Subject: [New-bugs-announce] [issue23306] zlib.crc32 raises OverflowError at
	argument-parsing time	on large strings
Message-ID: <1422056966.81.0.518971767948.issue23306@psf.upfronthosting.co.za>


New submission from Danny Yoo:

Reproduction steps:

---
$ python2.7 -c "import zlib;zlib.crc32('a'*(1<<31))"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
OverflowError: size does not fit in an int
---

We ran into this bug in zlib.crc32 when using zipfile.writestr() with a very large string; as soon as zipfile tried to write the crc checksum, it raised this error.


Python 3 does not appear to suffer from this bug.

----------
components: Library (Lib)
messages: 234587
nosy: Danny.Yoo, gregory.p.smith
priority: normal
severity: normal
status: open
title: zlib.crc32 raises OverflowError at argument-parsing time on large strings
type: behavior
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23306>
_______________________________________

From report at bugs.python.org  Sat Jan 24 07:37:50 2015
From: report at bugs.python.org (Jim Bridgewater)
Date: Sat, 24 Jan 2015 06:37:50 +0000
Subject: [New-bugs-announce] [issue23307] python hangs on call to numpy.outer
Message-ID: <1422081470.94.0.280484065129.issue23307@psf.upfronthosting.co.za>


New submission from Jim Bridgewater:

python hangs when this script is run and if it is allowed to continue running OS X produces system out of memory errors

----------
files: dome_projection.py
messages: 234593
nosy: jwbwater
priority: normal
severity: normal
status: open
title: python hangs on call to numpy.outer
type: crash
versions: Python 2.7
Added file: http://bugs.python.org/file37833/dome_projection.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23307>
_______________________________________

From report at bugs.python.org  Sat Jan 24 08:16:41 2015
From: report at bugs.python.org (David Motlagh)
Date: Sat, 24 Jan 2015 07:16:41 +0000
Subject: [New-bugs-announce] [issue23308] a bug in Instructions section 4.1
Message-ID: <1867408449.522169.1422083738920.JavaMail.yahoo@mail.yahoo.com>


New submission from David Motlagh:

Hi,
I either found a bug or ?am doing the steps wrong. ?Could you please check the instructions in Section 4.1 to determine if there really is a syntax error?
Thanks,David Motlagh?

----------
messages: 234596
nosy: Dmot
priority: normal
severity: normal
status: open
title: a bug in Instructions section 4.1

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23308>
_______________________________________

From report at bugs.python.org  Sat Jan 24 14:52:40 2015
From: report at bugs.python.org (Marien Zwart)
Date: Sat, 24 Jan 2015 13:52:40 +0000
Subject: [New-bugs-announce] [issue23309] Hang on interpreter shutdown if
	daemon thread prints to	stdout
Message-ID: <1422107560.4.0.0145420431804.issue23309@psf.upfronthosting.co.za>


New submission from Marien Zwart:

A script spawning a single daemon thread calling print() in a loop (like the attached) will usually hang on shutdown in Python 3.4.2 and hg rev 8d802fb6ae32. Attaching gdb at that point shows the following:

(gdb) thread apply all bt

Thread 1 (Thread 0x7fd927d58700 (LWP 30274)):
#0  sem_wait () at ../sysdeps/unix/sysv/linux/x86_64/sem_wait.S:85
#1  0x00000000005282fe in PyThread_acquire_lock_timed (lock=0x1c5ea30, microseconds=microseconds at entry=-1, 
    intr_flag=intr_flag at entry=0) at Python/thread_pthread.h:352
#2  0x0000000000528414 in PyThread_acquire_lock (lock=<optimized out>, waitflag=waitflag at entry=1)
    at Python/thread_pthread.h:556
#3  0x0000000000567e4c in _enter_buffered_busy (self=0x7fd927bc2b48) at ./Modules/_io/bufferedio.c:327
#4  buffered_flush (self=0x7fd927bc2b48, args=<optimized out>) at ./Modules/_io/bufferedio.c:874
#5  0x000000000042822a in PyObject_Call (func=0x7fd9277b69d8, arg=<optimized out>, kw=<optimized out>)
    at Objects/abstract.c:2086
#6  0x00000000004290e4 in call_function_tail (args=0x7fd927b8d048, callable=0x7fd9277b69d8) at Objects/abstract.c:2124
#7  callmethod (is_size_t=1, va=0x7fff5c6cf6b0, format=0x0, func=0x7fd9277b69d8) at Objects/abstract.c:2193
#8  _PyObject_CallMethodId_SizeT (o=<optimized out>, name=<optimized out>, format=0x0) at Objects/abstract.c:2279
#9  0x000000000042822a in PyObject_Call (func=0x7fd9277b6990, arg=<optimized out>, kw=<optimized out>)
    at Objects/abstract.c:2086
#10 0x0000000000428cc4 in call_function_tail (args=0x7fd927b8d048, callable=0x7fd9277b6990) at Objects/abstract.c:2124
#11 callmethod (is_size_t=0, va=0x7fff5c6cf7e0, format=0x5b9924 "", func=0x7fd9277b6990) at Objects/abstract.c:2193
#12 _PyObject_CallMethodId (o=o at entry=0x7fd927b5d3a8, name=name at entry=0x862b00 <PyId_flush>, 
    format=format at entry=0x5b9924 "") at Objects/abstract.c:2238
#13 0x000000000050a521 in flush_std_files () at Python/pylifecycle.c:488
#14 0x000000000050a5aa in Py_Finalize () at Python/pylifecycle.c:550
#15 0x000000000041fc92 in Py_Main (argc=-1, argv=0x1) at Modules/main.c:787
#16 0x000000000041be3c in main (argc=2, argv=<optimized out>) at ./Programs/python.c:69

The daemon thread has exited, and the main thread hangs trying to flush stdout.

I haven't fully tracked down what happens here, but I think it's this:

- daemon thread calls ENTER_BUFFERED on stdout
- daemon thread drops the GIL before writing to stdout
- main thread grabs the GIL and starts exiting
- main thread sets _Py_Finalizing, signaling daemon threads to exit
- main thread calls flush_std_files and drops the GIL
- daemon thread grabs the GIL and immediately exits, without reaching LEAVE_BUFFERED
- main thread deadlocks trying to ENTER_BUFFERED the same file

If that is what happens, I don't really see how to fix it (it's an example of daemon threads not releasing their resources, which the documentation warns about). But it's obviously unfortunate if merely writing to stdout/err is such a resource.

----------
components: Interpreter Core
files: dio.py
messages: 234615
nosy: marienz
priority: normal
severity: normal
status: open
title: Hang on interpreter shutdown if daemon thread prints to stdout
type: behavior
versions: Python 3.4, Python 3.5
Added file: http://bugs.python.org/file37835/dio.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23309>
_______________________________________

From report at bugs.python.org  Sat Jan 24 18:20:39 2015
From: report at bugs.python.org (berdario)
Date: Sat, 24 Jan 2015 17:20:39 +0000
Subject: [New-bugs-announce] [issue23310] MagicMock constructor
	configuration fails for magic	methods
Message-ID: <1422120039.57.0.541541438715.issue23310@psf.upfronthosting.co.za>


New submission from berdario:

I guess this should be expected... too much magic :P

>>> from unittest.mock import MagicMock
>>> MagicMock(**{'__hash__.return_value': "FIXME"})
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/nix/store/qlvbf3n3y34idxcgwwhsi9pq26v28q99-python3-3.4.2/lib/python3.4/unittest/mock.py", line 1772, in __init__
    _safe_super(MagicMixin, self).__init__(*args, **kw)
  File "/nix/store/qlvbf3n3y34idxcgwwhsi9pq26v28q99-python3-3.4.2/lib/python3.4/unittest/mock.py", line 881, in __init__
    _spec_state, _new_name, _new_parent, **kwargs
  File "/nix/store/qlvbf3n3y34idxcgwwhsi9pq26v28q99-python3-3.4.2/lib/python3.4/unittest/mock.py", line 410, in __init__
    self.configure_mock(**kwargs)
  File "/nix/store/qlvbf3n3y34idxcgwwhsi9pq26v28q99-python3-3.4.2/lib/python3.4/unittest/mock.py", line 560, in configure_mock
    setattr(obj, final, val)
AttributeError: 'method-wrapper' object has no attribute 'return_value'


The same happens with e.g. __str__

>>> m.configure_mock(**{'__hash__.return_value': 1})

works just fine, instead

----------
components: Library (Lib)
messages: 234623
nosy: berdario
priority: normal
severity: normal
status: open
title: MagicMock constructor configuration fails for magic methods
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23310>
_______________________________________

From report at bugs.python.org  Sat Jan 24 18:40:01 2015
From: report at bugs.python.org (Steve Dower)
Date: Sat, 24 Jan 2015 17:40:01 +0000
Subject: [New-bugs-announce] [issue23311] Update PC/example_nt and
	extending/windows.rst
Message-ID: <1422121201.94.0.918332856849.issue23311@psf.upfronthosting.co.za>


New submission from Steve Dower:

The installer for 3.5 will include an option to install debug binaries alongside the release ones, so we should update extending/windows.rst to explain that and generally describe how to build an extension. It should also refer to python3.dll at least once...

The example project in PC/example_nt is fairly outdated. It could be updated to automatically find a 3.5 install and use the debug or release binaries/libraries from there.

We could also include a 'test' project for PTVS (http://pytools.codeplex.com/) that helps write Python code for testing the extension and also has mixed-mode debugging enabled so people can step between the Python and the C code.

(In fact, the PTVS team wants to do a sample project like this anyway, so maybe we can remove PC/example_nt and point to theirs...)

----------
assignee: docs at python
components: Documentation, Windows
messages: 234624
nosy: docs at python, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Update PC/example_nt and extending/windows.rst
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23311>
_______________________________________

From report at bugs.python.org  Sat Jan 24 18:42:05 2015
From: report at bugs.python.org (Benjamin Peterson)
Date: Sat, 24 Jan 2015 17:42:05 +0000
Subject: [New-bugs-announce] [issue23312] google thinks the docs are mobile
	unfriendly
Message-ID: <1422121325.13.0.140473067698.issue23312@psf.upfronthosting.co.za>


New submission from Benjamin Peterson:

According to Google webmaster tools, the documentation has the following failings on mobile devices:

- The font size is too small. https://developers.google.com/speed/docs/insights/UseLegibleFontSizes
- The viewport is not configured.
- Touch controls are too close together.

You can look at the complaints on (for example) https://developers.google.com/speed/pagespeed/insights/?url=https%3A%2F%2Fdocs.python.org%2F3%2Flibrary%2Ffunctions.html

----------
assignee: docs at python
components: Documentation
messages: 234625
nosy: benjamin.peterson, docs at python
priority: normal
severity: normal
stage: needs patch
status: open
title: google thinks the docs are mobile unfriendly
type: enhancement
versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23312>
_______________________________________

From report at bugs.python.org  Sat Jan 24 20:01:57 2015
From: report at bugs.python.org (Gerrit Barrere)
Date: Sat, 24 Jan 2015 19:01:57 +0000
Subject: [New-bugs-announce] [issue23313] Wrong complex variable being
	altered
Message-ID: <1422126117.91.0.00617792248129.issue23313@psf.upfronthosting.co.za>


New submission from Gerrit Barrere:

The bug occurs on line 77 of the attached script. When I change the imaginary part of 'z' on line 77, it also changes the imaginary part of 'zload', which is supposed to remain constant.  Putting a dummy assignment 'z = z + 0' on line 76 fixes the bug.  This is all described in the comments around line 77 also.

I found this in the PyCharm IDE by putting a breakpoint just after line 77 and watching variables 'z' and 'zload'.  The function 'model()' is called repeatedly in this script, and you can see 'zload' changing every time the function is called.

----------
files: ComplexBug.py
messages: 234631
nosy: CarpeCimex
priority: normal
severity: normal
status: open
title: Wrong complex variable being altered
type: behavior
versions: Python 3.4
Added file: http://bugs.python.org/file37839/ComplexBug.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23313>
_______________________________________

From report at bugs.python.org  Sun Jan 25 00:27:24 2015
From: report at bugs.python.org (Steve Dower)
Date: Sat, 24 Jan 2015 23:27:24 +0000
Subject: [New-bugs-announce] [issue23314] Disabling CRT asserts in debug
	build
Message-ID: <1422142044.09.0.973511381337.issue23314@psf.upfronthosting.co.za>


New submission from Steve Dower:

When we completely switch Windows builds over to VC14, we're going to encounter some new assert dialogs from the CRT. These won't appear in release builds, but because the buildbots run tests against debug builds they will get in the way. A number of tests attempt operations on bad file descriptors, which will assert and terminate in MSVCRT (I have a fix for the termination coming, but the assertions will still appear).

Currently regrtest has a "-n" flag to disable assertions using an MSVCRT function. This works fine with the exception of subprocesses, that do not inherit the setting.

The setting is process-wide, so we can't just turn assertions off around the code that may assert. We also can't turn them off completely without affecting users (e.g. #3545 and #4804).

#4804 has the discussion leading to the current state where _PyVerify_fd and the -n flag above exist. The _PyVerify_fd function could break with future CRT changes (bearing in mind that Python 3.5 and later will automatically use the latest installed CRT), for which there is an official workaround coming soon, except it won't suppress the assertion dialog for debug builds.

I'd like to propose three options:

1. Make faulthandler.enable() also disable assertions in debug builds. (Assertions are for debugging, and enabling faulthandler - at least on Windows - seems to suggest that you want to override the debugger. We also already enable faulthandler for subprocesses in the test suite.)

2. Add an environment variable (PYTHONDISABLECRTASSERT?) and use it to disable assertions on startup. (It looks like at least some tests deliberately block envvars flowing through to subprocesses, so this may require test changes too.)

3. Add a new xoption and use it to disable assertions on startup.

Obviously we could do all three, though I kind of like the first one best.

Currently, I'm planning to make debug builds available alongside the release versions, as they are necessary for people to build debuggable versions of their extensions, so this is more important than it was in the past. But the problem is very much only on the buildbots/in the test suite and not on user's machines.

Thoughts?

----------
components: Windows
messages: 234640
nosy: haypo, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Disabling CRT asserts in debug build
type: crash
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23314>
_______________________________________

From report at bugs.python.org  Sun Jan 25 12:02:16 2015
From: report at bugs.python.org (Akira Li)
Date: Sun, 25 Jan 2015 11:02:16 +0000
Subject: [New-bugs-announce] [issue23315] tempfile.mkdtemp fails with
	non-ascii paths on Python 2
Message-ID: <1422183736.06.0.60780511869.issue23315@psf.upfronthosting.co.za>


New submission from Akira Li:

Python 2.7.9 (default, Jan 25 2015, 13:41:30) 
  [GCC 4.9.2] on linux2
  Type "help", "copyright", "credits" or "license" for more information.
  >>> import os, sys, tempfile
  >>> d = u'\u20ac'.encode(sys.getfilesystemencoding()) # non-ascii
  >>> if not os.path.isdir(d): os.makedirs(d)
  ... 
  >>> os.environ['TEMP'] = d
  >>> tempfile.mkdtemp(prefix=u'')
  Traceback (most recent call last):
    File "<stdin>", line 1, in <module>
    File ".../python2.7/tempfile.py", line 331, in mkdtemp
      file = _os.path.join(dir, prefix + name + suffix)
    File ".../python2.7/posixpath.py", line 80, in join
      path += '/' + b
  UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 13: ordinal not in range(128)

Related: https://bugs.python.org/issue1681974

----------
components: Unicode
messages: 234662
nosy: akira, ezio.melotti, haypo
priority: normal
severity: normal
status: open
title: tempfile.mkdtemp fails with non-ascii paths on Python 2
type: behavior
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23315>
_______________________________________

From report at bugs.python.org  Sun Jan 25 16:46:56 2015
From: report at bugs.python.org (Joshua Landau)
Date: Sun, 25 Jan 2015 15:46:56 +0000
Subject: [New-bugs-announce] [issue23316] Incorrect evaluation order of
	function arguments with	*args
Message-ID: <1422200816.31.0.910655700205.issue23316@psf.upfronthosting.co.za>


New submission from Joshua Landau:

It is claimed that all expressions are evaluated left-to-right, including in functions?. However,

    f(*a(), b=b())

will evaluate b() before a().

? https://docs.python.org/3/reference/expressions.html#evaluation-order

----------
components: Interpreter Core
messages: 234672
nosy: Joshua.Landau
priority: normal
severity: normal
status: open
title: Incorrect evaluation order of function arguments with *args
type: behavior
versions: Python 3.4, Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23316>
_______________________________________

From report at bugs.python.org  Sun Jan 25 18:16:03 2015
From: report at bugs.python.org (Justin Eldridge)
Date: Sun, 25 Jan 2015 17:16:03 +0000
Subject: [New-bugs-announce] [issue23317] Incorrect description of
	descriptor invocation in Python	Language Reference
Message-ID: <1422206163.24.0.274216799196.issue23317@psf.upfronthosting.co.za>


New submission from Justin Eldridge:

The section titled "Invoking Descriptors" in the Python Language Reference [1]
says:

    Class Binding
        If binding to a new-style class, A.x is transformed into the call: 
        A.__dict__['x'].__get__(None, A).

This suggests that __get__ is looked up on the instance of x, when I believe
that it is actually looked up on the type. That is, it's my understanding that
A.x invokes:

    type(A.__dict__['x']).__get__(A.__dict__['x'], None, Foo))

Here's some Python 3.4 code demonstrating this:

    class A:
        pass

    class Descriptor:
        def __get__(self, obj, cls):
            print("Getting!")
            
    A.x = Descriptor()

    def replacement_get(obj, cls):
        print("This is a replacement.")
        
    A.__dict__['x'].__get__ = replacement_get

Now, writing:

    >>> A.x
    Getting!

    >>> A.__dict__['x'].__get__(None, A)
    This is a replacement!

    >>> type(A.__dict__['x']).__get__(A.__dict__['x'], None, A)
    Getting!

The documentation makes a similar statement about instance binding that also
appears to be incorrect. This is the case in all versions of the document I
could find.

What I believe to be the actual behavior is implied by a later section in the
same document, which states that the implicit invocation of special methods is
only guaranteed to work correctly if the special method is defined on the type,
not the instance.  This suggests that the statements in "Invoking Descriptors"
aren't quite correct, and while the true behavior is a little more verbose, I
think it would be worthwhile to update the documentation so as to avoid
confusion.

[1]: https://docs.python.org/2/reference/datamodel.html#invoking-descriptors

----------
assignee: docs at python
components: Documentation
messages: 234676
nosy: Justin.Eldridge, docs at python
priority: normal
severity: normal
status: open
title: Incorrect description of descriptor invocation in Python Language Reference
type: enhancement
versions: Python 2.7, Python 3.2, Python 3.3, Python 3.4, Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23317>
_______________________________________

From report at bugs.python.org  Sun Jan 25 18:31:37 2015
From: report at bugs.python.org (Dave Notman)
Date: Sun, 25 Jan 2015 17:31:37 +0000
Subject: [New-bugs-announce] [issue23318] (compiled RegEx).split gives
	unexpected results if () in	pattern
Message-ID: <1422207097.55.0.514125060003.issue23318@psf.upfronthosting.co.za>


New submission from Dave Notman:

# Python 3.3.1 (default, Sep 25 2013, 19:30:50)
# Linux 3.8.0-35-generic #50-Ubuntu SMP Tue Dec 3 01:25:33 UTC 2013 i686 i686 i686 GNU/Linux

import re

splitter = re.compile( r'(\s*[+/&;,]\s*)|(\s+and\s+)' )
ll = splitter.split( 'Dave & Sam, Jane and Zoe' )
print(repr(ll))

print( 'Try again with revised RegEx' )
splitter = re.compile( r'(?:(?:\s*[+/&;,]\s*)|(?:\s+and\s+))' )
ll = splitter.split( 'Dave & Sam, Jane and Zoe' )
print(repr(ll))

Results:
['Dave', ' & ', None, 'Sam', ', ', None, 'Jane', None, ' and ', 'Zoe']
Try again with revised RegEx
['Dave', 'Sam', 'Jane', 'Zoe']

----------
components: Regular Expressions
messages: 234677
nosy: dnotmanj, ezio.melotti, mrabarnett
priority: normal
severity: normal
status: open
title: (compiled RegEx).split gives unexpected results if () in pattern
type: behavior
versions: Python 3.3

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23318>
_______________________________________

From report at bugs.python.org  Sun Jan 25 19:53:37 2015
From: report at bugs.python.org (Matthieu Gautier)
Date: Sun, 25 Jan 2015 18:53:37 +0000
Subject: [New-bugs-announce] =?utf-8?q?=5Bissue23319=5D_Missing_SWAP=5FINT?=
	=?utf-8?b?wqBpbiBJX3NldF9zdw==?=
Message-ID: <1422212017.26.0.671058309435.issue23319@psf.upfronthosting.co.za>


New submission from Matthieu Gautier:

I_set_sw function is missing a SWAP_INT.

This leads to wrong set of bitfield value.

Here is a script to reproduce:

----------
from ctypes import *

class HEADER(BigEndianStructure):
   _fields_ = ( ('pad', c_uint32, 16),
                ('v1', c_uint32, 4),
                ('v2', c_uint32, 12)
              )

b = bytearray(4)
header = HEADER.from_buffer(b)

header.type = 1
assert b == b'\x00\x00\x10\x00'

header.mode = 0x234
assert b == b'\x00\x00\x12\x34'
----------

----------
components: ctypes
files: ctypes_swap_uint.patch
keywords: patch
messages: 234682
nosy: mgautierfr
priority: normal
severity: normal
status: open
title: Missing SWAP_INT?in I_set_sw
type: behavior
versions: Python 3.4
Added file: http://bugs.python.org/file37856/ctypes_swap_uint.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23319>
_______________________________________

From report at bugs.python.org  Sun Jan 25 22:06:33 2015
From: report at bugs.python.org (Akira Li)
Date: Sun, 25 Jan 2015 21:06:33 +0000
Subject: [New-bugs-announce] [issue23320] devguide should mention rules
	about "paragraph reflow"	in the documentation
Message-ID: <1422219993.92.0.674256323538.issue23320@psf.upfronthosting.co.za>


New submission from Akira Li:

It is suggested in https://bugs.python.org/issue23251
that only a core Python developer may reflow paragraphs
while submitting patches for the Python documentation.

It should be codified in devguide: I haven't found the
word *reflow* in it.

----------
components: Devguide
messages: 234692
nosy: akira, ezio.melotti
priority: normal
severity: normal
status: open
title: devguide should mention rules about "paragraph reflow" in the documentation
type: behavior

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23320>
_______________________________________

From report at bugs.python.org  Mon Jan 26 00:16:13 2015
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sun, 25 Jan 2015 23:16:13 +0000
Subject: [New-bugs-announce] [issue23321] Crash in str.decode() with special
	error handler
Message-ID: <1422227773.79.0.793422498408.issue23321@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka:

Debugging build crashes in some circumstances in str.decode() with error handler which produces replacement string with length larger than malformed data. For example the backslashreplace error handler produces 4-character string for every illegal byte. All other standard error handlers produce no longer than 1 character for every illegal unit.

Here is a patch which fixes this issue. I'll commit it without review because buildbots are broken without it. This issue is open for reference and post-commit review.

----------
assignee: serhiy.storchaka
components: Interpreter Core
files: unicode_decode_call_errorhandler_writer.patch
keywords: patch
messages: 234705
nosy: haypo, serhiy.storchaka
priority: normal
severity: normal
stage: patch review
status: open
title: Crash in str.decode() with special error handler
type: crash
versions: Python 3.4, Python 3.5
Added file: http://bugs.python.org/file37861/unicode_decode_call_errorhandler_writer.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23321>
_______________________________________

From report at bugs.python.org  Mon Jan 26 08:31:17 2015
From: report at bugs.python.org (Devin Jeanpierre)
Date: Mon, 26 Jan 2015 07:31:17 +0000
Subject: [New-bugs-announce] [issue23322] parser module docs missing second
	example
Message-ID: <1422257477.04.0.712021667077.issue23322@psf.upfronthosting.co.za>


New submission from Devin Jeanpierre:

The port to reST missed the second example: https://docs.python.org/release/2.5/lib/node867.html

This is still referred to in the docs, so it is not deliberate. For example, the token module docs say "The second example for the parser module shows how to use the symbol module": https://docs.python.org/3.5/library/token.html#module-token

There is no second example, nor any use of the symbol module, in the docs: https://docs.python.org/3.5/library/parser.html

----------
assignee: docs at python
components: Documentation
messages: 234716
nosy: Devin Jeanpierre, docs at python
priority: normal
severity: normal
status: open
title: parser module docs missing second example
versions: Python 2.7, Python 3.2, Python 3.3, Python 3.4, Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23322>
_______________________________________

From report at bugs.python.org  Mon Jan 26 13:41:03 2015
From: report at bugs.python.org (Alessio)
Date: Mon, 26 Jan 2015 12:41:03 +0000
Subject: [New-bugs-announce] [issue23323] Issue with imaplib and append
	messages passing a tuple	with flags
Message-ID: <1422276063.41.0.625016569735.issue23323@psf.upfronthosting.co.za>


New submission from Alessio:

In example when appending a message with more than one flag in a tuple with imaplib:

print flags
('\\Answered', '\\Seen')

connection.append('INBOX', flags, date, msg)


---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-42-832414ffe4b5> in <module>()
----> 1 connection.append('INBOX', flags, date, msg[1][0][1])

/usr/lib/python2.7/imaplib.py in append(self, mailbox, flags, date_time, message)
    326         if flags:
    327             if (flags[0],flags[-1]) != ('(',')'):
--> 328                 flags = '(%s)' % flags
    329         else:
    330             flags = None

TypeError: not all arguments converted during string formatting


When I have only one flag to append:

print flags
Out[70]: ('\\Answered',)

connection.append('INBOX', flags, date, msg)
Out[74]: ('OK', ['[APPENDUID 1 1012] APPEND completed'])

Any ideas?

Thanks

----------
components: Library (Lib)
messages: 234738
nosy: Pilessio
priority: normal
severity: normal
status: open
title: Issue with imaplib and append messages passing a tuple with flags
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23323>
_______________________________________

From report at bugs.python.org  Mon Jan 26 21:36:56 2015
From: report at bugs.python.org (Alan Yorinks)
Date: Mon, 26 Jan 2015 20:36:56 +0000
Subject: [New-bugs-announce] [issue23324] Python 3.4.2 running slow on
	Ubuntu 14.10
Message-ID: <1422304616.63.0.532936343926.issue23324@psf.upfronthosting.co.za>


New submission from Alan Yorinks:

Folks,
   I am not trying to waste anyone's time.  If this is not the correct mailing list to get this problem resolved, please point me to the correct one.

   To summarize my problem, if I run the configuration below with python 3.4.2 on Linux, the program reacts to user input (changing a potentiometer) extremely slowly. If I run the exact same code on Windows, there is no slow down. Also, if I run the exact same setup using pypy 3.2.4, there is no slowdown.

   I also tried running the software on pypy 3.2.4 on Linux and there is no slowdown. The only problem I encounter is using Python 3.4.2 on Linux. Python 2.7.8 works without issue on both Linux and Windows.

   The setup to reproduce the problem is not complicated but requires PyMata to be installed on an Arduino Uno or Leonardo attached to the computer.

Prerequisites:
    OS: Ubuntu Linux 14.10.
   Python: 3.4.2
   PyMata 2.02 or can be installed from PyPi
   Requires PySerial 2.7
   Arduino Uno or Leonardo microcontroller with StandardFirmata installed.
  
   Program to be executed: located in the PyMata/examples/digital_analog_io directory. The file is polling_digital_analog_io.py.

How the problem exhibits itself:
   When adjusting the potentiometer to set the intensity level of the LED, the intensity level greatly lags the pot adjustment. On Python 2.7 on Linux and Windows, on Python 3.4.2 on Windows, and on pypy3.2.4 on Linux, there is no lag and the program behaves as expected.

   The only variable is the Linux version of Python 3.4.2, so it is my belief that python 3.4.2 for linux has issues. I will be happy to provide any additional information to help resolve this. Again if this is the wrong mailing list, please point me to the correct one.

----------
components: Interpreter Core
messages: 234774
nosy: MrYsLab
priority: normal
severity: normal
status: open
title: Python 3.4.2 running slow on Ubuntu 14.10
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23324>
_______________________________________

From report at bugs.python.org  Mon Jan 26 21:38:30 2015
From: report at bugs.python.org (Serhiy Storchaka)
Date: Mon, 26 Jan 2015 20:38:30 +0000
Subject: [New-bugs-announce] [issue23325] Turn SIG_DFL and SIG_IGN into
	functions
Message-ID: <1422304710.21.0.213579052731.issue23325@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka:

In C the SIG_DFL and SIG_IGN macros expand into integral expressions that are not equal to an address of any function. In Python they are int objects with platform depended values. Second argument of the signal() function should be SIG_DFL, SIG_IGN, or a function. The getsignal() function returns SIG_DFL, SIG_IGN, None, or a function. They are tested for identity in signal() and getsignal() returns identical values. So actually SIG_DFL and SIG_IGN are just singletons.

I propose to turn SIG_DFL and SIG_IGN into functions. Benefits:

1. They will have names and self-descriptive representation.
2. They will have docstrings.
3. The signature of signal() will be simpler. The type of second argument would be function.
4. Their pickling will be portable.

This patch depends on the backout of turning these constants to enums (issue21076).

----------
components: Library (Lib)
files: signal_std_handlers.patch
keywords: patch
messages: 234775
nosy: ethan.furman, giampaolo.rodola, serhiy.storchaka
priority: normal
severity: normal
stage: patch review
status: open
title: Turn SIG_DFL and SIG_IGN into functions
type: enhancement
versions: Python 3.5
Added file: http://bugs.python.org/file37868/signal_std_handlers.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23325>
_______________________________________

From report at bugs.python.org  Mon Jan 26 22:34:39 2015
From: report at bugs.python.org (Serhiy Storchaka)
Date: Mon, 26 Jan 2015 21:34:39 +0000
Subject: [New-bugs-announce] [issue23326] Remove redundant __ne__
	implementations
Message-ID: <1422308079.14.0.124539148403.issue23326@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka:

As far as default __ne__ implementation delegates to __eq__, concrete __ne__ implementations are mostly redundant. They make sens when default __ne__ did not handle non-comparable types correctly, but now it is fixed. Proposed patch removes correct but redundant __ne__ implementations (incorrect implementations were removed in issue21408).

----------
components: Library (Lib)
files: remove___ne__.patch
keywords: patch
messages: 234779
nosy: serhiy.storchaka, vadmium
priority: normal
severity: normal
stage: patch review
status: open
title: Remove redundant __ne__ implementations
type: enhancement
versions: Python 3.5
Added file: http://bugs.python.org/file37869/remove___ne__.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23326>
_______________________________________

From report at bugs.python.org  Mon Jan 26 23:43:56 2015
From: report at bugs.python.org (Swapneel Ambre)
Date: Mon, 26 Jan 2015 22:43:56 +0000
Subject: [New-bugs-announce] [issue23327] zipimport to import from non-ascii
	pathname on Windows
Message-ID: <1422312236.18.0.216782202017.issue23327@psf.upfronthosting.co.za>


New submission from Swapneel Ambre:

On Windows, using zipimport module APIs like get_filename on a file with non-ascii characters in the full path fails with 

UnicodeEncodeError: 'mbcs' codec can't encode characters in position 0--1: invalid character ( Full output attached in errorlog.txt ).

The issue is that Modules/zipimport.c has a function compile_source which tries to run PyUnicode_EncodeFSDefault on the pathname. On Windows, the default encoding is 'mbcs' which cannot handle unicode characters.

This has already been fixed in the import machinery on python 3 ( see issue http://bugs.python.org/issue13758, http://bugs.python.org/issue11619). The solution is to pass the pathname as Unicode directly to the compiler.

----------
components: Unicode, Windows
files: errorlog.txt
messages: 234786
nosy: amswap, ezio.melotti, haypo, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: zipimport to import from non-ascii pathname on Windows
type: crash
versions: Python 3.4
Added file: http://bugs.python.org/file37872/errorlog.txt

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23327>
_______________________________________

From report at bugs.python.org  Tue Jan 27 08:03:55 2015
From: report at bugs.python.org (Andy Reitz)
Date: Tue, 27 Jan 2015 07:03:55 +0000
Subject: [New-bugs-announce] [issue23328] urllib2 fails for proxy
	credentials that contain a '/'	character
Message-ID: <1422342235.17.0.905041944591.issue23328@psf.upfronthosting.co.za>


New submission from Andy Reitz:

On Python 2.7.9, if I set an https_proxy environment variable, where the password contains a '/' character, urllib2 fails. Given this test code:

  import os, urllib
  os.environ['http_proxy'] = "http://someuser:a/b at 10.11.12.13:1234"
  f = urllib.urlopen('http://www.python.org')
  data = f.read()
  print data

I expect this error message (because my sample proxy is totally bogus):

[areitz at SOMEHOST ~]$ python2.7 test.py
Traceback (most recent call last):
  File "test.py", line 3, in <module>
    f = urllib.urlopen('http://www.python.org')
  File "/usr/lib64/python2.7/urllib.py", line 87, in urlopen
    return opener.open(url)
  File "/usr/lib64/python2.7/urllib.py", line 213, in open
    return getattr(self, name)(url)
  File "/usr/lib64/python2.7/urllib.py", line 350, in open_http
    h.endheaders(data)
  File "/usr/lib64/python2.7/httplib.py", line 997, in endheaders
    self._send_output(message_body)
  File "/usr/lib64/python2.7/httplib.py", line 850, in _send_output
    self.send(msg)
  File "/usr/lib64/python2.7/httplib.py", line 812, in send
    self.connect()
  File "/usr/lib64/python2.7/httplib.py", line 793, in connect
    self.timeout, self.source_address)
  File "/usr/lib64/python2.7/socket.py", line 571, in create_connection
    raise err
IOError: [Errno socket error] [Errno 101] Network is unreachable

Instead, I receive this error:

[areitz at SOMEHOST ~]$ python2.7 test.py
Traceback (most recent call last):
  File "test.py", line 3, in <module>
    f = urllib.urlopen('http://www.python.org')
  File "/usr/lib64/python2.7/urllib.py", line 87, in urlopen
    return opener.open(url)
  File "/usr/lib64/python2.7/urllib.py", line 213, in open
    return getattr(self, name)(url)
  File "/usr/lib64/python2.7/urllib.py", line 339, in open_http
    h = httplib.HTTP(host)
  File "/usr/lib64/python2.7/httplib.py", line 1107, in __init__
    self._setup(self._connection_class(host, port, strict))
  File "/usr/lib64/python2.7/httplib.py", line 712, in __init__
    (self.host, self.port) = self._get_hostport(host, port)
  File "/usr/lib64/python2.7/httplib.py", line 754, in _get_hostport
    raise InvalidURL("nonnumeric port: '%s'" % host[i+1:])
httplib.InvalidURL: nonnumeric port: 'a'

Note that from the error, it seems as if urllib2 is incorrectly parsing the password from the proxy URL. When trying this with curl 7.19.7, I see the proper behavior (the correct password is parsed from the proxy URL).

----------
components: Library (Lib)
messages: 234809
nosy: Andy.Reitz
priority: normal
severity: normal
status: open
title: urllib2 fails for proxy credentials that contain a '/' character
type: behavior
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23328>
_______________________________________

From report at bugs.python.org  Tue Jan 27 11:14:52 2015
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 27 Jan 2015 10:14:52 +0000
Subject: [New-bugs-announce] [issue23329] _ssl cannot be compiled with
	LibreSSL anymore (on	OpenBSD 5.5) because of ALPN
Message-ID: <1422353692.3.0.280216678648.issue23329@psf.upfronthosting.co.za>


New submission from STINNER Victor:

Recently, the issue #20188 "ALPN support for TLS" was fixed. The problem is that the check for the ALPN feature doesn't work with LibreSSL:

/* ALPN added in OpenSSL 1.0.2 */
#if OPENSSL_VERSION_NUMBER >= 0x1000200fL && !defined(OPENSSL_NO_TLSEXT)
# define HAVE_ALPN
#endif

On the buildbot OpenBSD 5.5 with LibreSSL, OPENSSL_VERSION_NUMBER is 2.x instead of 1.0.x. See also the issue #23177.

A workaround would be to disable the feature if LIBRESSL_VERSION_NUMBER is defined.

http://buildbot.python.org/all/builders/x86%20OpenBSD%205.5%203.x/builds/1333/steps/test/logs/stdio

 using PTY: False
running build
running build_ext
ldd: /usr/lib/libreadline.a: not an ELF executable
INFO: Can't locate Tcl/Tk libs and/or headers
building '_ssl' extension
gcc -pthread -fPIC -fno-strict-aliasing -Wsign-compare -g -O0 -Wall -Wstrict-prototypes -Werror=declaration-after-statement -I./Include -I. -IInclude -I/usr/local/include -I/home/python-builds/3.x.borja-openbsd-x86/build/Include -I/home/python-builds/3.x.borja-openbsd-x86/build -c /home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ssl.c -o build/temp.openbsd-5.6-i386-3.5-pydebug/home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ssl.o
/home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ssl.c: In function 'PySSL_selected_alpn_protocol':
/home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ssl.c:1481: warning: implicit declaration of function 'SSL_get0_alpn_selected'
/home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ssl.c: In function '_set_alpn_protocols':
/home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ssl.c:2404: warning: implicit declaration of function 'SSL_CTX_set_alpn_protos'
/home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ssl.c:2406: warning: implicit declaration of function 'SSL_CTX_set_alpn_select_cb'
gcc -pthread -shared -fPIC build/temp.openbsd-5.6-i386-3.5-pydebug/home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ssl.o -L/usr/local/lib -lssl -lcrypto -o build/lib.openbsd-5.6-i386-3.5-pydebug/_ssl.so
building '_ctypes' extension
gcc -pthread -fPIC -fno-strict-aliasing -Wsign-compare -g -O0 -Wall -Wstrict-prototypes -Werror=declaration-after-statement -Ibuild/temp.openbsd-5.6-i386-3.5-pydebug/libffi/include -Ibuild/temp.openbsd-5.6-i386-3.5-pydebug/libffi -I/home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ctypes/libffi/src -I./Include -I. -IInclude -I/usr/local/include -I/home/python-builds/3.x.borja-openbsd-x86/build/Include -I/home/python-builds/3.x.borja-openbsd-x86/build -c /home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ctypes/_ctypes.c -o build/temp.openbsd-5.6-i386-3.5-pydebug/home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ctypes/_ctypes.o -Wall -fexceptions
gcc -pthread -fPIC -fno-strict-aliasing -Wsign-compare -g -O0 -Wall -Wstrict-prototypes -Werror=declaration-after-statement -Ibuild/temp.openbsd-5.6-i386-3.5-pydebug/libffi/include -Ibuild/temp.openbsd-5.6-i386-3.5-pydebug/libffi -I/home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ctypes/libffi/src -I./Include -I. -IInclude -I/usr/local/include -I/home/python-builds/3.x.borja-openbsd-x86/build/Include -I/home/python-builds/3.x.borja-openbsd-x86/build -c /home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ctypes/callbacks.c -o build/temp.openbsd-5.6-i386-3.5-pydebug/home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ctypes/callbacks.o -Wall -fexceptions
gcc -pthread -fPIC -fno-strict-aliasing -Wsign-compare -g -O0 -Wall -Wstrict-prototypes -Werror=declaration-after-statement -Ibuild/temp.openbsd-5.6-i386-3.5-pydebug/libffi/include -Ibuild/temp.openbsd-5.6-i386-3.5-pydebug/libffi -I/home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ctypes/libffi/src -I./Include -I. -IInclude -I/usr/local/include -I/home/python-builds/3.x.borja-openbsd-x86/build/Include -I/home/python-builds/3.x.borja-openbsd-x86/build -c /home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ctypes/callproc.c -o build/temp.openbsd-5.6-i386-3.5-pydebug/home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ctypes/callproc.o -Wall -fexceptions
gcc -pthread -fPIC -fno-strict-aliasing -Wsign-compare -g -O0 -Wall -Wstrict-prototypes -Werror=declaration-after-statement -Ibuild/temp.openbsd-5.6-i386-3.5-pydebug/libffi/include -Ibuild/temp.openbsd-5.6-i386-3.5-pydebug/libffi -I/home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ctypes/libffi/src -I./Include -I. -IInclude -I/usr/local/include -I/home/python-builds/3.x.borja-openbsd-x86/build/Include -I/home/python-builds/3.x.borja-openbsd-x86/build -c /home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ctypes/stgdict.c -o build/temp.openbsd-5.6-i386-3.5-pydebug/home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ctypes/stgdict.o -Wall -fexceptions
gcc -pthread -fPIC -fno-strict-aliasing -Wsign-compare -g -O0 -Wall -Wstrict-prototypes -Werror=declaration-after-statement -Ibuild/temp.openbsd-5.6-i386-3.5-pydebug/libffi/include -Ibuild/temp.openbsd-5.6-i386-3.5-pydebug/libffi -I/home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ctypes/libffi/src -I./Include -I. -IInclude -I/usr/local/include -I/home/python-builds/3.x.borja-openbsd-x86/build/Include -I/home/python-builds/3.x.borja-openbsd-x86/build -c /home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ctypes/cfield.c -o build/temp.openbsd-5.6-i386-3.5-pydebug/home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ctypes/cfield.o -Wall -fexceptions
gcc -pthread -fPIC -fno-strict-aliasing -Wsign-compare -g -O0 -Wall -Wstrict-prototypes -Werror=declaration-after-statement -Ibuild/temp.openbsd-5.6-i386-3.5-pydebug/libffi/include -Ibuild/temp.openbsd-5.6-i386-3.5-pydebug/libffi -I/home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ctypes/libffi/src -I./Include -I. -IInclude -I/usr/local/include -I/home/python-builds/3.x.borja-openbsd-x86/build/Include -I/home/python-builds/3.x.borja-openbsd-x86/build -c /home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ctypes/libffi/src/prep_cif.c -o build/temp.openbsd-5.6-i386-3.5-pydebug/home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ctypes/libffi/src/prep_cif.o -Wall -fexceptions
gcc -pthread -fPIC -fno-strict-aliasing -Wsign-compare -g -O0 -Wall -Wstrict-prototypes -Werror=declaration-after-statement -Ibuild/temp.openbsd-5.6-i386-3.5-pydebug/libffi/include -Ibuild/temp.openbsd-5.6-i386-3.5-pydebug/libffi -I/home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ctypes/libffi/src -I./Include -I. -IInclude -I/usr/local/include -I/home/python-builds/3.x.borja-openbsd-x86/build/Include -I/home/python-builds/3.x.borja-openbsd-x86/build -c /home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ctypes/libffi/src/closures.c -o build/temp.openbsd-5.6-i386-3.5-pydebug/home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ctypes/libffi/src/closures.o -Wall -fexceptions
gcc -pthread -fPIC -fno-strict-aliasing -Wsign-compare -g -O0 -Wall -Wstrict-prototypes -Werror=declaration-after-statement -Ibuild/temp.openbsd-5.6-i386-3.5-pydebug/libffi/include -Ibuild/temp.openbsd-5.6-i386-3.5-pydebug/libffi -I/home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ctypes/libffi/src -I./Include -I. -IInclude -I/usr/local/include -I/home/python-builds/3.x.borja-openbsd-x86/build/Include -I/home/python-builds/3.x.borja-openbsd-x86/build -c /home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ctypes/libffi/src/x86/ffi.c -o build/temp.openbsd-5.6-i386-3.5-pydebug/home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ctypes/libffi/src/x86/ffi.o -Wall -fexceptions
gcc -pthread -fPIC -fno-strict-aliasing -Wsign-compare -g -O0 -Wall -Wstrict-prototypes -Werror=declaration-after-statement -Ibuild/temp.openbsd-5.6-i386-3.5-pydebug/libffi/include -Ibuild/temp.openbsd-5.6-i386-3.5-pydebug/libffi -I/home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ctypes/libffi/src -I./Include -I. -IInclude -I/usr/local/include -I/home/python-builds/3.x.borja-openbsd-x86/build/Include -I/home/python-builds/3.x.borja-openbsd-x86/build -c /home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ctypes/libffi/src/x86/freebsd.S -o build/temp.openbsd-5.6-i386-3.5-pydebug/home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ctypes/libffi/src/x86/freebsd.o -Wall -fexceptions
gcc -pthread -shared -fPIC build/temp.openbsd-5.6-i386-3.5-pydebug/home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ctypes/_ctypes.o build/temp.openbsd-5.6-i386-3.5-pydebug/home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ctypes/callbacks.o build/temp.openbsd-5.6-i386-3.5-pydebug/home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ctypes/callproc.o build/temp.openbsd-5.6-i386-3.5-pydebug/home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ctypes/stgdict.o build/temp.openbsd-5.6-i386-3.5-pydebug/home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ctypes/cfield.o build/temp.openbsd-5.6-i386-3.5-pydebug/home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ctypes/libffi/src/prep_cif.o build/temp.openbsd-5.6-i386-3.5-pydebug/home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ctypes/libffi/src/closures.o build/temp.openbsd-5.6-i386-3.5-pydebug/home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ctypes/libffi/src/x86/ffi.o build/temp.openbsd-5.6-i386-3.5-pydebug/home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ctypes/libffi/src/x86/freebsd.o -L/usr/local/lib -o build/lib.openbsd-5.6-i386-3.5-pydebug/_ctypes.so
./python:build/lib.openbsd-5.6-i386-3.5-pydebug/_ssl.so: undefined symbol 'SSL_CTX_set_alpn_protos'
./python:build/lib.openbsd-5.6-i386-3.5-pydebug/_ssl.so: undefined symbol 'SSL_get0_alpn_selected'
./python:build/lib.openbsd-5.6-i386-3.5-pydebug/_ssl.so: undefined symbol 'SSL_CTX_set_alpn_select_cb'
*** WARNING: renaming "_ssl" since importing it failed: Cannot load specified object
./python:build/lib.openbsd-5.6-i386-3.5-pydebug/_ctypes.so: undefined symbol 'ffi_call_win32'
*** WARNING: renaming "_ctypes" since importing it failed: Cannot load specified object

Python build finished successfully!
The necessary bits to build these optional modules were not found:
_tkinter              ossaudiodev           spwd               
To find the necessary bits, look in setup.py in detect_modules() for the module's name.


Following modules built successfully but were removed because they could not be imported:
_ctypes               _ssl

----------
components: Extension Modules
messages: 234814
nosy: haypo, rpointel, spil
priority: normal
severity: normal
status: open
title: _ssl cannot be compiled with LibreSSL anymore (on OpenBSD 5.5) because of ALPN
type: compile error
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23329>
_______________________________________

From report at bugs.python.org  Tue Jan 27 12:13:17 2015
From: report at bugs.python.org (Thomas Roos)
Date: Tue, 27 Jan 2015 11:13:17 +0000
Subject: [New-bugs-announce] [issue23330] h2py.py regular expression missing
Message-ID: <1422357197.95.0.836847227653.issue23330@psf.upfronthosting.co.za>


New submission from Thomas Roos:

Hi,
my issue was that SO_BINDTODEVICE symbol was not defined in /Lib/plat-linux2/IN.py which is generated by h2py.py.
This is because the regex is missing out include dirs with "-" in the name.
In my yocto cross build system this define is in asm-generic, don't know about other build systems. 
.../usr/include/asm-generic/socket.h:#define SO_BINDTODEVICE       25
so could you please change following regex in h2py.py (patch attached)
-p_include = re.compile('^[\t ]*#[\t ]*include[\t ]+<([a-zA-Z0-9_/\.]+)')
+p_include = re.compile('^[\t ]*#[\t ]*include[\t ]+<([a-zA-Z0-9_/\.-]+)')

----------
components: Cross-Build
files: IN.py.patch
keywords: patch
messages: 234817
nosy: Thomas.Roos
priority: normal
severity: normal
status: open
title: h2py.py regular expression missing
type: compile error
versions: Python 2.7
Added file: http://bugs.python.org/file37879/IN.py.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23330>
_______________________________________

From report at bugs.python.org  Tue Jan 27 14:09:19 2015
From: report at bugs.python.org (Isaac Jurado)
Date: Tue, 27 Jan 2015 13:09:19 +0000
Subject: [New-bugs-announce] [issue23331] Add non-interactive version of
	Bdb.runcall
Message-ID: <1422364159.49.0.0950037579412.issue23331@psf.upfronthosting.co.za>


New submission from Isaac Jurado:

The Bdb.runcall method shows a prompt right at the beginning of the function.  If breakpoints are defined, it is sometimes handy to skip the prompt until the next breakpoint, if any.

This use case came up in our development environment for a Django application, from where we needed to set breakpoints only for certain request, allowing other request to run without interruption.  Our solution was to wrap with a WSGI middleware, something like this:

    import sys
    from bdb import BdbQuit
    from pdb import Pdb

    class DebuggingMiddleware(object):

        def __init__(self, app):
            self.aplicacion = app
            self.debugger = Pdb()
            our_setup_breakpoints_function(self.debugger)

        def __call__(self, environ, start_response):
            environ['DEBUGGER'] = self.debugger
            frame = sys._getframe()
            self.debugger.reset()
            frame.f_trace = self.debugger.trace_dispatch
            self.debugger.botframe = frame
            self.debugger._set_stopinfo(frame, None, -1)
            sys.settrace(self.debugger.trace_dispatch)
            try:
                return self.aplicacion(environ, start_response)
            except BdbQuit:
                pass  # Return None implicitly
            finally:
                self.debugger.quitting = 1
                sys.settrace(None)

As you can see, it is basically a mix of Bdb.set_trace and Bdb.set_continue which we came up by trial and error.  If there was something like Bdb.runcall_no_prompt or an extra flag to Bdb.runcall to trigger this behaviour, this copy and paste would not be necessary.

----------
components: Library (Lib)
messages: 234819
nosy: etanol
priority: normal
severity: normal
status: open
title: Add non-interactive version of Bdb.runcall
type: enhancement
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23331>
_______________________________________

From report at bugs.python.org  Tue Jan 27 18:37:29 2015
From: report at bugs.python.org (Mirko Vogt)
Date: Tue, 27 Jan 2015 17:37:29 +0000
Subject: [New-bugs-announce] [issue23332] datetime.isoformat() -> explicitly
	mark UTC string as	such
Message-ID: <1422380249.92.0.118157289878.issue23332@psf.upfronthosting.co.za>


New submission from Mirko Vogt:

I trapped into a pitfall today with web programming using DateTime.isoformat() in the backend and Javascript on the frontend side.

isoformat() string'yfies a DateTime-object according to ISO 8601, which states that, if no timezone is specified, the string is supposed to be interpreted as UTC implicitly. isoformat() doesn't append any TZ if the object is UTC - so far so good.

However when interacting with JavaScript that leads to errors, as the ECMAScript ed 6 draft states (and most browser act like that): "date time strings without a time zone are to be treated as local, not UTC)."[1]

That is not Python's fault - however considering the issues this behaviour could cause as well as following the philosophy 'explicit is better than implicit', I'd suggest explicitly marking UTC-strings as such by adding a trailing 'Z'. According ISO 8601 that is totally fine, optional though.

[1]https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Date/parse

----------
messages: 234829
nosy: mirkovogt
priority: normal
severity: normal
status: open
title: datetime.isoformat() -> explicitly mark UTC string as such
type: behavior

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23332>
_______________________________________

From report at bugs.python.org  Tue Jan 27 23:39:33 2015
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 27 Jan 2015 22:39:33 +0000
Subject: [New-bugs-announce] [issue23333] asyncio: call
	protocol.connection_lost() when the	creation of transport failed
Message-ID: <1422398373.98.0.602533535675.issue23333@psf.upfronthosting.co.za>


New submission from STINNER Victor:

While working on the issue #23243 ("asyncio: emit ResourceWarning warnings if transports/event loops are not explicitly closed"), I saw that SelectorEventLoop._accept_connection() currently ignores errors on the creation of a transport.

When a server gets an incoming connection: it calls sock.accept(), creates a protocol, and then create the transport. It doesn't wait until the connection_made() of the protocol is called (until the transport was successfully created).

For example, for a SSL server, the client may decide to close the connection because it doesn't trust the server certificate. In this case, the SSL handshake fails at server side.

Currently, the user of the asyncio API cannot decide how to handle this failure.

I propose to call the connection_lost() method of the protocol with the exception, even if the connection_made() method of the protocol was not called (and will never be called). Attached patch implements this idea.

It's a change in the undocumented "state machine" of protocols. Before, it was not possible to switch directly to connection_lost(): there is even an assertion which ensures that it never occurs in some unit tests.

A server may log the connection failure, blacklist temporarely the client IP, etc.

Problem: Since the protocol doesn't know the transport yet, it doesn't have access to the socket, and so cannot retrieve the IP address of the client.

Maybe a new method should be added to protocols to handle this case?

How do other event loops (Twisted, eventlet, Tornado, etc.) handle failures on incoming connection?

----------
components: asyncio
files: accept_error.patch
keywords: patch
messages: 234856
nosy: gvanrossum, haypo, yselivanov
priority: normal
severity: normal
status: open
title: asyncio: call protocol.connection_lost() when the creation of transport failed
versions: Python 3.4, Python 3.5
Added file: http://bugs.python.org/file37884/accept_error.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23333>
_______________________________________

From report at bugs.python.org  Wed Jan 28 03:04:56 2015
From: report at bugs.python.org (Demian Brecht)
Date: Wed, 28 Jan 2015 02:04:56 +0000
Subject: [New-bugs-announce] [issue23334] http.client refactor
Message-ID: <1422410696.55.0.10985599454.issue23334@psf.upfronthosting.co.za>


New submission from Demian Brecht:

This is an attempt to bring a little more sanity to the http.client module through improvements to the architecture. The overarching intention of the patch is to modularize the HTTP versions, providing the following benefits:

* Make each protocol easier to work on independent of one another
* Make integrating future versions easier. This is intended as a stepping stone to integrating support for HTTP 2
* Separation of concerns between connection and application protocol

Immediate issues that this solves:
* Content-Length when a list is passed in. Currently the content length is set to the size of the list rather than the sum of the elements of the list
* Provides a little more user-friendly errors when invalid objects are passed in as header values


Note: This is still in a WIP progress state but shouldn't take much longer to get into a commit-able state. There's some work to be done on deserialization and it's entirely documentation. However, tests are passing so I figured now would be a good time to get initial feedback on the work. In hindsight, a PEP would likely have been best (it was initially intended to be put into httplib3, but I thought I might as well try it as a patch submission given it's largely backwards compatible).

----------
components: Library (Lib)
files: http_proto.patch
keywords: patch
messages: 234866
nosy: demian.brecht
priority: normal
severity: normal
status: open
title: http.client refactor
type: enhancement
versions: Python 3.6
Added file: http://bugs.python.org/file37886/http_proto.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23334>
_______________________________________

From report at bugs.python.org  Wed Jan 28 05:21:26 2015
From: report at bugs.python.org (Ned Deily)
Date: Wed, 28 Jan 2015 04:21:26 +0000
Subject: [New-bugs-announce] [issue23335] _ssl.c cannot be compiled with
	older versions of OpenSSL
Message-ID: <1422418886.09.0.55350529771.issue23335@psf.upfronthosting.co.za>


New submission from Ned Deily:

_ssl.c compilation is broken on default and 27 when building with older (pre-1.0.1 ?) versions of OpenSSL:
  
/py/dev/3x/source/Modules/_ssl.c:2296:24: error: use of undeclared identifier
      'OPENSSL_NPN_NEGOTIATED'
    if (alpn && ret != OPENSSL_NPN_NEGOTIATED)

The code added by eaa38b75cc78 (default) and 94ec4d8cf104 (2.7) doesn't account for the possibility that NPN is not available.  (The Snow Leopard buildbots are down at the moment but I would expect them to be failing with this.)

----------
messages: 234871
nosy: benjamin.peterson, ned.deily
priority: critical
severity: normal
stage: needs patch
status: open
title: _ssl.c cannot be compiled with older versions of OpenSSL
versions: Python 2.7, Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23335>
_______________________________________

From report at bugs.python.org  Wed Jan 28 07:02:27 2015
From: report at bugs.python.org (cindykrafft)
Date: Wed, 28 Jan 2015 06:02:27 +0000
Subject: [New-bugs-announce] [issue23336] Thread.LockType is misnamed
Message-ID: <1422424947.14.0.409463431872.issue23336@psf.upfronthosting.co.za>


New submission from cindykrafft:

The tp_name field in thread.LockType is set as "thread.lock". This is incorrect and conflicts with the value set in dummy_thread.

Python 2.7.6 (default, Sep  9 2014, 15:04:36) 
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.39)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import thread
>>> import dummy_thread
>>> thread.LockType.__name__
'lock'
>>> dummy_thread.LockType.__name__
'LockType'

Code which depends on the ability to look up a class based on module and name (see the following code taken from pickle.py) breaks due to this behavior, so my preferred fix would be to change tp_name to a value consistent with dummy_thread.

try:
    __import__(module)
    mod = sys.modules[module]
    klass = getattr(mod, name)
except (ImportError, KeyError, AttributeError):
    raise PicklingError(
        "Can't pickle %r: it's not found as %s.%s" %
         (obj, module, name))

Happy to submit a patch if someone could confirm.

----------
components: Library (Lib)
messages: 234873
nosy: cindykrafft
priority: normal
severity: normal
status: open
title: Thread.LockType is misnamed
type: behavior
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23336>
_______________________________________

From report at bugs.python.org  Wed Jan 28 11:24:42 2015
From: report at bugs.python.org (Marc)
Date: Wed, 28 Jan 2015 10:24:42 +0000
Subject: [New-bugs-announce] [issue23337] Run python with restricted rights
Message-ID: <1422440682.84.0.195958964107.issue23337@psf.upfronthosting.co.za>


New submission from Marc:

Hi,

We work in a school within a domain and pupils are using different restricted account on this domain. We tried to install Python 3.4 and it's work with an Administrator Account.


With Children account , we got the message "IDLE's subprocess didn't make connection. Either IDLE can't start a subprocess or personal firewall software is blocking the connection"  .


We checked if there is any python file in the  python.exe directory like advised on http://stackoverflow.com/questions/874757/python-idle-subprocess-error.

We also tried to give more right on the python directory for the children account.

If anyone have a solution please let me know.

Thanks

----------
components: Windows
files: python_error.png
messages: 234883
nosy: marcd, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Run python with restricted rights
type: crash
versions: Python 3.4
Added file: http://bugs.python.org/file37889/python_error.png

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23337>
_______________________________________

From report at bugs.python.org  Wed Jan 28 11:52:51 2015
From: report at bugs.python.org (Makoto Kato)
Date: Wed, 28 Jan 2015 10:52:51 +0000
Subject: [New-bugs-announce] [issue23338] PyErr_Format in ctypes uses
	invalid parameter
Message-ID: <1422442371.36.0.968367348117.issue23338@psf.upfronthosting.co.za>


New submission from Makoto Kato:

This is CYGWIN build only.

When ctypes cannot find function in CDataType_in_dll, it uses PyErr_Format.  But it passes invalid parameters.

----------
components: ctypes
messages: 234886
nosy: mkato
priority: normal
severity: normal
status: open
title: PyErr_Format in ctypes uses invalid parameter
type: crash
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23338>
_______________________________________

From report at bugs.python.org  Wed Jan 28 11:59:47 2015
From: report at bugs.python.org (ThiefMaster)
Date: Wed, 28 Jan 2015 10:59:47 +0000
Subject: [New-bugs-announce] [issue23339] dict_values should be comparable
	with a set
Message-ID: <1422442787.53.0.167316115622.issue23339@psf.upfronthosting.co.za>


New submission from ThiefMaster:

>>> d = {'1': '2'}
>>> {'1'} < d.keys()
False
>>> {'1'} < set(d.values())
False
>>> {'1'} < d.values()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unorderable types: set() < dict_values()


Same for e.g. the `-` operator.

Since dict_keys acts like a real set I think dict_values should do so, too. At least if all the values are hashable.

----------
components: Library (Lib)
messages: 234888
nosy: ThiefMaster
priority: normal
severity: normal
status: open
title: dict_values should be comparable with a set
versions: Python 2.7, Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23339>
_______________________________________

From report at bugs.python.org  Wed Jan 28 16:49:05 2015
From: report at bugs.python.org (Alex Potapenko)
Date: Wed, 28 Jan 2015 15:49:05 +0000
Subject: [New-bugs-announce] [issue23340] armv7l C++ exceptions issue
Message-ID: <1422460145.75.0.667594511432.issue23340@psf.upfronthosting.co.za>


New submission from Alex Potapenko:

I run Python on an arm-brcm-linux-uclibcgnueabi router. Python was cross-compiled using hndtools-arm-linux-2.6.36-uclibc-4.5.3 toolchain. While trying to use deluge, I realised that there's something wrong with handling C++ exceptions in C++ extension modules: they're not being caught at all. I tested whether C++ exception handling works on my system in general, and concluded it does work fine. I then wrote a simple C++ extension module with a try-catch block in the init function, that has a "throw 1" in the try section and a "catch (...)" section (see module.cpp), and I got "terminate called after throwing an instance of 'int'" when trying to load the module. Tested this with Python 2.7.9 and 3.4.2, however I had similar issues with other versions, such as 2.7.3 and 2.6.9.

I'm not sure whether this is a Python issue, or a specific build issue, but I'm stuck here and any help is greatly appreciated!

----------
components: Extension Modules
files: module.cpp
messages: 234899
nosy: alllexx
priority: normal
severity: normal
status: open
title: armv7l C++ exceptions issue
type: behavior
versions: Python 2.7
Added file: http://bugs.python.org/file37893/module.cpp

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23340>
_______________________________________

From report at bugs.python.org  Wed Jan 28 20:26:05 2015
From: report at bugs.python.org (Dan LaMotte)
Date: Wed, 28 Jan 2015 19:26:05 +0000
Subject: [New-bugs-announce] [issue23341] Issue parsing valid cookie
Message-ID: <1422473165.15.0.717728203911.issue23341@psf.upfronthosting.co.za>


New submission from Dan LaMotte:

I recently discovered that a valid cookie (by the RFC) is not parse-able by the Cookie library in python's standard library.

  import Cookie
  c = Cookie.SimpleCookie('key=[ab]cd[ef]')
  print c.keys() # yields []

When quoted, it works fine:

  c = Cookie.SimpleCookie('key="[ab]cd[ef]"')
  print c.keys() # yields ['key']

I noticed the issue after upgrading to Python 2.7.9 (was previously at 2.7.2).  The issue cropped up in our internal Django Web site when another internal site used a cookie in a similar format to the above and due to the sort order of the cookies, it appeared before the sessionid cookie we use with Django.  Effectively, parsing of the cookie header stops and the sessionid is never read which ... to Django ... means you are not logged in.  So, attempt to login, no errors, redirect to new page after successful login and you still appear not logged in.

References:

cookie-value in http://tools.ietf.org/html/rfc6265#section-4.1
token in http://tools.ietf.org/html/rfc2616#section-2.2

cookie-pair = cookie-name "=" cookie-value
cookie-name = token
...

The code correctly disallows brackets [ and ] in cookie-name's, but ends up disallowing them in cookie-value's as well which is not RFC Compliant.

We noticed this issue in Chrome but not Firefox.  Our guess is that Firefox quotes its cookie-values which the code handles just fine.

----------
messages: 234908
nosy: dlamotte
priority: normal
severity: normal
status: open
title: Issue parsing valid cookie
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23341>
_______________________________________

From report at bugs.python.org  Wed Jan 28 23:13:38 2015
From: report at bugs.python.org (Thomas Kluyver)
Date: Wed, 28 Jan 2015 22:13:38 +0000
Subject: [New-bugs-announce] [issue23342] run() - unified high-level
	interface for subprocess
Message-ID: <1422483218.99.0.771753896574.issue23342@psf.upfronthosting.co.za>


New submission from Thomas Kluyver:

This follows on from the python-ideas thread starting here: https://mail.python.org/pipermail/python-ideas/2015-January/031479.html

subprocess gains:

- A CompletedProcess class representing a process that has finished, with attributes args, returncode, stdout and stderr
- A run() function which runs a process to completion and returns a CompletedProcess instance, aiming to unify the functionality of call, check_call and check_output
- CalledProcessError and TimeoutExceeded now have a stderr attribute, to avoid throwing away potentially relevant information.

Things I'm not sure about:

1. Should run() capture stdout/stderr by default? I opted not to, for consistency with Popen and with shells.
2. I gave run() a check_returncode parameter, but it feels quite a long name for a parameter. Is 'check' clear enough to use as the parameter name?
3. Popen has an 'args' attribute, while CalledProcessError and TimeoutExpired have 'cmd'. CompletedProcess sits between those cases, so which name should it use? For now, it's args.

----------
components: Library (Lib)
files: subprocess_run.patch
keywords: patch
messages: 234918
nosy: takluyver
priority: normal
severity: normal
status: open
title: run() - unified high-level interface for subprocess
type: enhancement
versions: Python 3.5
Added file: http://bugs.python.org/file37897/subprocess_run.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23342>
_______________________________________

From report at bugs.python.org  Wed Jan 28 23:20:44 2015
From: report at bugs.python.org (Hobs)
Date: Wed, 28 Jan 2015 22:20:44 +0000
Subject: [New-bugs-announce] [issue23343] operator precedence table for `not
	x` has an operand, while the others do not
Message-ID: <1422483644.76.0.913289618164.issue23343@psf.upfronthosting.co.za>


New submission from Hobs:

Shouldn't the [operator precedence table](https://docs.python.org/3.4/reference/expressions.html#operator-precedence), 5th row, 1st column, just say "`not`" rather than "`not` x"? The other rows are identified by the keyword for the operator and don't include any operand(s). Is there some other expression that includes `not` that should have a different position in the precedence table?

----------
assignee: docs at python
components: Documentation
messages: 234919
nosy: Hobson.Lane, docs at python
priority: normal
severity: normal
status: open
title: operator precedence table for `not x` has an operand, while the others do not
type: enhancement
versions: Python 2.7, Python 3.2, Python 3.3, Python 3.4, Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23343>
_______________________________________

From report at bugs.python.org  Wed Jan 28 23:22:06 2015
From: report at bugs.python.org (Serhiy Storchaka)
Date: Wed, 28 Jan 2015 22:22:06 +0000
Subject: [New-bugs-announce] [issue23344] Faster marshalling
Message-ID: <1422483726.85.0.28053556963.issue23344@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka:

Currently writing marshalled data to buffer is not very efficient. Data is written byte by byte with testing conditions p->fp != NULL and p->ptr != p->end for every byte. Proposed patch makes writing to buffer faster.

Benchmark results:

$ ./python -m timeit -s "import marshal; d = compile(open('Lib/_pydecimal.py').read(), '_pydecimal.py', 'exec')" -- "marshal.dumps(d)"
Unpatched: 100 loops, best of 3: 4.64 msec per loop
Patched: 100 loops, best of 3: 3.39 msec per loop

$ ./python -m timeit -s "import marshal; a = ['%010x' % i for i in range(10**4)]" -- "marshal.dumps(a)"
Unpatched: 1000 loops, best of 3: 1.96 msec per loop
Patched: 1000 loops, best of 3: 1.32 msec per loop

$ ./python -m timeit -s "import marshal; a = ['%0100x' % i for i in range(10**4)]" -- "marshal.dumps(a)"
Unpatched: 100 loops, best of 3: 10.3 msec per loop
Patched: 100 loops, best of 3: 3.39 msec per loop

----------
components: Interpreter Core
files: marshal_faster_write.patch
keywords: patch
messages: 234920
nosy: serhiy.storchaka
priority: normal
severity: normal
stage: patch review
status: open
title: Faster marshalling
type: performance
versions: Python 3.5
Added file: http://bugs.python.org/file37898/marshal_faster_write.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23344>
_______________________________________

From report at bugs.python.org  Thu Jan 29 03:04:12 2015
From: report at bugs.python.org (Ned Deily)
Date: Thu, 29 Jan 2015 02:04:12 +0000
Subject: [New-bugs-announce] [issue23345] test_ssl fails on OS X 10.10.2
	with latest patch level	of OpenSSL libs
Message-ID: <1422497052.81.0.264375405718.issue23345@psf.upfronthosting.co.za>


New submission from Ned Deily:

With the latest maintenance release of OS X 10.10 (10.10.2), the OpenSSL libs have reached a patch level that fails the sanity test in test_ssl:

test_ssl: testing with 'OpenSSL 0.9.8zc 15 Oct 2014' (0, 9, 8, 28, 15)
          under Mac ('10.10.2', ('', '', ''), 'x86_64')
          HAS_SNI = True
          OP_ALL = 0x     7ff
[...]
======================================================================
FAIL: test_openssl_version (test.test_ssl.BasicSocketTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/py/dev/3x/source/Lib/test/test_ssl.py", line 309, in test_openssl_version
    self.assertLessEqual(patch, 26)
AssertionError: 28 not less than or equal to 26

Is there anything special about 26 or can the value just be bumped?

----------
messages: 234935
nosy: ned.deily, pitrou
priority: critical
severity: normal
stage: needs patch
status: open
title: test_ssl fails on OS X 10.10.2 with latest patch level of OpenSSL libs
versions: Python 2.7, Python 3.4, Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23345>
_______________________________________

From report at bugs.python.org  Thu Jan 29 05:52:53 2015
From: report at bugs.python.org (Denis Sukhonin)
Date: Thu, 29 Jan 2015 04:52:53 +0000
Subject: [New-bugs-announce] [issue23346] shutil.rmtree doesn't work
	correctly on FreeBSD.
Message-ID: <1422507173.88.0.069796427202.issue23346@psf.upfronthosting.co.za>


New submission from Denis Sukhonin:

shutil.rmtree doesn't work correctly on FreeBSD 9.1.

For example if I create a path /tmp/test and try to remove it, I get an exception:

>>> shutil.rmtree('/tmp/test')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.4/shutil.py", line 463, in rmtree
    _rmtree_safe_fd(fd, path, onerror)
  File "/usr/local/lib/python3.4/shutil.py", line 385, in _rmtree_safe_fd
    onerror(os.listdir, path, sys.exc_info())
  File "/usr/local/lib/python3.4/shutil.py", line 382, in _rmtree_safe_fd
    names = os.listdir(topfd)
OSError: [Errno 22] Invalid argument: '/tmp/test'

---

shutil._use_fd_functions has value True. When I change it to False, the shutil.rmtree works perfecty.

Version info:
>>> print(sys.version)
3.4.2 (default, Dec 22 2014, 21:56:20) 
[GCC 4.2.1 20070831 patched [FreeBSD]]
>>> print(sys.platform)
freebsd9

$ uname -r
9.1-RELEASE

----------
components: Library (Lib)
messages: 234946
nosy: negval
priority: normal
severity: normal
status: open
title: shutil.rmtree doesn't work correctly on FreeBSD.
type: behavior
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23346>
_______________________________________

From report at bugs.python.org  Thu Jan 29 12:56:49 2015
From: report at bugs.python.org (STINNER Victor)
Date: Thu, 29 Jan 2015 11:56:49 +0000
Subject: [New-bugs-announce] [issue23347] asyncio: fix and refactor creation
	of subprocess	transports
Message-ID: <1422532609.81.0.0177270416952.issue23347@psf.upfronthosting.co.za>


New submission from STINNER Victor:

Attached patch fix and refactors the creation of asyncio subprocess transports. It fixes the code handling errors.

Changes:

* Add a wait() method to wait until the child process exit

* The constructor now accepts an optional waiter parameter. The _post_init() coroutine must not be called explicitly anymore. It makes subprocess transports closer to other transports, and it gives more freedom if we want later to change completly how subprocess transports are created.

* close() now kills the process instead of kindly terminate it: the child process may ignore SIGTERM and continue to run. Call explicitly terminate() and wait() if you want to kindly terminate the child process.

* close() now logs a warning in debug mode if the process is still running and needs to be killed

* _make_subprocess_transport() is now fully asynchronous again: if the creation of the transport failed, wait asynchronously for the process eixt. Before the wait was synchronous. This change requires close() to *kill*, and not terminate, the child process.

* Remove the _kill_wait() method, replaced with a more agressive close() method. It fixes _make_subprocess_transport() on error. BaseSubprocessTransport.close() calls the close() method of pipe transports, whereas _kill_wait() closed directly pipes of the subprocess.Popen object without unregistering file descriptors from the selector (which caused severe bugs).

* These changes make subprocess.py much simpler!

wait() is a coroutine, which is something uncommon. Is it ok to start using coroutines in transports, at least for subprocess transports?

----------
components: asyncio
files: subprocess_wait.patch
keywords: patch
messages: 234963
nosy: gvanrossum, haypo, yselivanov
priority: normal
severity: normal
status: open
title: asyncio: fix and refactor creation of subprocess transports
versions: Python 3.4, Python 3.5
Added file: http://bugs.python.org/file37906/subprocess_wait.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23347>
_______________________________________

From report at bugs.python.org  Thu Jan 29 15:42:00 2015
From: report at bugs.python.org (Guillaume)
Date: Thu, 29 Jan 2015 14:42:00 +0000
Subject: [New-bugs-announce] [issue23348] distutils.LooseVersion fails to
	compare two valid	versions
Message-ID: <1422542520.54.0.739308889883.issue23348@psf.upfronthosting.co.za>


New submission from Guillaume:

$ python2.7 -c "from distutils.version import LooseVersion as V; print V('3.16.0-0.bpo.4-amd64') > V('3.16-0.bpo.2-amd64')"
False
$ python3.4 -c "from distutils.version import LooseVersion as V; print(V('3.16.0-0.bpo.4-amd64') > V('3.16-0.bpo.2-amd64'))" 
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python3.4/distutils/version.py", line 70, in __gt__
    c = self._cmp(other)
  File "/usr/lib/python3.4/distutils/version.py", line 343, in _cmp
    if self.version < other.version:
TypeError: unorderable types: int() < str()

Same thing with python3.2 et python3.3.

I find this on Debian. They recently change the numerotation of backported kernels.

----------
components: Distutils
messages: 234975
nosy: dstufft, eric.araujo, maethor
priority: normal
severity: normal
status: open
title: distutils.LooseVersion fails to compare two valid versions
type: behavior
versions: Python 3.2, Python 3.3, Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23348>
_______________________________________

From report at bugs.python.org  Fri Jan 30 02:07:08 2015
From: report at bugs.python.org (Richard Hansen)
Date: Fri, 30 Jan 2015 01:07:08 +0000
Subject: [New-bugs-announce] [issue23349] memoryview.to_bytes() and
	PyBuffer_ToContiguous()	off-by-one error for non-contiguous buffers
Message-ID: <1422580028.66.0.282678068687.issue23349@psf.upfronthosting.co.za>


New submission from Richard Hansen:

PyBuffer_ToContiguous() has an off-by-one error when copying a buffer it thinks is non-contiguous.

To reproduce, put the following in foo.pyx and compile with Cython v0.21.2:

    cpdef foo():
        cdef unsigned char[:] v = bytearray("testing")
        print repr(memoryview(v).tobytes())

>>> import foo
>>> foo.foo()
'estingt'

(This issue was fixed for Python 3.x in issue #12834 but it was not fixed for Python 2.7.)

----------
components: Interpreter Core
files: PyBuffer_ToContiguous.patch
keywords: patch
messages: 235011
nosy: rhansen
priority: normal
severity: normal
status: open
title: memoryview.to_bytes() and PyBuffer_ToContiguous() off-by-one error for non-contiguous buffers
type: behavior
versions: Python 2.7
Added file: http://bugs.python.org/file37912/PyBuffer_ToContiguous.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23349>
_______________________________________

From report at bugs.python.org  Fri Jan 30 02:12:42 2015
From: report at bugs.python.org (Demian Brecht)
Date: Fri, 30 Jan 2015 01:12:42 +0000
Subject: [New-bugs-announce] [issue23350] Content-length is incorrect when
	request body is a list	or tuple
Message-ID: <1422580362.11.0.520529959905.issue23350@psf.upfronthosting.co.za>


New submission from Demian Brecht:

Rather than summing the value of each element of a list or tuple to use as the value of the content-length header, the length of the list or tuple is used.

----------
files: list_content_length.patch
keywords: patch
messages: 235012
nosy: demian.brecht
priority: normal
severity: normal
status: open
title: Content-length is incorrect when request body is a list or tuple
Added file: http://bugs.python.org/file37913/list_content_length.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23350>
_______________________________________

From report at bugs.python.org  Fri Jan 30 02:28:12 2015
From: report at bugs.python.org (Piotr Jurkiewicz)
Date: Fri, 30 Jan 2015 01:28:12 +0000
Subject: [New-bugs-announce] [issue23351] socket.settimeout(5.0) does not
	have any effect
Message-ID: <1422581292.34.0.901742947923.issue23351@psf.upfronthosting.co.za>


New submission from Piotr Jurkiewicz:

After setting socket.settimeout(5.0), socket.send() returns immediately, instead of returning after specified timeout.

Steps to reproduce:

Open two python interpreters.

In the first one (the receiver) execute:

>>> import socket
>>> r = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
>>> r.bind("test.sock")

In the second one (the sender) execute:

>>> import socket
>>> s = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)

Then run the following command 11 times:

>>> s.sendto("msg", "test.sock")

On the 12 run command will block. This happens because datagram sockets queue on Linux is 11 messages long. Interrupt the command.

So far so good.

Then set sender socket timeout:

>>> s.settimeout(5.0)

Expected behavior:

s.sendto() should block for a 5 seconds and THEN raise error 11 (EAGAIN/EWOULDBLOCK).

Actual behavior:

s.sendto() raises the error IMMEDIATELY.

>>> s.sendto("msg", "test.sock")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
socket.error: [Errno 11] Resource temporarily unavailable

So, in fact, s.settimeout(5.0) does not have any effect.

I think that problem is that settimeout() sets the socket to the non-blocking mode (docs say: "Timeout mode internally sets the socket in non-blocking mode.").

As described [here](http://stackoverflow.com/q/13556972/2178047) setting timeout on non-blocking sockets is impossible.

In fact, when I set timeout manually with setsockopt(), everything works as expected:

>>> s.setblocking(1)              #go back to blocking mode
>>> tv = struct.pack("ll", 5, 0)
>>> s.setsockopt(socket.SOL_SOCKET, socket.SO_SNDTIMEO, tv)

Now s.sendto() raises the error after 5 seconds, as expected.

----------
components: IO, Library (Lib)
messages: 235013
nosy: piotrjurkiewicz
priority: normal
severity: normal
status: open
title: socket.settimeout(5.0) does not have any effect
type: behavior
versions: Python 2.7, Python 3.2, Python 3.3, Python 3.4, Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23351>
_______________________________________

From report at bugs.python.org  Fri Jan 30 02:33:45 2015
From: report at bugs.python.org (Richard Hansen)
Date: Fri, 30 Jan 2015 01:33:45 +0000
Subject: [New-bugs-announce] [issue23352] PyBuffer_IsContiguous() sometimes
	returns wrong result	if suboffsets != NULL
Message-ID: <1422581625.15.0.953074491863.issue23352@psf.upfronthosting.co.za>


New submission from Richard Hansen:

According to https://docs.python.org/2/c-api/buffer.html#the-new-style-py-buffer-struct if the suboffsets member of Py_buffer is non-NULL and all members of the array are negative, the buffer may be contiguous.

PyBuffer_IsContiguous() does not behave this way.  It assumes that if the suboffsets member is non-NULL then at least one member of the array is non-negative, and thus assumes that the buffer is non-contiguous.  This is not always correct.

One consequence of this bug is PyBuffer_ToContiguous() (and thus memoryview.tobytes()) falls back to slow (and currently buggy, see issue #23349) memory copying code.

Attached is a patch that fixes this bug.  The patch is against 2.7 but can be trivially modified to apply to 3.x.

----------
components: Interpreter Core
files: PyBuffer_IsContiguous.patch
keywords: patch
messages: 235014
nosy: rhansen
priority: normal
severity: normal
status: open
title: PyBuffer_IsContiguous() sometimes returns wrong result if suboffsets != NULL
versions: Python 2.7, Python 3.2, Python 3.3, Python 3.4, Python 3.5, Python 3.6
Added file: http://bugs.python.org/file37914/PyBuffer_IsContiguous.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23352>
_______________________________________

From report at bugs.python.org  Fri Jan 30 15:08:56 2015
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 30 Jan 2015 14:08:56 +0000
Subject: [New-bugs-announce] [issue23353] gnerator bug with exception:
	tstate->exc_value is not	cleared after an except block
Message-ID: <1422626936.33.0.830645939586.issue23353@psf.upfronthosting.co.za>


New submission from STINNER Victor:

Since my changeset a5efd5021ca1, the Python test suite starts to fail randomly. Running test_asyncio modifies sys.exc_info(): it is not (None, None, None) after the execution of test_asyncio. The problem comes from test_cancel_make_subprocess_transport_exec() of Lib/test/test_asyncio/test_subprocess.py.

I tried to write a simple script which does not depend on Python to reproduce the issue. See attached excinfo_bug2.py script.

Output:
---
exc_info after except (<class '__main__.MyException'>, MyException(), <traceback object at 0x7f09201ad7c8>)
exc_info at exit (<class '__main__.MyException'>, MyException(), <traceback object at 0x7f09201abd08>)
---

exc_info is supposed to be (None, None, None), at least at exit.

I will try to write an even simpler script to identify the bug.

----------
files: excinfo_bug2.py
messages: 235033
nosy: haypo
priority: normal
severity: normal
status: open
title: gnerator bug with exception: tstate->exc_value is not cleared after an except block
versions: Python 3.4, Python 3.5
Added file: http://bugs.python.org/file37922/excinfo_bug2.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23353>
_______________________________________

From report at bugs.python.org  Sat Jan 31 04:40:41 2015
From: report at bugs.python.org (SoniEx2)
Date: Sat, 31 Jan 2015 03:40:41 +0000
Subject: [New-bugs-announce] [issue23354] Loading 2 GiLOC file which raises
	exception causes wrong	traceback
Message-ID: <1422675641.92.0.509373230042.issue23354@psf.upfronthosting.co.za>


New submission from SoniEx2:

I loaded a file with 2 GiLOC followed by "assert False" and this was the error/traceback:

Traceback (most recent call last):
  File "test.py", line -2147483647, in <module>
AssertionError

----------
components: Interpreter Core
messages: 235079
nosy: SoniEx2
priority: normal
severity: normal
status: open
title: Loading 2 GiLOC file which raises exception causes wrong traceback
type: behavior
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23354>
_______________________________________

From report at bugs.python.org  Sat Jan 31 06:25:02 2015
From: report at bugs.python.org (kai keliikuli)
Date: Sat, 31 Jan 2015 05:25:02 +0000
Subject: [New-bugs-announce] [issue23355] rsplit duplicates split behavior
Message-ID: <1422681902.98.0.718785741175.issue23355@psf.upfronthosting.co.za>


New submission from kai keliikuli:

'a b'.split() == 'a b'.rsplit()

----------
messages: 235080
nosy: kai.keliikuli
priority: normal
severity: normal
status: open
title: rsplit duplicates split behavior
type: behavior
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23355>
_______________________________________

From report at bugs.python.org  Sat Jan 31 09:40:39 2015
From: report at bugs.python.org (py.user)
Date: Sat, 31 Jan 2015 08:40:39 +0000
Subject: [New-bugs-announce] [issue23356] In argparse docs simplify example
	about argline
Message-ID: <1422693639.9.0.487352753219.issue23356@psf.upfronthosting.co.za>


New submission from py.user:

The example is:

def convert_arg_line_to_args(self, arg_line):
    for arg in arg_line.split():
        if not arg.strip():
            continue
        yield arg

str.split() with default delimiters never returns empty or whitespace strings in the list.

>>> '  x  x  '.split()
['x', 'x']
>>> '  '.split()
[]
>>>

Therefore, the if condition doesn't ever continue the loop.
It can be written:

def convert_arg_line_to_args(self, arg_line):
    for arg in arg_line.split():
        yield arg

It's the same as:

def convert_arg_line_to_args(self, arg_line):
    return iter(arg_line.split())

I guess, nothing uses next() for the result:

def convert_arg_line_to_args(self, arg_line):
    return arg_line.split()

Applied a patch with the last variant.

----------
assignee: docs at python
components: Documentation, Library (Lib)
files: args_ex_argline.diff
keywords: patch
messages: 235089
nosy: docs at python, py.user
priority: normal
severity: normal
status: open
title: In argparse docs simplify example about argline
type: performance
versions: Python 2.7, Python 3.4
Added file: http://bugs.python.org/file37934/args_ex_argline.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23356>
_______________________________________

From report at bugs.python.org  Sat Jan 31 10:50:15 2015
From: report at bugs.python.org (=?utf-8?q?Ra=C3=BAl_Cumplido?=)
Date: Sat, 31 Jan 2015 09:50:15 +0000
Subject: [New-bugs-announce] [issue23357] pyvenv help shows incorrect usage
Message-ID: <1422697815.11.0.353784279673.issue23357@psf.upfronthosting.co.za>


New submission from Ra?l Cumplido:

Currently when you execute pyvenv the help message is wrong:

host@~  $ pyvenv
usage: venv [-h] [--system-site-packages] [--symlinks | --copies] [--clear]
            [--upgrade] [--without-pip]
            ENV_DIR [ENV_DIR ...]
venv: error: the following arguments are required: ENV_DIR

Based on the docs on https://docs.python.org/3/library/venv.html it should be:

host@~  $ pyvenv
usage: pyvenv [-h] [--system-site-packages] [--symlinks | --copies] [--clear]
              [--upgrade] [--without-pip]
              ENV_DIR [ENV_DIR ...]
pyvenv: error: the following arguments are required: ENV_DIR

Basically instead of saying usage venv it should say usage pyvenv but if you execute the module it should be venv:

host@~  $ python3.5 -m venv
usage: venv [-h] [--system-site-packages] [--symlinks | --copies] [--clear]
            [--upgrade] [--without-pip]
            ENV_DIR [ENV_DIR ...]
venv: error: the following arguments are required: ENV_DIR

----------
components: Library (Lib)
messages: 235094
nosy: raulcd
priority: normal
severity: normal
status: open
title: pyvenv help shows incorrect usage
type: enhancement
versions: Python 3.4, Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23357>
_______________________________________

From report at bugs.python.org  Sat Jan 31 12:42:02 2015
From: report at bugs.python.org (Martin Panter)
Date: Sat, 31 Jan 2015 11:42:02 +0000
Subject: [New-bugs-announce] [issue23358] BaseServer missing from
	socketserver.__all__
Message-ID: <1422704522.18.0.713790375362.issue23358@psf.upfronthosting.co.za>


New submission from Martin Panter:

This patch adds BaseServer to __all__. It also adds a test case that should automatically fail if someone defines a new function or class in the ?socketserver? module and does not update __all__.

----------
components: Library (Lib)
files: socketserver-all.patch
keywords: patch
messages: 235102
nosy: vadmium
priority: normal
severity: normal
status: open
title: BaseServer missing from socketserver.__all__
type: behavior
versions: Python 3.5
Added file: http://bugs.python.org/file37938/socketserver-all.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23358>
_______________________________________

From report at bugs.python.org  Sat Jan 31 22:37:58 2015
From: report at bugs.python.org (Raymond Hettinger)
Date: Sat, 31 Jan 2015 21:37:58 +0000
Subject: [New-bugs-announce] [issue23359] Speed-up set_lookkey()
Message-ID: <1422740278.04.0.89058521092.issue23359@psf.upfronthosting.co.za>


New submission from Raymond Hettinger:

This patch applies three techniques to tighten-up the generated code for the lookkey() for sets.

I'm not sure I want to do this because it expands the size of the code quite a bit (much like the previously existing lookkey specializations did).  As such, it offends my sense of beauty and neatness.

That said, it greatly tightens the inner search loop and in the handful of timings I've done so far, it provides a good speed-up for smaller sets (ones that fit in L1 and L2 cache) and minor improvements for bigger sets (where running time is dominated by cost of fetching a cache-line with a key/hash pair).

The three transformations are:

1.)  Make two variants of lookkey, one with an early-out for a dummy entry and one that treats a dummy entry as an active entry.   This eliminates the freeslot tracking in both cases.  

In the first case (early-out), it frees one register (the one for the freeslot), it simplifies the inner-loop test (no need to see if freeslot is NULL), it provides an early-out (return on dummy rather than continuing to search), and it simplifies the return code (just return entry, no need to check the freeslot).

In the second case (dummies treated as active), it frees two registers (one for freeslot and the other for &dummy) and it removes the dummy/freeslot inner-loop test entirely.

That eliminated inner inner-loop code used to look like this in gcc-4.9:

    cmpq    %r15, %rbp      entry->key != dummny
    jne L375                                   |
    testq   %r13, %r13      freelslot == NULL  |
    cmove   %rbx, %r13                         |
L375:                                      <---|

And the eliminated inner loop code was even worse with clang:

    leaq    __dummy_struct(%rip), %rax
    cmpq    %rax, %r14                entry->key==dummy
    sete    %al
    testq   %rbx, %rbx                freeslot==NULL
    sete    %cl
    testb   %cl, %al
    cmovneq %r13, %rbx                freeslot ?= entry


2.) Avoid the &mask step for computing indices when not needed. Using the same technique that is used in set_insert_clean, we test to see if there is a possible wrap-around during the linear probes.  If not, we can skip the lengthy address calculation for table[(i+j)&mask] and instead use entry++.

This saves the sequentially dependent steps of add j, the bitwise-and with the mask, a shift left by four to scale the index, and the add to the table start address.  It replaces those steps with a single entry++ which codes as an add $16.

3).  In the linear_probe inner-loop for the ignore dummy case, replace (entry->key==NULL) with (entry->hash==0 && entry->key==NULL).   While that looks like a step backward adding an extra test, that second test is essentially free (a register compare and branch which is nearly 100% predictable).   The benefit is that the inner-loop only needs to fetch the entry->hash value and doesn't need to fetch entry->key.   This doesn't sound like much but it cuts the number of memory accesses per loop in half.  

Individually, three transformations don't seem like much, but they take already tight code and cut the work more than in-half (for the L1 and L2 case).   Collectively, it frees a couple of registers for better register allocation, it reduces the number of compares inside the loop, in substantially reduces the time for the entry address calculation, and in the ignore-dummy case cuts the number of memory accesses per loop in half.

The resulting code is astonishingly tight compared to the original and looks almost as tight as the code in set_insert_clean().


--- inner-loop of set_lookkey_dummy_allowed() ---
L393:
    cmpq    %r8, (%rbx)        entry->key == dummy
    je  L453
    cmpq    %r12, %rbx         entry == limit
    movq    %rbx, %r14
    je  L506
L412:
    movq    16(%r14), %rbp
    leaq    16(%r14), %rbx
    testq   %rbp, %rbp         entry->key NULL
    je  L449
    cmpq    8(%rbx), %r15      entry->hash == hash
    jne L393


--- inner-loop of set_lookkey_dummy_ignored() ---
L846:
    cmpq    %r13, %rbx         entry == limit
    je  L942
    addq    $16, %rbx          entry++
    movq    8(%rbx), %rax      entry->hash == NULL
    testq   %rax, %rax
    jne L845
    cmpq    $0, (%rbx)         entry->key == NULL (not in the inner loop)
    je  L905
L845:
    cmpq    %r12, %rax         entry->hash == tgt
    jne L846


Before anything can go forward, more timings are needed and I need to come to terms with the code expansion back to size it was before the old lookkey specializations where removed.  For now, I just want to document the work that has been done.

Anyway, here it is if someone wants to experiment with it.  Expect benefits in the cases where the inner-loop matters (small to medium sized sets that have collisions -- giant sets benefit as well but their lookup speed is dominated by the cost to fetch a single random cache line).

----------
files: hoisted_and_specialized.diff
keywords: patch
messages: 235131
nosy: rhettinger
priority: low
severity: normal
stage: patch review
status: open
title: Speed-up set_lookkey()
type: performance
versions: Python 3.5
Added file: http://bugs.python.org/file37945/hoisted_and_specialized.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23359>
_______________________________________