[Python-checkins] r67874 - in python/branches/py3k-issue1717: Doc/extending/embedding.rst Doc/extending/extending.rst Doc/howto/functional.rst Doc/library/dis.rst Doc/library/functions.rst Doc/library/stdtypes.rst Doc/reference/lexical_analysis.rst Doc/tutorial/datastructures.rst Doc/tutorial/modules.rst Doc/whatsnew/2.7.rst Doc/whatsnew/3.0.rst Doc/whatsnew/3.1.rst Include/opcode.h Lib/bdb.py Lib/decimal.py Lib/doctest.py Lib/http/client.py Lib/io.py Lib/lib2to3/fixer_base.py Lib/lib2to3/fixer_util.py Lib/lib2to3/fixes/fix_apply.py Lib/lib2to3/fixes/fix_has_key.py Lib/lib2to3/fixes/fix_imports.py Lib/lib2to3/fixes/fix_imports2.py Lib/lib2to3/fixes/fix_intern.py Lib/lib2to3/fixes/fix_isinstance.py Lib/lib2to3/fixes/fix_long.py Lib/lib2to3/fixes/fix_reduce.py Lib/lib2to3/fixes/fix_repr.py Lib/lib2to3/fixes/fix_urllib.py Lib/lib2to3/fixes/fix_xrange.py Lib/lib2to3/main.py Lib/lib2to3/pygram.py Lib/lib2to3/pytree.py Lib/lib2to3/refactor.py Lib/lib2to3/tests/benchmark.py Lib/lib2to3/tests/data/infinite_recursion.py Lib/lib2to3/tests/test_fixers.py Lib/lib2to3/tests/test_util.py Lib/linecache.py Lib/opcode.py Lib/pdb.py Lib/runpy.py Lib/tarfile.py Lib/test/decimaltestdata/extra.decTest Lib/test/test_array.py Lib/test/test_cmd_line_script.py Lib/test/test_dis.py Lib/test/test_doctest.py Lib/test/test_httplib.py Lib/test/test_inspect.py Lib/test/test_io.py Lib/test/test_long.py Lib/test/test_normalization.py Lib/test/test_tarfile.py Lib/test/test_textwrap.py Lib/test/test_tokenize.py Lib/test/test_urllib2_localnet.py Lib/test/test_urllibnet.py Lib/test/test_with.py Lib/test/test_zipimport.py Lib/test/test_zipimport_support.py Lib/textwrap.py Lib/tokenize.py Lib/webbrowser.py Makefile.pre.in Misc/ACKS Misc/NEWS Modules/_fileio.c Modules/arraymodule.c Modules/gcmodule.c Modules/mathmodule.c Modules/posixmodule.c Modules/zipimport.c Objects/floatobject.c Objects/longobject.c Objects/unicodeobject.c Parser/tokenizer.c Python/ceval.c Python/compile.c Python/import.c Tools/msi/crtlicense.txt
mark.dickinson
python-checkins at python.org
Sat Dec 20 14:43:16 CET 2008
Author: mark.dickinson
Date: Sat Dec 20 14:43:13 2008
New Revision: 67874
Log:
Merged revisions 67682-67683,67689,67695,67697,67702-67703,67708,67711,67713,67715,67719,67724,67747,67749,67753,67755,67759,67763,67780-67783,67786,67789,67792,67794,67798,67803,67810,67814,67823-67824,67834,67839-67841,67843,67846,67851,67858,67860,67865,67871 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/branches/py3k
................
r67682 | georg.brandl | 2008-12-09 23:48:44 +0000 (Tue, 09 Dec 2008) | 1 line
#4592: fix embedding example with new C API changes.
................
r67683 | fred.drake | 2008-12-10 06:02:39 +0000 (Wed, 10 Dec 2008) | 2 lines
simplify imports
................
r67689 | amaury.forgeotdarc | 2008-12-10 23:49:33 +0000 (Wed, 10 Dec 2008) | 13 lines
Merged revisions 67688 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk
........
r67688 | amaury.forgeotdarc | 2008-12-11 00:22:49 +0100 (jeu., 11 déc. 2008) | 6 lines
#4559: When a context manager's __exit__() method returns an object whose
conversion to bool raises an exception, 'with' loses that exception.
Reviewed by Jeffrey Yasskin.
Already ported to 2.5, will port to 2.6 and 3.0
........
................
r67695 | facundo.batista | 2008-12-11 04:20:07 +0000 (Thu, 11 Dec 2008) | 6 lines
Issue #4084: Fix max, min, max_mag and min_mag Decimal methods to
give correct results in the case where one argument is a quiet NaN
and the other is a finite number that requires rounding.
Thanks Mark Dickinson.
................
r67697 | jeffrey.yasskin | 2008-12-11 06:18:33 +0000 (Thu, 11 Dec 2008) | 14 lines
Merged revisions 67666,67685 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk
........
r67666 | jeffrey.yasskin | 2008-12-08 10:55:24 -0800 (Mon, 08 Dec 2008) | 3 lines
Issue 4597: Fix several cases in EvalFrameEx where an exception could be
"raised" without setting x, err, or why to let the eval loop know.
........
r67685 | jeffrey.yasskin | 2008-12-09 23:35:02 -0800 (Tue, 09 Dec 2008) | 2 lines
Update Misc/NEWS for r67666.
........
................
r67702 | mark.dickinson | 2008-12-11 09:22:25 +0000 (Thu, 11 Dec 2008) | 12 lines
Blocked revisions 67694 via svnmerge
........
r67694 | facundo.batista | 2008-12-11 04:19:46 +0000 (Thu, 11 Dec 2008) | 6 lines
Issue #4084: Fix max, min, max_mag and min_mag Decimal methods to
give correct results in the case where one argument is a quiet NaN
and the other is a finite number that requires rounding.
Thanks Mark Dickinson.
........
................
r67703 | mark.dickinson | 2008-12-11 18:03:03 +0000 (Thu, 11 Dec 2008) | 4 lines
Issue #2173: fix build failure on OS X. device_encoding was returning an
empty string, causing an (invisible) LookupError on any attempt to write
to sys.stdout.
................
r67708 | mark.dickinson | 2008-12-11 21:56:00 +0000 (Thu, 11 Dec 2008) | 12 lines
Merged revisions 67707 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk
........
r67707 | mark.dickinson | 2008-12-11 19:28:08 +0000 (Thu, 11 Dec 2008) | 5 lines
Issues #3167, #3682: tests for math.log and math.log10 were failing on
Solaris and OpenBSD. Fix this by handling special values and domain
errors directly in mathmodule.c, passing only positive nonspecial floats
to the system log/log10.
........
................
r67711 | benjamin.peterson | 2008-12-12 01:25:05 +0000 (Fri, 12 Dec 2008) | 1 line
raise a SyntaxError in detect_encoding() when a codec lookup fails like the builtin parser #4021
................
r67713 | benjamin.peterson | 2008-12-12 01:33:38 +0000 (Fri, 12 Dec 2008) | 1 line
reuse tokenize.detect_encoding for linecache #4016
................
r67715 | benjamin.peterson | 2008-12-12 02:02:24 +0000 (Fri, 12 Dec 2008) | 1 line
revert r67713. it causes build problems
................
r67719 | lars.gustaebel | 2008-12-12 14:58:38 +0000 (Fri, 12 Dec 2008) | 9 lines
Merged revisions 67717 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk
........
r67717 | lars.gustaebel | 2008-12-12 14:58:03 +0100 (Fri, 12 Dec 2008) | 2 lines
Issue #4616: TarFile.utime(): Restore directory times on Windows.
........
................
r67724 | benjamin.peterson | 2008-12-13 03:03:41 +0000 (Sat, 13 Dec 2008) | 1 line
string.maketrans -> str.maketrans
................
r67747 | antoine.pitrou | 2008-12-13 23:20:54 +0000 (Sat, 13 Dec 2008) | 3 lines
Issue #4163: textwrap module: allow word splitting on a hyphen preceded by a non-ASCII letter.
................
r67749 | benjamin.peterson | 2008-12-14 01:53:41 +0000 (Sun, 14 Dec 2008) | 8 lines
Blocked revisions 67748 via svnmerge
........
r67748 | benjamin.peterson | 2008-12-13 19:46:11 -0600 (Sat, 13 Dec 2008) | 1 line
remove has_key usage
........
................
r67753 | nick.coghlan | 2008-12-14 11:50:48 +0000 (Sun, 14 Dec 2008) | 13 lines
Merged revisions 67750-67751 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk
........
r67750 | nick.coghlan | 2008-12-14 20:54:50 +1000 (Sun, 14 Dec 2008) | 1 line
Fix several issues relating to access to source code inside zipfiles. Initial work by Alexander Belopolsky. See Misc/NEWS in this checkin for details.
........
r67751 | nick.coghlan | 2008-12-14 21:09:40 +1000 (Sun, 14 Dec 2008) | 1 line
Add file that was missed from r67750
........
................
r67755 | benjamin.peterson | 2008-12-14 15:09:34 +0000 (Sun, 14 Dec 2008) | 1 line
tip-toe around dictionary keys view in the tutorial
................
r67759 | antoine.pitrou | 2008-12-14 16:36:46 +0000 (Sun, 14 Dec 2008) | 3 lines
Issue #4574: reading an UTF16-encoded text file crashes if \r on 64-char boundary.
................
r67763 | antoine.pitrou | 2008-12-14 17:44:09 +0000 (Sun, 14 Dec 2008) | 8 lines
Blocked revisions 67762 via svnmerge
........
r67762 | antoine.pitrou | 2008-12-14 18:40:51 +0100 (dim., 14 déc. 2008) | 3 lines
Backport r67759 (fix io.IncrementalNewlineDecoder for UTF-16 et al.).
........
................
r67780 | jeremy.hylton | 2008-12-15 03:00:50 +0000 (Mon, 15 Dec 2008) | 2 lines
Use True/False for ints instead of 1/0. That's so Python 2.0.
................
r67781 | jeremy.hylton | 2008-12-15 03:08:30 +0000 (Mon, 15 Dec 2008) | 2 lines
Reflow long line.
................
r67782 | georg.brandl | 2008-12-15 08:28:37 +0000 (Mon, 15 Dec 2008) | 2 lines
#4667: fix some 2.x leftovers in the tutorial.
................
r67783 | georg.brandl | 2008-12-15 08:29:32 +0000 (Mon, 15 Dec 2008) | 2 lines
#4668: wrap iterator returns in list() in functional howto.
................
r67786 | georg.brandl | 2008-12-15 08:43:10 +0000 (Mon, 15 Dec 2008) | 2 lines
#4603: Note that inconsistent tab/space use is now illegal.
................
r67789 | georg.brandl | 2008-12-15 09:16:15 +0000 (Mon, 15 Dec 2008) | 2 lines
Use :samp: role.
................
r67792 | nick.coghlan | 2008-12-15 12:01:34 +0000 (Mon, 15 Dec 2008) | 9 lines
Merged revisions 67790 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk
........
r67790 | nick.coghlan | 2008-12-15 21:41:05 +1000 (Mon, 15 Dec 2008) | 1 line
Issue #4197: Fix the remaining part of the doctest-in-zipfile problem by giving linecache access to the module globals when available
........
................
r67794 | antoine.pitrou | 2008-12-15 13:08:55 +0000 (Mon, 15 Dec 2008) | 13 lines
Merged revisions 67777,67779 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk
........
r67777 | antoine.pitrou | 2008-12-14 23:33:55 +0100 (dim., 14 déc. 2008) | 3 lines
try to fix failure in test_bad_address on some buildbots
........
r67779 | antoine.pitrou | 2008-12-15 01:39:51 +0100 (lun., 15 déc. 2008) | 1 line
modify other occurrence of test_bad_address
........
................
r67798 | amaury.forgeotdarc | 2008-12-15 22:10:56 +0000 (Mon, 15 Dec 2008) | 10 lines
Blocked revisions 67797 via svnmerge
........
r67797 | amaury.forgeotdarc | 2008-12-15 22:47:57 +0100 (Mon, 15 Dec 2008) | 4 lines
#3954: Fix error handling code in _hotshot.logreader
Will port to 2.6. hotshot was deleted from python 3.
........
................
r67803 | antoine.pitrou | 2008-12-15 23:01:43 +0000 (Mon, 15 Dec 2008) | 4 lines
Issue #4663: increase the chunk size to improve TextIOWrapper performance on small reads.
Text I/O is still 10x to 100x slower than 2.x!
................
r67810 | benjamin.peterson | 2008-12-16 03:57:54 +0000 (Tue, 16 Dec 2008) | 118 lines
Merged revisions 67806 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk
................
r67806 | benjamin.peterson | 2008-12-15 21:35:28 -0600 (Mon, 15 Dec 2008) | 111 lines
Merged revisions 67427,67431,67433,67435,67630,67652,67656-67657,67674-67675,67678-67679,67705-67706,67716,67723,67765-67771,67774,67776,67778 via svnmerge from
svn+ssh://pythondev@svn.python.org/sandbox/trunk/2to3/lib2to3
........
r67427 | benjamin.peterson | 2008-11-28 16:07:41 -0600 (Fri, 28 Nov 2008) | 1 line
fix spelling in comment
........
r67431 | benjamin.peterson | 2008-11-28 17:14:08 -0600 (Fri, 28 Nov 2008) | 1 line
add a scripts directory; move things to it
........
r67433 | benjamin.peterson | 2008-11-28 17:18:48 -0600 (Fri, 28 Nov 2008) | 1 line
run svneol.py
........
r67435 | benjamin.peterson | 2008-11-28 17:25:03 -0600 (Fri, 28 Nov 2008) | 1 line
rename pre/post_order_mapping to pre/post_order_heads
........
r67630 | alexandre.vassalotti | 2008-12-06 21:51:56 -0600 (Sat, 06 Dec 2008) | 2 lines
Fix typo in the urllib2.HTTPDigestAuthHandler fixer.
........
r67652 | armin.ronacher | 2008-12-07 15:39:43 -0600 (Sun, 07 Dec 2008) | 5 lines
Added a fixer that cleans up a tuple argument to isinstance after the tokens
in it were fixed. This is mainly used to remove double occurrences of
tokens as a leftover of the long -> int / unicode -> str conversion.
........
r67656 | armin.ronacher | 2008-12-07 16:54:16 -0600 (Sun, 07 Dec 2008) | 3 lines
Added missing copyright fo 2to3 fix_isinstance.
........
r67657 | armin.ronacher | 2008-12-07 18:29:35 -0600 (Sun, 07 Dec 2008) | 3 lines
2to3: intern and reduce fixes now add the imports if missing. Because that is a common task the fixer_util module now has a function "touch_import" that adds imports if missing.
........
r67674 | benjamin.peterson | 2008-12-08 19:58:11 -0600 (Mon, 08 Dec 2008) | 1 line
copy permission bits when making backup files #4602
........
r67675 | benjamin.peterson | 2008-12-08 19:59:11 -0600 (Mon, 08 Dec 2008) | 1 line
add forgotten import
........
r67678 | benjamin.peterson | 2008-12-08 20:08:30 -0600 (Mon, 08 Dec 2008) | 1 line
fix #4602 for real
........
r67679 | armin.ronacher | 2008-12-09 00:54:03 -0600 (Tue, 09 Dec 2008) | 3 lines
Removed redudant code from the 2to3 long fixer. This fixes #4590.
........
r67705 | benjamin.peterson | 2008-12-11 13:04:08 -0600 (Thu, 11 Dec 2008) | 1 line
put trailers after a range call after the list()
........
r67706 | benjamin.peterson | 2008-12-11 13:17:57 -0600 (Thu, 11 Dec 2008) | 1 line
add html related modules to the fix_imports mapping
........
r67716 | benjamin.peterson | 2008-12-11 22:16:47 -0600 (Thu, 11 Dec 2008) | 1 line
consolidate tests
........
r67723 | benjamin.peterson | 2008-12-12 19:49:31 -0600 (Fri, 12 Dec 2008) | 1 line
fix name
........
r67765 | benjamin.peterson | 2008-12-14 14:05:05 -0600 (Sun, 14 Dec 2008) | 1 line
run fix_isinstance after fix_long and fix_unicode
........
r67766 | benjamin.peterson | 2008-12-14 14:13:05 -0600 (Sun, 14 Dec 2008) | 1 line
use run_order instead of order
........
r67767 | benjamin.peterson | 2008-12-14 14:28:12 -0600 (Sun, 14 Dec 2008) | 1 line
don't retain parenthesis if there is only one item left
........
r67768 | benjamin.peterson | 2008-12-14 14:32:30 -0600 (Sun, 14 Dec 2008) | 1 line
use insert_child()
........
r67769 | benjamin.peterson | 2008-12-14 14:59:10 -0600 (Sun, 14 Dec 2008) | 1 line
parenthesize doesn't belong in pygram or FixerBase
........
r67770 | alexandre.vassalotti | 2008-12-14 15:15:36 -0600 (Sun, 14 Dec 2008) | 2 lines
Fix typo: html.paser -> html.parser.
........
r67771 | benjamin.peterson | 2008-12-14 15:22:09 -0600 (Sun, 14 Dec 2008) | 1 line
altering .children needs to call changed()
........
r67774 | benjamin.peterson | 2008-12-14 15:55:38 -0600 (Sun, 14 Dec 2008) | 1 line
employ an evil hack to fix multiple names in the same import statement
........
r67776 | benjamin.peterson | 2008-12-14 16:22:38 -0600 (Sun, 14 Dec 2008) | 1 line
make a common mixin class for Test_imports and friends
........
r67778 | alexandre.vassalotti | 2008-12-14 17:48:20 -0600 (Sun, 14 Dec 2008) | 2 lines
Make fix_imports refactor multiple imports as.
........
................
................
r67814 | antoine.pitrou | 2008-12-16 14:25:45 +0000 (Tue, 16 Dec 2008) | 3 lines
Try to fix #4674: test_normalization failure on some buildbots.
................
r67823 | mark.dickinson | 2008-12-17 16:16:41 +0000 (Wed, 17 Dec 2008) | 10 lines
Blocked revisions 67822 via svnmerge
........
r67822 | mark.dickinson | 2008-12-17 16:14:37 +0000 (Wed, 17 Dec 2008) | 4 lines
Issue #3439: add bit_length method to int and long.
Thanks Fredrik Johansson and Victor Stinner for code,
Raymond Hettinger for review.
........
................
r67824 | mark.dickinson | 2008-12-17 16:19:07 +0000 (Wed, 17 Dec 2008) | 2 lines
Forward merge of r67822 to py3k: add bit_length method to int.
................
r67834 | antoine.pitrou | 2008-12-17 23:18:19 +0000 (Wed, 17 Dec 2008) | 10 lines
Merged revisions 67832 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk
........
r67832 | antoine.pitrou | 2008-12-17 23:46:54 +0100 (mer., 17 déc. 2008) | 4 lines
Issue #2467: gc.DEBUG_STATS reports invalid elapsed times.
Patch by Neil Schemenauer, very slightly modified.
........
................
r67839 | antoine.pitrou | 2008-12-18 11:06:25 +0000 (Thu, 18 Dec 2008) | 9 lines
Merged revisions 67818 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk
........
r67818 | antoine.pitrou | 2008-12-17 01:38:28 +0100 (mer., 17 déc. 2008) | 3 lines
Issue #2183: Simplify and optimize bytecode for list comprehensions.
........
................
r67840 | antoine.pitrou | 2008-12-18 17:08:32 +0000 (Thu, 18 Dec 2008) | 3 lines
Issue #4583: crash after resizing an array.array which has buffer exports.
................
r67841 | kristjan.jonsson | 2008-12-18 17:08:57 +0000 (Thu, 18 Dec 2008) | 2 lines
Add missing Py_CHARMASK when calling isspace().
Found by enabling runtime tests on windows, by disabling the _set_invalid_parameter_handler() fiddling.
................
r67843 | kristjan.jonsson | 2008-12-18 17:15:54 +0000 (Thu, 18 Dec 2008) | 5 lines
Fix an issue in the tokenizer, where a file is opened by fd, but the underlying PyFileIO object wasn created with the closefd attribute true.
Also fix error handling for close() int _fileio.c . It was incorrect, looking for a negative refcount, and so errors weren't raised. This is why this issue wasn't caught.
There is a second reason why it isn't seen: Class IOBase in io.py has a try:/except: around the close() funtion in the __del__() method. This also masks these error conditions.
This issue was discovered by removing the _set_invalid_parameter_handler() fiddling, thus enabling the C runtime checks on windows.
................
r67846 | mark.dickinson | 2008-12-18 19:49:35 +0000 (Thu, 18 Dec 2008) | 10 lines
Merged revisions 67844 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk
........
r67844 | mark.dickinson | 2008-12-18 19:46:21 +0000 (Thu, 18 Dec 2008) | 3 lines
Issue 4692: bogus 'Make' in Makefile.pre.in; replace with '$MAKE'.
Thanks Ned Deily.
........
................
r67851 | raymond.hettinger | 2008-12-19 09:11:49 +0000 (Fri, 19 Dec 2008) | 9 lines
Fix-up and clean-up docs for int.bit_length().
* Replace dramatic footnote with in-line comment about possible round-off errors in logarithms of large numbers.
* Add comments to the pure python code equivalent.
* replace floor() with int() in the mathematical equivalent so the type is correct (should be an int, not a float).
* add abs() to the mathematical equivalent so that it matches the previous line that it is supposed to be equivalent to.
* make one combined example with a negative input.
................
r67858 | mark.dickinson | 2008-12-19 17:48:51 +0000 (Fri, 19 Dec 2008) | 9 lines
Merged revisions 67857 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk
........
r67857 | mark.dickinson | 2008-12-19 17:46:51 +0000 (Fri, 19 Dec 2008) | 2 lines
Fix typo in Python equivalent for bit_length.
........
................
r67860 | amaury.forgeotdarc | 2008-12-19 22:58:37 +0000 (Fri, 19 Dec 2008) | 11 lines
Merged revisions 67859 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk
........
r67859 | amaury.forgeotdarc | 2008-12-19 23:56:48 +0100 (ven., 19 déc. 2008) | 4 lines
#4700: crtlicense.txt is displayed by the license() command and should be kept ascii-only.
Will port to 3.0
........
................
r67865 | benjamin.peterson | 2008-12-20 03:20:23 +0000 (Sat, 20 Dec 2008) | 1 line
fix syntax
................
r67871 | antoine.pitrou | 2008-12-20 13:14:23 +0000 (Sat, 20 Dec 2008) | 4 lines
Issue #3106: Speedup some comparisons. This also removes the last call
to Py_CmpToRich from the codebase (in longobject.c).
................
Added:
python/branches/py3k-issue1717/Lib/lib2to3/fixes/fix_isinstance.py
- copied unchanged from r67871, /python/branches/py3k/Lib/lib2to3/fixes/fix_isinstance.py
python/branches/py3k-issue1717/Lib/lib2to3/fixes/fix_reduce.py
- copied unchanged from r67871, /python/branches/py3k/Lib/lib2to3/fixes/fix_reduce.py
python/branches/py3k-issue1717/Lib/test/test_zipimport_support.py
- copied unchanged from r67871, /python/branches/py3k/Lib/test/test_zipimport_support.py
Removed:
python/branches/py3k-issue1717/Lib/lib2to3/tests/benchmark.py
Modified:
python/branches/py3k-issue1717/ (props changed)
python/branches/py3k-issue1717/Doc/extending/embedding.rst
python/branches/py3k-issue1717/Doc/extending/extending.rst
python/branches/py3k-issue1717/Doc/howto/functional.rst
python/branches/py3k-issue1717/Doc/library/dis.rst
python/branches/py3k-issue1717/Doc/library/functions.rst
python/branches/py3k-issue1717/Doc/library/stdtypes.rst
python/branches/py3k-issue1717/Doc/reference/lexical_analysis.rst
python/branches/py3k-issue1717/Doc/tutorial/datastructures.rst
python/branches/py3k-issue1717/Doc/tutorial/modules.rst
python/branches/py3k-issue1717/Doc/whatsnew/2.7.rst
python/branches/py3k-issue1717/Doc/whatsnew/3.0.rst
python/branches/py3k-issue1717/Doc/whatsnew/3.1.rst
python/branches/py3k-issue1717/Include/opcode.h
python/branches/py3k-issue1717/Lib/bdb.py
python/branches/py3k-issue1717/Lib/decimal.py
python/branches/py3k-issue1717/Lib/doctest.py
python/branches/py3k-issue1717/Lib/http/client.py
python/branches/py3k-issue1717/Lib/io.py
python/branches/py3k-issue1717/Lib/lib2to3/fixer_base.py
python/branches/py3k-issue1717/Lib/lib2to3/fixer_util.py
python/branches/py3k-issue1717/Lib/lib2to3/fixes/fix_apply.py
python/branches/py3k-issue1717/Lib/lib2to3/fixes/fix_has_key.py
python/branches/py3k-issue1717/Lib/lib2to3/fixes/fix_imports.py
python/branches/py3k-issue1717/Lib/lib2to3/fixes/fix_imports2.py (contents, props changed)
python/branches/py3k-issue1717/Lib/lib2to3/fixes/fix_intern.py
python/branches/py3k-issue1717/Lib/lib2to3/fixes/fix_long.py
python/branches/py3k-issue1717/Lib/lib2to3/fixes/fix_repr.py
python/branches/py3k-issue1717/Lib/lib2to3/fixes/fix_urllib.py (contents, props changed)
python/branches/py3k-issue1717/Lib/lib2to3/fixes/fix_xrange.py
python/branches/py3k-issue1717/Lib/lib2to3/main.py
python/branches/py3k-issue1717/Lib/lib2to3/pygram.py
python/branches/py3k-issue1717/Lib/lib2to3/pytree.py
python/branches/py3k-issue1717/Lib/lib2to3/refactor.py
python/branches/py3k-issue1717/Lib/lib2to3/tests/data/infinite_recursion.py (props changed)
python/branches/py3k-issue1717/Lib/lib2to3/tests/test_fixers.py
python/branches/py3k-issue1717/Lib/lib2to3/tests/test_util.py
python/branches/py3k-issue1717/Lib/linecache.py
python/branches/py3k-issue1717/Lib/opcode.py
python/branches/py3k-issue1717/Lib/pdb.py
python/branches/py3k-issue1717/Lib/runpy.py
python/branches/py3k-issue1717/Lib/tarfile.py
python/branches/py3k-issue1717/Lib/test/decimaltestdata/extra.decTest
python/branches/py3k-issue1717/Lib/test/test_array.py
python/branches/py3k-issue1717/Lib/test/test_cmd_line_script.py
python/branches/py3k-issue1717/Lib/test/test_dis.py
python/branches/py3k-issue1717/Lib/test/test_doctest.py
python/branches/py3k-issue1717/Lib/test/test_httplib.py
python/branches/py3k-issue1717/Lib/test/test_inspect.py
python/branches/py3k-issue1717/Lib/test/test_io.py
python/branches/py3k-issue1717/Lib/test/test_long.py
python/branches/py3k-issue1717/Lib/test/test_normalization.py
python/branches/py3k-issue1717/Lib/test/test_tarfile.py
python/branches/py3k-issue1717/Lib/test/test_textwrap.py
python/branches/py3k-issue1717/Lib/test/test_tokenize.py
python/branches/py3k-issue1717/Lib/test/test_urllib2_localnet.py
python/branches/py3k-issue1717/Lib/test/test_urllibnet.py
python/branches/py3k-issue1717/Lib/test/test_with.py
python/branches/py3k-issue1717/Lib/test/test_zipimport.py
python/branches/py3k-issue1717/Lib/textwrap.py
python/branches/py3k-issue1717/Lib/tokenize.py
python/branches/py3k-issue1717/Lib/webbrowser.py
python/branches/py3k-issue1717/Makefile.pre.in
python/branches/py3k-issue1717/Misc/ACKS
python/branches/py3k-issue1717/Misc/NEWS
python/branches/py3k-issue1717/Modules/_fileio.c
python/branches/py3k-issue1717/Modules/arraymodule.c
python/branches/py3k-issue1717/Modules/gcmodule.c
python/branches/py3k-issue1717/Modules/mathmodule.c
python/branches/py3k-issue1717/Modules/posixmodule.c
python/branches/py3k-issue1717/Modules/zipimport.c
python/branches/py3k-issue1717/Objects/floatobject.c
python/branches/py3k-issue1717/Objects/longobject.c
python/branches/py3k-issue1717/Objects/unicodeobject.c
python/branches/py3k-issue1717/Parser/tokenizer.c
python/branches/py3k-issue1717/Python/ceval.c
python/branches/py3k-issue1717/Python/compile.c
python/branches/py3k-issue1717/Python/import.c
python/branches/py3k-issue1717/Tools/msi/crtlicense.txt
Modified: python/branches/py3k-issue1717/Doc/extending/embedding.rst
==============================================================================
--- python/branches/py3k-issue1717/Doc/extending/embedding.rst (original)
+++ python/branches/py3k-issue1717/Doc/extending/embedding.rst Sat Dec 20 14:43:13 2008
@@ -223,11 +223,17 @@
NULL, NULL, NULL, NULL
};
+ static PyObject*
+ PyInit_emb(void)
+ {
+ return PyModule_Create(&EmbModule);
+ }
+
Insert the above code just above the :cfunc:`main` function. Also, insert the
-following two statements directly after :cfunc:`Py_Initialize`::
+following two statements before the call to :cfunc:`Py_Initialize`::
numargs = argc;
- PyModule_Create(&EmbModule);
+ PyImport_AppendInittab("emb", &PyInit_emb);
These two lines initialize the ``numargs`` variable, and make the
:func:`emb.numargs` function accessible to the embedded Python interpreter.
Modified: python/branches/py3k-issue1717/Doc/extending/extending.rst
==============================================================================
--- python/branches/py3k-issue1717/Doc/extending/extending.rst (original)
+++ python/branches/py3k-issue1717/Doc/extending/extending.rst Sat Dec 20 14:43:13 2008
@@ -342,7 +342,7 @@
so that it then gets inserted into ``sys.modules``.
When embedding Python, the :cfunc:`PyInit_spam` function is not called
-automatically unless there's an entry in the :cdata:`_PyImport_Inittab` table.
+automatically unless there's an entry in the :cdata:`PyImport_Inittab` table.
To add the module to the initialization table, use :cfunc:`PyImport_AppendInittab`,
optionally followed by an import of the module::
Modified: python/branches/py3k-issue1717/Doc/howto/functional.rst
==============================================================================
--- python/branches/py3k-issue1717/Doc/howto/functional.rst (original)
+++ python/branches/py3k-issue1717/Doc/howto/functional.rst Sat Dec 20 14:43:13 2008
@@ -634,7 +634,7 @@
... return s.upper()
- >>> map(upper, ['sentence', 'fragment'])
+ >>> list(map(upper, ['sentence', 'fragment']))
['SENTENCE', 'FRAGMENT']
>>> [upper(s) for s in ['sentence', 'fragment']]
['SENTENCE', 'FRAGMENT']
@@ -650,7 +650,7 @@
>>> def is_even(x):
... return (x % 2) == 0
- >>> filter(is_even, range(10))
+ >>> list(filter(is_even, range(10)))
[0, 2, 4, 6, 8]
Modified: python/branches/py3k-issue1717/Doc/library/dis.rst
==============================================================================
--- python/branches/py3k-issue1717/Doc/library/dis.rst (original)
+++ python/branches/py3k-issue1717/Doc/library/dis.rst Sat Dec 20 14:43:13 2008
@@ -357,14 +357,25 @@
address to jump to (which should be a ``FOR_ITER`` instruction).
-.. opcode:: SET_ADD ()
+.. opcode:: SET_ADD (i)
- Calls ``set.add(TOS1, TOS)``. Used to implement set comprehensions.
+ Calls ``set.add(TOS1[-i], TOS)``. Used to implement set comprehensions.
-.. opcode:: LIST_APPEND ()
+.. opcode:: LIST_APPEND (i)
- Calls ``list.append(TOS1, TOS)``. Used to implement list comprehensions.
+ Calls ``list.append(TOS[-i], TOS)``. Used to implement list comprehensions.
+
+
+.. opcode:: MAP_ADD (i)
+
+ Calls ``dict.setitem(TOS1[-i], TOS, TOS1)``. Used to implement dict
+ comprehensions.
+
+
+For all of the SET_ADD, LIST_APPEND and MAP_ADD instructions, while the
+added value or key/value pair is popped off, the container object remains on
+the stack so that it is available for further iterations of the loop.
.. opcode:: LOAD_LOCALS ()
Modified: python/branches/py3k-issue1717/Doc/library/functions.rst
==============================================================================
--- python/branches/py3k-issue1717/Doc/library/functions.rst (original)
+++ python/branches/py3k-issue1717/Doc/library/functions.rst Sat Dec 20 14:43:13 2008
@@ -289,7 +289,7 @@
:func:`enumerate` is useful for obtaining an indexed series: ``(0, seq[0])``,
``(1, seq[1])``, ``(2, seq[2])``, .... For example:
- >>> for i, season in enumerate(['Spring', 'Summer', 'Fall', 'Winter')]:
+ >>> for i, season in enumerate(['Spring', 'Summer', 'Fall', 'Winter']):
... print(i, season)
0 Spring
1 Summer
Modified: python/branches/py3k-issue1717/Doc/library/stdtypes.rst
==============================================================================
--- python/branches/py3k-issue1717/Doc/library/stdtypes.rst (original)
+++ python/branches/py3k-issue1717/Doc/library/stdtypes.rst Sat Dec 20 14:43:13 2008
@@ -419,6 +419,36 @@
overflow check.
+Additional Methods on Integer Types
+-----------------------------------
+
+.. method:: int.bit_length()
+
+ Return the number of bits necessary to represent an integer in binary,
+ excluding the sign and leading zeros::
+
+ >>> n = -37
+ >>> bin(n)
+ '-0b100101'
+ >>> n.bit_length()
+ 6
+
+ More precisely, if ``x`` is nonzero, then ``x.bit_length()`` is the
+ unique positive integer ``k`` such that ``2**(k-1) <= abs(x) < 2**k``.
+ Equivalently, when ``abs(x)`` is small enough to have a correctly
+ rounded logarithm, then ``k = 1 + int(log(abs(x), 2))``.
+ If ``x`` is zero, then ``x.bit_length()`` returns ``0``.
+
+ Equivalent to::
+
+ def bit_length(self):
+ s = bin(self) # binary representation: bin(-37) --> '-0b100101'
+ s = s.lstrip('-0b') # remove leading zeros and minus sign
+ return len(s) # len('100101') --> 6
+
+ .. versionadded:: 3.1
+
+
Additional Methods on Float
---------------------------
@@ -1090,12 +1120,9 @@
ordinals, strings or ``None``. Unmapped characters are left untouched.
Characters mapped to ``None`` are deleted.
- A *map* for :meth:`translate` is usually best created by
- :meth:`str.maketrans`.
-
- You can use the :func:`maketrans` helper function in the :mod:`string` module to
- create a translation table. For string objects, set the *table* argument to
- ``None`` for translations that only delete characters:
+ You can use :meth:`str.maketrans` to create a translation table. For string
+ objects, set the *table* argument to ``None`` for translations that only
+ delete characters:
.. note::
Modified: python/branches/py3k-issue1717/Doc/reference/lexical_analysis.rst
==============================================================================
--- python/branches/py3k-issue1717/Doc/reference/lexical_analysis.rst (original)
+++ python/branches/py3k-issue1717/Doc/reference/lexical_analysis.rst Sat Dec 20 14:43:13 2008
@@ -174,14 +174,18 @@
to compute the indentation level of the line, which in turn is used to determine
the grouping of statements.
-First, tabs are replaced (from left to right) by one to eight spaces such that
-the total number of characters up to and including the replacement is a multiple
-of eight (this is intended to be the same rule as used by Unix). The total
-number of spaces preceding the first non-blank character then determines the
-line's indentation. Indentation cannot be split over multiple physical lines
-using backslashes; the whitespace up to the first backslash determines the
+Tabs are replaced (from left to right) by one to eight spaces such that the
+total number of characters up to and including the replacement is a multiple of
+eight (this is intended to be the same rule as used by Unix). The total number
+of spaces preceding the first non-blank character then determines the line's
+indentation. Indentation cannot be split over multiple physical lines using
+backslashes; the whitespace up to the first backslash determines the
indentation.
+Indentation is rejected as inconsistent if a source file mixes tabs and spaces
+in a way that makes the meaning dependent on the worth of a tab in spaces; a
+:exc:`TabError` is raised in that case.
+
**Cross-platform compatibility note:** because of the nature of text editors on
non-UNIX platforms, it is unwise to use a mixture of spaces and tabs for the
indentation in a single source file. It should also be noted that different
Modified: python/branches/py3k-issue1717/Doc/tutorial/datastructures.rst
==============================================================================
--- python/branches/py3k-issue1717/Doc/tutorial/datastructures.rst (original)
+++ python/branches/py3k-issue1717/Doc/tutorial/datastructures.rst Sat Dec 20 14:43:13 2008
@@ -440,9 +440,9 @@
value associated with that key is forgotten. It is an error to extract a value
using a non-existent key.
-The :meth:`keys` method of a dictionary object returns a list of all the keys
+Performing ``list(d.keys())`` on a dictionary returns a list of all the keys
used in the dictionary, in arbitrary order (if you want it sorted, just apply
-the :meth:`sort` method to the list of keys). To check whether a single key is
+the :meth:`sorted` function instead). To check whether a single key is
in the dictionary, use the :keyword:`in` keyword.
Here is a small example using a dictionary::
@@ -458,6 +458,8 @@
>>> tel
{'guido': 4127, 'irv': 4127, 'jack': 4098}
>>> list(tel.keys())
+ ['irv', 'guido', 'jack']
+ >>> sorted(tel.keys())
['guido', 'irv', 'jack']
>>> 'guido' in tel
True
Modified: python/branches/py3k-issue1717/Doc/tutorial/modules.rst
==============================================================================
--- python/branches/py3k-issue1717/Doc/tutorial/modules.rst (original)
+++ python/branches/py3k-issue1717/Doc/tutorial/modules.rst Sat Dec 20 14:43:13 2008
@@ -112,7 +112,7 @@
For efficiency reasons, each module is only imported once per interpreter
session. Therefore, if you change your modules, you must restart the
interpreter -- or, if it's just one module you want to test interactively,
- use :func:`reload`, e.g. ``reload(modulename)``.
+ use :func:`imp.reload`, e.g. ``import imp; imp.reload(modulename)``.
.. _tut-modulesasscripts:
Modified: python/branches/py3k-issue1717/Doc/whatsnew/2.7.rst
==============================================================================
--- python/branches/py3k-issue1717/Doc/whatsnew/2.7.rst (original)
+++ python/branches/py3k-issue1717/Doc/whatsnew/2.7.rst Sat Dec 20 14:43:13 2008
@@ -66,7 +66,23 @@
Some smaller changes made to the core Python language are:
-* List of changes to be written here.
+* The :func:`int` and :func:`long` types gained a ``bit_length``
+ method that returns the number of bits necessary to represent
+ its argument in binary::
+
+ >>> n = 37
+ >>> bin(37)
+ '0b100101'
+ >>> n.bit_length()
+ 6
+ >>> n = 2**123-1
+ >>> n.bit_length()
+ 123
+ >>> (n+1).bit_length()
+ 124
+
+ (Contributed by Fredrik Johansson and Victor Stinner; :issue:`3439`.)
+
.. ======================================================================
Modified: python/branches/py3k-issue1717/Doc/whatsnew/3.0.rst
==============================================================================
--- python/branches/py3k-issue1717/Doc/whatsnew/3.0.rst (original)
+++ python/branches/py3k-issue1717/Doc/whatsnew/3.0.rst Sat Dec 20 14:43:13 2008
@@ -696,15 +696,15 @@
* Exceptions no longer behave as sequences. Use the :attr:`args`
attribute instead.
-* :pep:`3109`: Raising exceptions. You must now use :keyword:`raise`
- *Exception*(*args*) instead of :keyword:`raise` *Exception*, *args*.
+* :pep:`3109`: Raising exceptions. You must now use :samp:`raise
+ {Exception}({args})` instead of :samp:`raise {Exception}, {args}`.
Additionally, you can no longer explicitly specify a traceback;
instead, if you *have* to do this, you can assign directly to the
:attr:`__traceback__` attribute (see below).
* :pep:`3110`: Catching exceptions. You must now use
- :keyword:`except` *SomeException* :keyword:`as` *variable* instead
- of :keyword:`except` *SomeException*, *variable*. Moreover, the
+ :samp:`except {SomeException} as {variable}` instead
+ of :samp:`except {SomeException}, {variable}`. Moreover, the
*variable* is explicitly deleted when the :keyword:`except` block
is left.
Modified: python/branches/py3k-issue1717/Doc/whatsnew/3.1.rst
==============================================================================
--- python/branches/py3k-issue1717/Doc/whatsnew/3.1.rst (original)
+++ python/branches/py3k-issue1717/Doc/whatsnew/3.1.rst Sat Dec 20 14:43:13 2008
@@ -66,4 +66,26 @@
.. ======================================================================
+Other Language Changes
+======================
+
+Some smaller changes made to the core Python language are:
+
+* The :func:`int` type gained a ``bit_length`` method that returns the
+ number of bits necessary to represent its argument in binary::
+
+ >>> n = 37
+ >>> bin(37)
+ '0b100101'
+ >>> n.bit_length()
+ 6
+ >>> n = 2**123-1
+ >>> n.bit_length()
+ 123
+ >>> (n+1).bit_length()
+ 124
+
+ (Contributed by Fredrik Johansson and Victor Stinner; :issue:`3439`.)
+
+
.. ======================================================================
Modified: python/branches/py3k-issue1717/Include/opcode.h
==============================================================================
--- python/branches/py3k-issue1717/Include/opcode.h (original)
+++ python/branches/py3k-issue1717/Include/opcode.h Sat Dec 20 14:43:13 2008
@@ -21,8 +21,6 @@
#define UNARY_INVERT 15
-#define SET_ADD 17
-#define LIST_APPEND 18
#define BINARY_POWER 19
#define BINARY_MULTIPLY 20
@@ -133,6 +131,10 @@
/* Support for opargs more than 16 bits long */
#define EXTENDED_ARG 143
+#define LIST_APPEND 145
+#define SET_ADD 146
+#define MAP_ADD 147
+
/* EXCEPT_HANDLER is a special, implicit block type which is created when
entering an except handler. It is not an opcode but we define it here
Modified: python/branches/py3k-issue1717/Lib/bdb.py
==============================================================================
--- python/branches/py3k-issue1717/Lib/bdb.py (original)
+++ python/branches/py3k-issue1717/Lib/bdb.py Sat Dec 20 14:43:13 2008
@@ -346,7 +346,7 @@
rv = frame.f_locals['__return__']
s = s + '->'
s = s + reprlib.repr(rv)
- line = linecache.getline(filename, lineno)
+ line = linecache.getline(filename, lineno, frame.f_globals)
if line: s = s + lprefix + line.strip()
return s
@@ -588,7 +588,7 @@
name = frame.f_code.co_name
if not name: name = '???'
fn = self.canonic(frame.f_code.co_filename)
- line = linecache.getline(fn, frame.f_lineno)
+ line = linecache.getline(fn, frame.f_lineno, frame.f_globals)
print('+++', fn, frame.f_lineno, name, ':', line.strip())
def user_return(self, frame, retval):
print('+++ return', retval)
Modified: python/branches/py3k-issue1717/Lib/decimal.py
==============================================================================
--- python/branches/py3k-issue1717/Lib/decimal.py (original)
+++ python/branches/py3k-issue1717/Lib/decimal.py Sat Dec 20 14:43:13 2008
@@ -2663,10 +2663,10 @@
sn = self._isnan()
on = other._isnan()
if sn or on:
- if on == 1 and sn != 2:
- return self._fix_nan(context)
- if sn == 1 and on != 2:
- return other._fix_nan(context)
+ if on == 1 and sn == 0:
+ return self._fix(context)
+ if sn == 1 and on == 0:
+ return other._fix(context)
return self._check_nans(other, context)
c = self._cmp(other)
@@ -2705,10 +2705,10 @@
sn = self._isnan()
on = other._isnan()
if sn or on:
- if on == 1 and sn != 2:
- return self._fix_nan(context)
- if sn == 1 and on != 2:
- return other._fix_nan(context)
+ if on == 1 and sn == 0:
+ return self._fix(context)
+ if sn == 1 and on == 0:
+ return other._fix(context)
return self._check_nans(other, context)
c = self._cmp(other)
@@ -3263,10 +3263,10 @@
sn = self._isnan()
on = other._isnan()
if sn or on:
- if on == 1 and sn != 2:
- return self._fix_nan(context)
- if sn == 1 and on != 2:
- return other._fix_nan(context)
+ if on == 1 and sn == 0:
+ return self._fix(context)
+ if sn == 1 and on == 0:
+ return other._fix(context)
return self._check_nans(other, context)
c = self.copy_abs()._cmp(other.copy_abs())
@@ -3293,10 +3293,10 @@
sn = self._isnan()
on = other._isnan()
if sn or on:
- if on == 1 and sn != 2:
- return self._fix_nan(context)
- if sn == 1 and on != 2:
- return other._fix_nan(context)
+ if on == 1 and sn == 0:
+ return self._fix(context)
+ if sn == 1 and on == 0:
+ return other._fix(context)
return self._check_nans(other, context)
c = self.copy_abs()._cmp(other.copy_abs())
Modified: python/branches/py3k-issue1717/Lib/doctest.py
==============================================================================
--- python/branches/py3k-issue1717/Lib/doctest.py (original)
+++ python/branches/py3k-issue1717/Lib/doctest.py Sat Dec 20 14:43:13 2008
@@ -813,7 +813,15 @@
# given object's docstring.
try:
file = inspect.getsourcefile(obj) or inspect.getfile(obj)
- source_lines = linecache.getlines(file)
+ if module is not None:
+ # Supply the module globals in case the module was
+ # originally loaded via a PEP 302 loader and
+ # file is not a valid filesystem path
+ source_lines = linecache.getlines(file, module.__dict__)
+ else:
+ # No access to a loader, so assume it's a normal
+ # filesystem path
+ source_lines = linecache.getlines(file)
if not source_lines:
source_lines = None
except TypeError:
@@ -1427,8 +1435,10 @@
d = self._name2ft
for name, (f, t) in other._name2ft.items():
if name in d:
- print("*** DocTestRunner.merge: '" + name + "' in both" \
- " testers; summing outcomes.")
+ # Don't print here by default, since doing
+ # so breaks some of the buildbots
+ #print("*** DocTestRunner.merge: '" + name + "' in both" \
+ # " testers; summing outcomes.")
f2, t2 = d[name]
f = f + f2
t = t + t2
Modified: python/branches/py3k-issue1717/Lib/http/client.py
==============================================================================
--- python/branches/py3k-issue1717/Lib/http/client.py (original)
+++ python/branches/py3k-issue1717/Lib/http/client.py Sat Dec 20 14:43:13 2008
@@ -359,8 +359,8 @@
if self.version == 9:
self.length = None
- self.chunked = 0
- self.will_close = 1
+ self.chunked = False
+ self.will_close = True
self.msg = email.message_from_string('')
return
@@ -373,10 +373,10 @@
# are we using the chunked-style of transfer encoding?
tr_enc = self.msg.get("transfer-encoding")
if tr_enc and tr_enc.lower() == "chunked":
- self.chunked = 1
+ self.chunked = True
self.chunk_left = None
else:
- self.chunked = 0
+ self.chunked = False
# will the connection close at the end of the response?
self.will_close = self._check_close()
@@ -411,7 +411,7 @@
if (not self.will_close and
not self.chunked and
self.length is None):
- self.will_close = 1
+ self.will_close = True
def _check_close(self):
conn = self.msg.get("connection")
Modified: python/branches/py3k-issue1717/Lib/io.py
==============================================================================
--- python/branches/py3k-issue1717/Lib/io.py (original)
+++ python/branches/py3k-issue1717/Lib/io.py Sat Dec 20 14:43:13 2008
@@ -1282,25 +1282,23 @@
"""
def __init__(self, decoder, translate, errors='strict'):
codecs.IncrementalDecoder.__init__(self, errors=errors)
- self.buffer = b''
self.translate = translate
self.decoder = decoder
self.seennl = 0
+ self.pendingcr = False
def decode(self, input, final=False):
# decode input (with the eventual \r from a previous pass)
- if self.buffer:
- input = self.buffer + input
-
output = self.decoder.decode(input, final=final)
+ if self.pendingcr and (output or final):
+ output = "\r" + output
+ self.pendingcr = False
# retain last \r even when not translating data:
# then readline() is sure to get \r\n in one pass
if output.endswith("\r") and not final:
output = output[:-1]
- self.buffer = b'\r'
- else:
- self.buffer = b''
+ self.pendingcr = True
# Record which newlines are read
crlf = output.count('\r\n')
@@ -1319,20 +1317,19 @@
def getstate(self):
buf, flag = self.decoder.getstate()
- return buf + self.buffer, flag
+ flag <<= 1
+ if self.pendingcr:
+ flag |= 1
+ return buf, flag
def setstate(self, state):
buf, flag = state
- if buf.endswith(b'\r'):
- self.buffer = b'\r'
- buf = buf[:-1]
- else:
- self.buffer = b''
- self.decoder.setstate((buf, flag))
+ self.pendingcr = bool(flag & 1)
+ self.decoder.setstate((buf, flag >> 1))
def reset(self):
self.seennl = 0
- self.buffer = b''
+ self.pendingcr = False
self.decoder.reset()
_LF = 1
@@ -1376,7 +1373,7 @@
write contains a newline character.
"""
- _CHUNK_SIZE = 128
+ _CHUNK_SIZE = 2048
def __init__(self, buffer, encoding=None, errors=None, newline=None,
line_buffering=False):
Modified: python/branches/py3k-issue1717/Lib/lib2to3/fixer_base.py
==============================================================================
--- python/branches/py3k-issue1717/Lib/lib2to3/fixer_base.py (original)
+++ python/branches/py3k-issue1717/Lib/lib2to3/fixer_base.py Sat Dec 20 14:43:13 2008
@@ -94,10 +94,6 @@
"""
raise NotImplementedError()
- def parenthesize(self, node):
- """Wrapper around pygram.parenthesize()."""
- return pygram.parenthesize(node)
-
def new_name(self, template="xxx_todo_changeme"):
"""Return a string suitable for use as an identifier
Modified: python/branches/py3k-issue1717/Lib/lib2to3/fixer_util.py
==============================================================================
--- python/branches/py3k-issue1717/Lib/lib2to3/fixer_util.py (original)
+++ python/branches/py3k-issue1717/Lib/lib2to3/fixer_util.py Sat Dec 20 14:43:13 2008
@@ -158,6 +158,9 @@
### Misc
###########################################################
+def parenthesize(node):
+ return Node(syms.atom, [LParen(), node, RParen()])
+
consuming_calls = set(["sorted", "list", "set", "any", "all", "tuple", "sum",
"min", "max"])
@@ -232,20 +235,77 @@
suite.parent = parent
return suite
-def does_tree_import(package, name, node):
- """ Returns true if name is imported from package at the
- top level of the tree which node belongs to.
- To cover the case of an import like 'import foo', use
- Null for the package and 'foo' for the name. """
+def find_root(node):
+ """Find the top level namespace."""
# Scamper up to the top level namespace
while node.type != syms.file_input:
assert node.parent, "Tree is insane! root found before "\
"file_input node was found."
node = node.parent
+ return node
- binding = find_binding(name, node, package)
+def does_tree_import(package, name, node):
+ """ Returns true if name is imported from package at the
+ top level of the tree which node belongs to.
+ To cover the case of an import like 'import foo', use
+ None for the package and 'foo' for the name. """
+ binding = find_binding(name, find_root(node), package)
return bool(binding)
+def is_import(node):
+ """Returns true if the node is an import statement."""
+ return node.type in (syms.import_name, syms.import_from)
+
+def touch_import(package, name, node):
+ """ Works like `does_tree_import` but adds an import statement
+ if it was not imported. """
+ def is_import_stmt(node):
+ return node.type == syms.simple_stmt and node.children and \
+ is_import(node.children[0])
+
+ root = find_root(node)
+
+ if does_tree_import(package, name, root):
+ return
+
+ add_newline_before = False
+
+ # figure out where to insert the new import. First try to find
+ # the first import and then skip to the last one.
+ insert_pos = offset = 0
+ for idx, node in enumerate(root.children):
+ if not is_import_stmt(node):
+ continue
+ for offset, node2 in enumerate(root.children[idx:]):
+ if not is_import_stmt(node2):
+ break
+ insert_pos = idx + offset
+ break
+
+ # if there are no imports where we can insert, find the docstring.
+ # if that also fails, we stick to the beginning of the file
+ if insert_pos == 0:
+ for idx, node in enumerate(root.children):
+ if node.type == syms.simple_stmt and node.children and \
+ node.children[0].type == token.STRING:
+ insert_pos = idx + 1
+ add_newline_before
+ break
+
+ if package is None:
+ import_ = Node(syms.import_name, [
+ Leaf(token.NAME, 'import'),
+ Leaf(token.NAME, name, prefix=' ')
+ ])
+ else:
+ import_ = FromImport(package, [Leaf(token.NAME, name, prefix=' ')])
+
+ children = [import_, Newline()]
+ if add_newline_before:
+ children.insert(0, Newline())
+ root.insert_child(insert_pos, Node(syms.simple_stmt, children))
+
+
_def_syms = set([syms.classdef, syms.funcdef])
def find_binding(name, node, package=None):
""" Returns the node which binds variable name, otherwise None.
@@ -285,7 +345,7 @@
if ret:
if not package:
return ret
- if ret.type in (syms.import_name, syms.import_from):
+ if is_import(ret):
return ret
return None
Modified: python/branches/py3k-issue1717/Lib/lib2to3/fixes/fix_apply.py
==============================================================================
--- python/branches/py3k-issue1717/Lib/lib2to3/fixes/fix_apply.py (original)
+++ python/branches/py3k-issue1717/Lib/lib2to3/fixes/fix_apply.py Sat Dec 20 14:43:13 2008
@@ -9,7 +9,7 @@
from .. import pytree
from ..pgen2 import token
from .. import fixer_base
-from ..fixer_util import Call, Comma
+from ..fixer_util import Call, Comma, parenthesize
class FixApply(fixer_base.BaseFix):
@@ -39,7 +39,7 @@
(func.type != syms.power or
func.children[-2].type == token.DOUBLESTAR)):
# Need to parenthesize
- func = self.parenthesize(func)
+ func = parenthesize(func)
func.set_prefix("")
args = args.clone()
args.set_prefix("")
Modified: python/branches/py3k-issue1717/Lib/lib2to3/fixes/fix_has_key.py
==============================================================================
--- python/branches/py3k-issue1717/Lib/lib2to3/fixes/fix_has_key.py (original)
+++ python/branches/py3k-issue1717/Lib/lib2to3/fixes/fix_has_key.py Sat Dec 20 14:43:13 2008
@@ -33,7 +33,7 @@
from .. import pytree
from ..pgen2 import token
from .. import fixer_base
-from ..fixer_util import Name
+from ..fixer_util import Name, parenthesize
class FixHasKey(fixer_base.BaseFix):
@@ -86,7 +86,7 @@
after = [n.clone() for n in after]
if arg.type in (syms.comparison, syms.not_test, syms.and_test,
syms.or_test, syms.test, syms.lambdef, syms.argument):
- arg = self.parenthesize(arg)
+ arg = parenthesize(arg)
if len(before) == 1:
before = before[0]
else:
@@ -98,12 +98,12 @@
n_op = pytree.Node(syms.comp_op, (n_not, n_op))
new = pytree.Node(syms.comparison, (arg, n_op, before))
if after:
- new = self.parenthesize(new)
+ new = parenthesize(new)
new = pytree.Node(syms.power, (new,) + tuple(after))
if node.parent.type in (syms.comparison, syms.expr, syms.xor_expr,
syms.and_expr, syms.shift_expr,
syms.arith_expr, syms.term,
syms.factor, syms.power):
- new = self.parenthesize(new)
+ new = parenthesize(new)
new.set_prefix(prefix)
return new
Modified: python/branches/py3k-issue1717/Lib/lib2to3/fixes/fix_imports.py
==============================================================================
--- python/branches/py3k-issue1717/Lib/lib2to3/fixes/fix_imports.py (original)
+++ python/branches/py3k-issue1717/Lib/lib2to3/fixes/fix_imports.py Sat Dec 20 14:43:13 2008
@@ -42,6 +42,8 @@
'DocXMLRPCServer': 'xmlrpc.server',
'SimpleXMLRPCServer': 'xmlrpc.server',
'httplib': 'http.client',
+ 'htmlentitydefs' : 'html.entities',
+ 'HTMLParser' : 'html.parser',
'Cookie': 'http.cookies',
'cookielib': 'http.cookiejar',
'BaseHTTPServer': 'http.server',
@@ -64,16 +66,17 @@
mod_list = ' | '.join(["module_name='%s'" % key for key in mapping])
bare_names = alternates(mapping.keys())
- yield """name_import=import_name< 'import' ((%s)
- | dotted_as_names< any* (%s) any* >) >
+ yield """name_import=import_name< 'import' ((%s) |
+ multiple_imports=dotted_as_names< any* (%s) any* >) >
""" % (mod_list, mod_list)
yield """import_from< 'from' (%s) 'import' ['(']
( any | import_as_name< any 'as' any > |
import_as_names< any* >) [')'] >
""" % mod_list
- yield """import_name< 'import'
- dotted_as_name< (%s) 'as' any > >
- """ % mod_list
+ yield """import_name< 'import' (dotted_as_name< (%s) 'as' any > |
+ multiple_imports=dotted_as_names<
+ any* dotted_as_name< (%s) 'as' any > any* >) >
+ """ % (mod_list, mod_list)
# Find usages of module members in code e.g. thread.foo(bar)
yield "power< bare_with_attr=(%s) trailer<'.' any > any* >" % bare_names
@@ -100,8 +103,8 @@
match = super(FixImports, self).match
results = match(node)
if results:
- # Module usage could be in the trailier of an attribute lookup, so
- # we might have nested matches when "bare_with_attr" is present.
+ # Module usage could be in the trailer of an attribute lookup, so we
+ # might have nested matches when "bare_with_attr" is present.
if "bare_with_attr" not in results and \
any([match(obj) for obj in attr_chain(node, "parent")]):
return False
@@ -116,11 +119,21 @@
import_mod = results.get("module_name")
if import_mod:
new_name = self.mapping[(import_mod or mod_name).value]
+ import_mod.replace(Name(new_name, prefix=import_mod.get_prefix()))
if "name_import" in results:
# If it's not a "from x import x, y" or "import x as y" import,
# marked its usage to be replaced.
self.replace[import_mod.value] = new_name
- import_mod.replace(Name(new_name, prefix=import_mod.get_prefix()))
+ if "multiple_imports" in results:
+ # This is a nasty hack to fix multiple imports on a
+ # line (e.g., "import StringIO, urlparse"). The problem is that I
+ # can't figure out an easy way to make a pattern recognize the
+ # keys of MAPPING randomly sprinkled in an import statement.
+ while True:
+ results = self.match(node)
+ if not results:
+ break
+ self.transform(node, results)
else:
# Replace usage of the module.
bare_name = results["bare_with_attr"][0]
Modified: python/branches/py3k-issue1717/Lib/lib2to3/fixes/fix_imports2.py
==============================================================================
--- python/branches/py3k-issue1717/Lib/lib2to3/fixes/fix_imports2.py (original)
+++ python/branches/py3k-issue1717/Lib/lib2to3/fixes/fix_imports2.py Sat Dec 20 14:43:13 2008
@@ -11,6 +11,6 @@
class FixImports2(fix_imports.FixImports):
- order = "post"
+ run_order = 6
mapping = MAPPING
Modified: python/branches/py3k-issue1717/Lib/lib2to3/fixes/fix_intern.py
==============================================================================
--- python/branches/py3k-issue1717/Lib/lib2to3/fixes/fix_intern.py (original)
+++ python/branches/py3k-issue1717/Lib/lib2to3/fixes/fix_intern.py Sat Dec 20 14:43:13 2008
@@ -8,7 +8,7 @@
# Local imports
from .. import pytree
from .. import fixer_base
-from ..fixer_util import Name, Attr
+from ..fixer_util import Name, Attr, touch_import
class FixIntern(fixer_base.BaseFix):
@@ -40,4 +40,5 @@
newarglist,
results["rpar"].clone()])] + after)
new.set_prefix(node.get_prefix())
+ touch_import(None, 'sys', node)
return new
Modified: python/branches/py3k-issue1717/Lib/lib2to3/fixes/fix_long.py
==============================================================================
--- python/branches/py3k-issue1717/Lib/lib2to3/fixes/fix_long.py (original)
+++ python/branches/py3k-issue1717/Lib/lib2to3/fixes/fix_long.py Sat Dec 20 14:43:13 2008
@@ -2,8 +2,6 @@
# Licensed to PSF under a Contributor Agreement.
"""Fixer that turns 'long' into 'int' everywhere.
-
-This also strips the trailing 'L' or 'l' from long loterals.
"""
# Local imports
@@ -14,22 +12,13 @@
class FixLong(fixer_base.BaseFix):
- PATTERN = """
- (long_type = 'long' | number = NUMBER)
- """
+ PATTERN = "'long'"
static_long = Name("long")
static_int = Name("int")
def transform(self, node, results):
- long_type = results.get("long_type")
- number = results.get("number")
- new = None
- if long_type:
- assert node == self.static_long, node
- new = self.static_int.clone()
- if number and node.value[-1] in ("l", "L"):
- new = Number(node.value[:-1])
- if new is not None:
- new.set_prefix(node.get_prefix())
- return new
+ assert node == self.static_long, node
+ new = self.static_int.clone()
+ new.set_prefix(node.get_prefix())
+ return new
Modified: python/branches/py3k-issue1717/Lib/lib2to3/fixes/fix_repr.py
==============================================================================
--- python/branches/py3k-issue1717/Lib/lib2to3/fixes/fix_repr.py (original)
+++ python/branches/py3k-issue1717/Lib/lib2to3/fixes/fix_repr.py Sat Dec 20 14:43:13 2008
@@ -5,7 +5,7 @@
# Local imports
from .. import fixer_base
-from ..fixer_util import Call, Name
+from ..fixer_util import Call, Name, parenthesize
class FixRepr(fixer_base.BaseFix):
@@ -18,5 +18,5 @@
expr = results["expr"].clone()
if expr.type == self.syms.testlist1:
- expr = self.parenthesize(expr)
+ expr = parenthesize(expr)
return Call(Name("repr"), [expr], prefix=node.get_prefix())
Modified: python/branches/py3k-issue1717/Lib/lib2to3/fixes/fix_urllib.py
==============================================================================
--- python/branches/py3k-issue1717/Lib/lib2to3/fixes/fix_urllib.py (original)
+++ python/branches/py3k-issue1717/Lib/lib2to3/fixes/fix_urllib.py Sat Dec 20 14:43:13 2008
@@ -29,7 +29,7 @@
'AbstractBasicAuthHandler',
'HTTPBasicAuthHandler', 'ProxyBasicAuthHandler',
'AbstractDigestAuthHandler',
- 'HTTPDigestAuthHander', 'ProxyDigestAuthHandler',
+ 'HTTPDigestAuthHandler', 'ProxyDigestAuthHandler',
'HTTPHandler', 'HTTPSHandler', 'FileHandler',
'FTPHandler', 'CacheFTPHandler',
'UnknownHandler']),
Modified: python/branches/py3k-issue1717/Lib/lib2to3/fixes/fix_xrange.py
==============================================================================
--- python/branches/py3k-issue1717/Lib/lib2to3/fixes/fix_xrange.py (original)
+++ python/branches/py3k-issue1717/Lib/lib2to3/fixes/fix_xrange.py Sat Dec 20 14:43:13 2008
@@ -12,7 +12,9 @@
class FixXrange(fixer_base.BaseFix):
PATTERN = """
- power< (name='range'|name='xrange') trailer< '(' [any] ')' > any* >
+ power<
+ (name='range'|name='xrange') trailer< '(' args=any ')' >
+ rest=any* >
"""
def transform(self, node, results):
@@ -30,11 +32,14 @@
def transform_range(self, node, results):
if not self.in_special_context(node):
- arg = node.clone()
- arg.set_prefix("")
- call = Call(Name("list"), [arg])
- call.set_prefix(node.get_prefix())
- return call
+ range_call = Call(Name("range"), [results["args"].clone()])
+ # Encase the range call in list().
+ list_call = Call(Name("list"), [range_call],
+ prefix=node.get_prefix())
+ # Put things that were after the range() call after the list call.
+ for n in results["rest"]:
+ list_call.append_child(n)
+ return list_call
return node
P1 = "power< func=NAME trailer< '(' node=any ')' > any* >"
Modified: python/branches/py3k-issue1717/Lib/lib2to3/main.py
==============================================================================
--- python/branches/py3k-issue1717/Lib/lib2to3/main.py (original)
+++ python/branches/py3k-issue1717/Lib/lib2to3/main.py Sat Dec 20 14:43:13 2008
@@ -5,6 +5,7 @@
import sys
import os
import logging
+import shutil
import optparse
from . import refactor
@@ -39,6 +40,7 @@
# Actually write the new file
super(StdoutRefactoringTool, self).write_file(new_text,
filename, old_text)
+ shutil.copymode(filename, backup)
def print_output(self, lines):
for line in lines:
@@ -56,7 +58,7 @@
Returns a suggested exit status (0, 1, 2).
"""
# Set up option parser
- parser = optparse.OptionParser(usage="refactor.py [options] file|dir ...")
+ parser = optparse.OptionParser(usage="2to3 [options] file|dir ...")
parser.add_option("-d", "--doctests_only", action="store_true",
help="Fix up doctests only")
parser.add_option("-f", "--fix", action="append", default=[],
Modified: python/branches/py3k-issue1717/Lib/lib2to3/pygram.py
==============================================================================
--- python/branches/py3k-issue1717/Lib/lib2to3/pygram.py (original)
+++ python/branches/py3k-issue1717/Lib/lib2to3/pygram.py Sat Dec 20 14:43:13 2008
@@ -29,10 +29,3 @@
python_grammar = driver.load_grammar(_GRAMMAR_FILE)
python_symbols = Symbols(python_grammar)
-
-
-def parenthesize(node):
- return pytree.Node(python_symbols.atom,
- (pytree.Leaf(token.LPAR, "("),
- node,
- pytree.Leaf(token.RPAR, ")")))
Modified: python/branches/py3k-issue1717/Lib/lib2to3/pytree.py
==============================================================================
--- python/branches/py3k-issue1717/Lib/lib2to3/pytree.py (original)
+++ python/branches/py3k-issue1717/Lib/lib2to3/pytree.py Sat Dec 20 14:43:13 2008
@@ -279,18 +279,21 @@
child.parent = self
self.children[i].parent = None
self.children[i] = child
+ self.changed()
def insert_child(self, i, child):
"""Equivalent to 'node.children.insert(i, child)'. This method also
sets the child's parent attribute appropriately."""
child.parent = self
self.children.insert(i, child)
+ self.changed()
def append_child(self, child):
"""Equivalent to 'node.children.append(child)'. This method also
sets the child's parent attribute appropriately."""
child.parent = self
self.children.append(child)
+ self.changed()
class Leaf(Base):
Modified: python/branches/py3k-issue1717/Lib/lib2to3/refactor.py
==============================================================================
--- python/branches/py3k-issue1717/Lib/lib2to3/refactor.py (original)
+++ python/branches/py3k-issue1717/Lib/lib2to3/refactor.py Sat Dec 20 14:43:13 2008
@@ -123,8 +123,8 @@
logger=self.logger)
self.pre_order, self.post_order = self.get_fixers()
- self.pre_order_mapping = get_headnode_dict(self.pre_order)
- self.post_order_mapping = get_headnode_dict(self.post_order)
+ self.pre_order_heads = get_headnode_dict(self.pre_order)
+ self.post_order_heads = get_headnode_dict(self.post_order)
self.files = [] # List of files that were or should be modified
@@ -294,8 +294,8 @@
for fixer in all_fixers:
fixer.start_tree(tree, name)
- self.traverse_by(self.pre_order_mapping, tree.pre_order())
- self.traverse_by(self.post_order_mapping, tree.post_order())
+ self.traverse_by(self.pre_order_heads, tree.pre_order())
+ self.traverse_by(self.post_order_heads, tree.post_order())
for fixer in all_fixers:
fixer.finish_tree(tree, name)
Deleted: python/branches/py3k-issue1717/Lib/lib2to3/tests/benchmark.py
==============================================================================
--- python/branches/py3k-issue1717/Lib/lib2to3/tests/benchmark.py Sat Dec 20 14:43:13 2008
+++ (empty file)
@@ -1,58 +0,0 @@
-#!/usr/bin/env python2.5
-"""
-This is a benchmarking script to test the speed of 2to3's pattern matching
-system. It's equivalent to "refactor.py -f all" for every Python module
-in sys.modules, but without engaging the actual transformations.
-"""
-
-__author__ = "Collin Winter <collinw at gmail.com>"
-
-# Python imports
-import os.path
-import sys
-from time import time
-
-# Test imports
-from .support import adjust_path
-adjust_path()
-
-# Local imports
-from .. import refactor
-
-### Mock code for refactor.py and the fixers
-###############################################################################
-class Options:
- def __init__(self, **kwargs):
- for k, v in list(kwargs.items()):
- setattr(self, k, v)
-
- self.verbose = False
-
-def dummy_transform(*args, **kwargs):
- pass
-
-### Collect list of modules to match against
-###############################################################################
-files = []
-for mod in list(sys.modules.values()):
- if mod is None or not hasattr(mod, '__file__'):
- continue
- f = mod.__file__
- if f.endswith('.pyc'):
- f = f[:-1]
- if f.endswith('.py'):
- files.append(f)
-
-### Set up refactor and run the benchmark
-###############################################################################
-options = Options(fix=["all"], print_function=False, doctests_only=False)
-refactor = refactor.RefactoringTool(options)
-for fixer in refactor.fixers:
- # We don't want them to actually fix the tree, just match against it.
- fixer.transform = dummy_transform
-
-t = time()
-for f in files:
- print("Matching", f)
- refactor.refactor_file(f)
-print("%d seconds to match %d files" % (time() - t, len(sys.modules)))
Modified: python/branches/py3k-issue1717/Lib/lib2to3/tests/test_fixers.py
==============================================================================
--- python/branches/py3k-issue1717/Lib/lib2to3/tests/test_fixers.py (original)
+++ python/branches/py3k-issue1717/Lib/lib2to3/tests/test_fixers.py Sat Dec 20 14:43:13 2008
@@ -293,30 +293,30 @@
def test_prefix_preservation(self):
b = """x = intern( a )"""
- a = """x = sys.intern( a )"""
+ a = """import sys\nx = sys.intern( a )"""
self.check(b, a)
b = """y = intern("b" # test
)"""
- a = """y = sys.intern("b" # test
+ a = """import sys\ny = sys.intern("b" # test
)"""
self.check(b, a)
b = """z = intern(a+b+c.d, )"""
- a = """z = sys.intern(a+b+c.d, )"""
+ a = """import sys\nz = sys.intern(a+b+c.d, )"""
self.check(b, a)
def test(self):
b = """x = intern(a)"""
- a = """x = sys.intern(a)"""
+ a = """import sys\nx = sys.intern(a)"""
self.check(b, a)
b = """z = intern(a+b+c.d,)"""
- a = """z = sys.intern(a+b+c.d,)"""
+ a = """import sys\nz = sys.intern(a+b+c.d,)"""
self.check(b, a)
b = """intern("y%s" % 5).replace("y", "")"""
- a = """sys.intern("y%s" % 5).replace("y", "")"""
+ a = """import sys\nsys.intern("y%s" % 5).replace("y", "")"""
self.check(b, a)
# These should not be refactored
@@ -337,6 +337,35 @@
s = """intern()"""
self.unchanged(s)
+class Test_reduce(FixerTestCase):
+ fixer = "reduce"
+
+ def test_simple_call(self):
+ b = "reduce(a, b, c)"
+ a = "from functools import reduce\nreduce(a, b, c)"
+ self.check(b, a)
+
+ def test_call_with_lambda(self):
+ b = "reduce(lambda x, y: x + y, seq)"
+ a = "from functools import reduce\nreduce(lambda x, y: x + y, seq)"
+ self.check(b, a)
+
+ def test_unchanged(self):
+ s = "reduce(a)"
+ self.unchanged(s)
+
+ s = "reduce(a, b=42)"
+ self.unchanged(s)
+
+ s = "reduce(a, b, c, d)"
+ self.unchanged(s)
+
+ s = "reduce(**c)"
+ self.unchanged(s)
+
+ s = "reduce()"
+ self.unchanged(s)
+
class Test_print(FixerTestCase):
fixer = "print"
@@ -1044,33 +1073,39 @@
a = """z = type(x) in (int, int)"""
self.check(b, a)
- def test_4(self):
- b = """a = 12L"""
- a = """a = 12"""
+ def test_prefix_preservation(self):
+ b = """x = long( x )"""
+ a = """x = int( x )"""
self.check(b, a)
- def test_5(self):
- b = """b = 0x12l"""
- a = """b = 0x12"""
+class Test_isinstance(FixerTestCase):
+ fixer = "isinstance"
+
+ def test_remove_multiple_items(self):
+ b = """isinstance(x, (int, int, int))"""
+ a = """isinstance(x, int)"""
self.check(b, a)
- def test_unchanged_1(self):
- s = """a = 12"""
- self.unchanged(s)
+ b = """isinstance(x, (int, float, int, int, float))"""
+ a = """isinstance(x, (int, float))"""
+ self.check(b, a)
- def test_unchanged_2(self):
- s = """b = 0x12"""
- self.unchanged(s)
+ b = """isinstance(x, (int, float, int, int, float, str))"""
+ a = """isinstance(x, (int, float, str))"""
+ self.check(b, a)
- def test_unchanged_3(self):
- s = """c = 3.14"""
- self.unchanged(s)
+ b = """isinstance(foo() + bar(), (x(), y(), x(), int, int))"""
+ a = """isinstance(foo() + bar(), (x(), y(), x(), int))"""
+ self.check(b, a)
def test_prefix_preservation(self):
- b = """x = long( x )"""
- a = """x = int( x )"""
+ b = """if isinstance( foo(), ( bar, bar, baz )) : pass"""
+ a = """if isinstance( foo(), ( bar, baz )) : pass"""
self.check(b, a)
+ def test_unchanged(self):
+ self.unchanged("isinstance(x, (str, int))")
+
class Test_dict(FixerTestCase):
fixer = "dict"
@@ -1287,6 +1322,14 @@
a = """x = list(range(10, 3, 9)) + [4]"""
self.check(b, a)
+ b = """x = range(10)[::-1]"""
+ a = """x = list(range(10))[::-1]"""
+ self.check(b, a)
+
+ b = """x = range(10) [3]"""
+ a = """x = list(range(10)) [3]"""
+ self.check(b, a)
+
def test_xrange_in_for(self):
b = """for i in xrange(10):\n j=i"""
a = """for i in range(10):\n j=i"""
@@ -1422,9 +1465,8 @@
s = "foo(xreadlines)"
self.unchanged(s)
-class Test_imports(FixerTestCase):
- fixer = "imports"
- from ..fixes.fix_imports import MAPPING as modules
+
+class ImportsFixerTests:
def test_import_module(self):
for old, new in self.modules.items():
@@ -1522,18 +1564,36 @@
self.check(b, a)
+class Test_imports(FixerTestCase, ImportsFixerTests):
+ fixer = "imports"
+ from ..fixes.fix_imports import MAPPING as modules
+
+ def test_multiple_imports(self):
+ b = """import urlparse, cStringIO"""
+ a = """import urllib.parse, io"""
+ self.check(b, a)
+
+ def test_multiple_imports_as(self):
+ b = """
+ import copy_reg as bar, HTMLParser as foo, urlparse
+ s = urlparse.spam(bar.foo())
+ """
+ a = """
+ import copyreg as bar, html.parser as foo, urllib.parse
+ s = urllib.parse.spam(bar.foo())
+ """
+ self.check(b, a)
+
-class Test_imports2(Test_imports):
+class Test_imports2(FixerTestCase, ImportsFixerTests):
fixer = "imports2"
from ..fixes.fix_imports2 import MAPPING as modules
-class Test_imports_fixer_order(Test_imports):
-
- fixer = None
+class Test_imports_fixer_order(FixerTestCase, ImportsFixerTests):
def setUp(self):
- Test_imports.setUp(self, ['imports', 'imports2'])
+ super(Test_imports_fixer_order, self).setUp(['imports', 'imports2'])
from ..fixes.fix_imports2 import MAPPING as mapping2
self.modules = mapping2.copy()
from ..fixes.fix_imports import MAPPING as mapping1
Modified: python/branches/py3k-issue1717/Lib/lib2to3/tests/test_util.py
==============================================================================
--- python/branches/py3k-issue1717/Lib/lib2to3/tests/test_util.py (original)
+++ python/branches/py3k-issue1717/Lib/lib2to3/tests/test_util.py Sat Dec 20 14:43:13 2008
@@ -526,6 +526,33 @@
b = 7"""
self.failIf(self.find_binding("a", s))
+class Test_touch_import(support.TestCase):
+
+ def test_after_docstring(self):
+ node = parse('"""foo"""\nbar()')
+ fixer_util.touch_import(None, "foo", node)
+ self.assertEqual(str(node), '"""foo"""\nimport foo\nbar()\n\n')
+
+ def test_after_imports(self):
+ node = parse('"""foo"""\nimport bar\nbar()')
+ fixer_util.touch_import(None, "foo", node)
+ self.assertEqual(str(node), '"""foo"""\nimport bar\nimport foo\nbar()\n\n')
+
+ def test_beginning(self):
+ node = parse('bar()')
+ fixer_util.touch_import(None, "foo", node)
+ self.assertEqual(str(node), 'import foo\nbar()\n\n')
+
+ def test_from_import(self):
+ node = parse('bar()')
+ fixer_util.touch_import("cgi", "escape", node)
+ self.assertEqual(str(node), 'from cgi import escape\nbar()\n\n')
+
+ def test_name_import(self):
+ node = parse('bar()')
+ fixer_util.touch_import(None, "cgi", node)
+ self.assertEqual(str(node), 'import cgi\nbar()\n\n')
+
if __name__ == "__main__":
import __main__
Modified: python/branches/py3k-issue1717/Lib/linecache.py
==============================================================================
--- python/branches/py3k-issue1717/Lib/linecache.py (original)
+++ python/branches/py3k-issue1717/Lib/linecache.py Sat Dec 20 14:43:13 2008
@@ -89,21 +89,20 @@
get_source = getattr(loader, 'get_source', None)
if name and get_source:
- if basename.startswith(name.split('.')[-1]+'.'):
- try:
- data = get_source(name)
- except (ImportError, IOError):
- pass
- else:
- if data is None:
- # No luck, the PEP302 loader cannot find the source
- # for this module.
- return []
- cache[filename] = (
- len(data), None,
- [line+'\n' for line in data.splitlines()], fullname
- )
- return cache[filename][2]
+ try:
+ data = get_source(name)
+ except (ImportError, IOError):
+ pass
+ else:
+ if data is None:
+ # No luck, the PEP302 loader cannot find the source
+ # for this module.
+ return []
+ cache[filename] = (
+ len(data), None,
+ [line+'\n' for line in data.splitlines()], fullname
+ )
+ return cache[filename][2]
# Try looking through the module search path.
Modified: python/branches/py3k-issue1717/Lib/opcode.py
==============================================================================
--- python/branches/py3k-issue1717/Lib/opcode.py (original)
+++ python/branches/py3k-issue1717/Lib/opcode.py Sat Dec 20 14:43:13 2008
@@ -57,8 +57,6 @@
def_op('UNARY_INVERT', 15)
-def_op('SET_ADD', 17)
-def_op('LIST_APPEND', 18)
def_op('BINARY_POWER', 19)
def_op('BINARY_MULTIPLY', 20)
@@ -169,4 +167,9 @@
def_op('EXTENDED_ARG', 143)
EXTENDED_ARG = 143
+def_op('LIST_APPEND', 145)
+def_op('SET_ADD', 146)
+def_op('MAP_ADD', 147)
+
+
del def_op, name_op, jrel_op, jabs_op
Modified: python/branches/py3k-issue1717/Lib/pdb.py
==============================================================================
--- python/branches/py3k-issue1717/Lib/pdb.py (original)
+++ python/branches/py3k-issue1717/Lib/pdb.py Sat Dec 20 14:43:13 2008
@@ -438,7 +438,7 @@
Return `lineno` if it is, 0 if not (e.g. a docstring, comment, blank
line or EOF). Warning: testing is not comprehensive.
"""
- line = linecache.getline(filename, lineno)
+ line = linecache.getline(filename, lineno, self.curframe.f_globals)
if not line:
print('End of file', file=self.stdout)
return 0
@@ -768,7 +768,7 @@
breaklist = self.get_file_breaks(filename)
try:
for lineno in range(first, last+1):
- line = linecache.getline(filename, lineno)
+ line = linecache.getline(filename, lineno, self.curframe.f_globals)
if not line:
print('[EOF]', file=self.stdout)
break
Modified: python/branches/py3k-issue1717/Lib/runpy.py
==============================================================================
--- python/branches/py3k-issue1717/Lib/runpy.py (original)
+++ python/branches/py3k-issue1717/Lib/runpy.py Sat Dec 20 14:43:13 2008
@@ -65,13 +65,14 @@
# This helper is needed due to a missing component in the PEP 302
# loader protocol (specifically, "get_filename" is non-standard)
+# Since we can't introduce new features in maintenance releases,
+# support was added to zipimporter under the name '_get_filename'
def _get_filename(loader, mod_name):
- try:
- get_filename = loader.get_filename
- except AttributeError:
- return None
- else:
- return get_filename(mod_name)
+ for attr in ("get_filename", "_get_filename"):
+ meth = getattr(loader, attr, None)
+ if meth is not None:
+ return meth(mod_name)
+ return None
# Helper to get the loader, code and filename for a module
def _get_module_details(mod_name):
Modified: python/branches/py3k-issue1717/Lib/tarfile.py
==============================================================================
--- python/branches/py3k-issue1717/Lib/tarfile.py (original)
+++ python/branches/py3k-issue1717/Lib/tarfile.py Sat Dec 20 14:43:13 2008
@@ -2265,10 +2265,6 @@
"""
if not hasattr(os, 'utime'):
return
- if sys.platform == "win32" and tarinfo.isdir():
- # According to msdn.microsoft.com, it is an error (EACCES)
- # to use utime() on directories.
- return
try:
os.utime(targetpath, (tarinfo.mtime, tarinfo.mtime))
except EnvironmentError as e:
Modified: python/branches/py3k-issue1717/Lib/test/decimaltestdata/extra.decTest
==============================================================================
--- python/branches/py3k-issue1717/Lib/test/decimaltestdata/extra.decTest (original)
+++ python/branches/py3k-issue1717/Lib/test/decimaltestdata/extra.decTest Sat Dec 20 14:43:13 2008
@@ -154,6 +154,23 @@
extr1302 fma 0E123 -Inf sNaN789 -> NaN Invalid_operation
extr1302 fma -Inf 0E-456 sNaN148 -> NaN Invalid_operation
+-- max/min/max_mag/min_mag bug in 2.5.2/2.6/3.0: max(NaN, finite) gave
+-- incorrect answers when the finite number required rounding; similarly
+-- for the other thre functions
+maxexponent: 999
+minexponent: -999
+precision: 6
+rounding: half_even
+extr1400 max NaN 1234567 -> 1.23457E+6 Inexact Rounded
+extr1401 max 3141590E-123 NaN1729 -> 3.14159E-117 Rounded
+extr1402 max -7.654321 -NaN -> -7.65432 Inexact Rounded
+extr1410 min -NaN -765432.1 -> -765432 Inexact Rounded
+extr1411 min 3141592 NaN -> 3.14159E+6 Inexact Rounded
+extr1420 max_mag 0.1111111 -NaN123 -> 0.111111 Inexact Rounded
+extr1421 max_mag NaN999999999 0.001234567 -> 0.00123457 Inexact Rounded
+extr1430 min_mag 9181716151 -NaN -> 9.18172E+9 Inexact Rounded
+extr1431 min_mag NaN4 1.818180E100 -> 1.81818E+100 Rounded
+
-- Tests for the is_* boolean operations
precision: 9
maxExponent: 999
Modified: python/branches/py3k-issue1717/Lib/test/test_array.py
==============================================================================
--- python/branches/py3k-issue1717/Lib/test/test_array.py (original)
+++ python/branches/py3k-issue1717/Lib/test/test_array.py Sat Dec 20 14:43:13 2008
@@ -8,6 +8,7 @@
from weakref import proxy
import array, io, math
from pickle import loads, dumps
+import operator
class ArraySubclass(array.array):
pass
@@ -709,8 +710,23 @@
def test_buffer(self):
a = array.array(self.typecode, self.example)
- b = bytes(memoryview(a))
+ m = memoryview(a)
+ b = bytes(m)
+ self.assertEqual(b, a.tostring())
self.assertEqual(b[0], a.tostring()[0])
+ # Resizing is forbidden when there are buffer exports
+ self.assertRaises(BufferError, a.append, a[0])
+ self.assertRaises(BufferError, a.extend, a[0:1])
+ self.assertRaises(BufferError, a.remove, a[0])
+ self.assertRaises(BufferError, a.fromlist, a.tolist())
+ self.assertRaises(BufferError, a.fromstring, a.tostring())
+ if self.typecode == 'u':
+ self.assertRaises(BufferError, a.fromunicode, a.tounicode())
+ self.assertRaises(BufferError, operator.setitem, a, slice(0, 0), a)
+ self.assertRaises(BufferError, operator.delitem, a, 0)
+ self.assertRaises(BufferError, operator.delitem, a, slice(0, 1))
+ self.assertRaises(BufferError, operator.irepeat, a, 2)
+ self.assertRaises(BufferError, operator.irepeat, a, 0)
def test_weakref(self):
s = array.array(self.typecode, self.example)
Modified: python/branches/py3k-issue1717/Lib/test/test_cmd_line_script.py
==============================================================================
--- python/branches/py3k-issue1717/Lib/test/test_cmd_line_script.py (original)
+++ python/branches/py3k-issue1717/Lib/test/test_cmd_line_script.py Sat Dec 20 14:43:13 2008
@@ -75,36 +75,66 @@
compiled_name = script_name + 'o'
return compiled_name
-def _make_test_zip(zip_dir, zip_basename, script_name):
+def _make_test_zip(zip_dir, zip_basename, script_name, name_in_zip=None):
zip_filename = zip_basename+os.path.extsep+"zip"
zip_name = os.path.join(zip_dir, zip_filename)
zip_file = zipfile.ZipFile(zip_name, 'w')
- zip_file.write(script_name, os.path.basename(script_name))
+ if name_in_zip is None:
+ name_in_zip = os.path.basename(script_name)
+ zip_file.write(script_name, name_in_zip)
zip_file.close()
- # if verbose:
+ #if verbose:
# zip_file = zipfile.ZipFile(zip_name, 'r')
# print("Contents of %r:" % zip_name)
# zip_file.printdir()
# zip_file.close()
- return zip_name
+ return zip_name, os.path.join(zip_name, name_in_zip)
def _make_test_pkg(pkg_dir):
os.mkdir(pkg_dir)
_make_test_script(pkg_dir, '__init__', '')
+def _make_test_zip_pkg(zip_dir, zip_basename, pkg_name, script_basename,
+ source=test_source, depth=1):
+ init_name = _make_test_script(zip_dir, '__init__', '')
+ init_basename = os.path.basename(init_name)
+ script_name = _make_test_script(zip_dir, script_basename, source)
+ pkg_names = [os.sep.join([pkg_name]*i) for i in range(1, depth+1)]
+ script_name_in_zip = os.path.join(pkg_names[-1], os.path.basename(script_name))
+ zip_filename = zip_basename+os.extsep+'zip'
+ zip_name = os.path.join(zip_dir, zip_filename)
+ zip_file = zipfile.ZipFile(zip_name, 'w')
+ for name in pkg_names:
+ init_name_in_zip = os.path.join(name, init_basename)
+ zip_file.write(init_name, init_name_in_zip)
+ zip_file.write(script_name, script_name_in_zip)
+ zip_file.close()
+ os.unlink(init_name)
+ os.unlink(script_name)
+ #if verbose:
+ # zip_file = zipfile.ZipFile(zip_name, 'r')
+ # print 'Contents of %r:' % zip_name
+ # zip_file.printdir()
+ # zip_file.close()
+ return zip_name, os.path.join(zip_name, script_name_in_zip)
+
# There's no easy way to pass the script directory in to get
# -m to work (avoiding that is the whole point of making
# directories and zipfiles executable!)
# So we fake it for testing purposes with a custom launch script
launch_source = """\
import sys, os.path, runpy
-sys.path[0:0] = os.path.dirname(__file__)
+sys.path.insert(0, %s)
runpy._run_module_as_main(%r)
"""
-def _make_launch_script(script_dir, script_basename, module_name):
- return _make_test_script(script_dir, script_basename,
- launch_source % module_name)
+def _make_launch_script(script_dir, script_basename, module_name, path=None):
+ if path is None:
+ path = "os.path.dirname(__file__)"
+ else:
+ path = repr(path)
+ source = launch_source % (path, module_name)
+ return _make_test_script(script_dir, script_basename, source)
class CmdLineTest(unittest.TestCase):
def _check_script(self, script_name, expected_file,
@@ -155,15 +185,15 @@
def test_zipfile(self):
with temp_dir() as script_dir:
script_name = _make_test_script(script_dir, '__main__')
- zip_name = _make_test_zip(script_dir, 'test_zip', script_name)
- self._check_script(zip_name, None, zip_name, '')
+ zip_name, run_name = _make_test_zip(script_dir, 'test_zip', script_name)
+ self._check_script(zip_name, run_name, zip_name, '')
def test_zipfile_compiled(self):
with temp_dir() as script_dir:
script_name = _make_test_script(script_dir, '__main__')
compiled_name = _compile_test_script(script_name)
- zip_name = _make_test_zip(script_dir, 'test_zip', compiled_name)
- self._check_script(zip_name, None, zip_name, '')
+ zip_name, run_name = _make_test_zip(script_dir, 'test_zip', compiled_name)
+ self._check_script(zip_name, run_name, zip_name, '')
def test_module_in_package(self):
with temp_dir() as script_dir:
@@ -171,8 +201,19 @@
_make_test_pkg(pkg_dir)
script_name = _make_test_script(pkg_dir, 'script')
launch_name = _make_launch_script(script_dir, 'launch', 'test_pkg.script')
- self._check_script(launch_name, script_name,
- script_name, 'test_pkg')
+ self._check_script(launch_name, script_name, script_name, 'test_pkg')
+
+ def test_module_in_package_in_zipfile(self):
+ with temp_dir() as script_dir:
+ zip_name, run_name = _make_test_zip_pkg(script_dir, 'test_zip', 'test_pkg', 'script')
+ launch_name = _make_launch_script(script_dir, 'launch', 'test_pkg.script', zip_name)
+ self._check_script(launch_name, run_name, run_name, 'test_pkg')
+
+ def test_module_in_subpackage_in_zipfile(self):
+ with temp_dir() as script_dir:
+ zip_name, run_name = _make_test_zip_pkg(script_dir, 'test_zip', 'test_pkg', 'script', depth=2)
+ launch_name = _make_launch_script(script_dir, 'launch', 'test_pkg.test_pkg.script', zip_name)
+ self._check_script(launch_name, run_name, run_name, 'test_pkg.test_pkg')
def test_main():
Modified: python/branches/py3k-issue1717/Lib/test/test_dis.py
==============================================================================
--- python/branches/py3k-issue1717/Lib/test/test_dis.py (original)
+++ python/branches/py3k-issue1717/Lib/test/test_dis.py Sat Dec 20 14:43:13 2008
@@ -55,29 +55,25 @@
dis_bug1333982 = """\
%-4d 0 LOAD_CONST 1 (0)
- 3 JUMP_IF_TRUE 41 (to 47)
+ 3 JUMP_IF_TRUE 33 (to 39)
6 POP_TOP
7 LOAD_GLOBAL 0 (AssertionError)
10 BUILD_LIST 0
- 13 DUP_TOP
- 14 STORE_FAST 1 (_[1])
- 17 LOAD_FAST 0 (x)
- 20 GET_ITER
- >> 21 FOR_ITER 13 (to 37)
- 24 STORE_FAST 2 (s)
- 27 LOAD_FAST 1 (_[1])
- 30 LOAD_FAST 2 (s)
- 33 LIST_APPEND
- 34 JUMP_ABSOLUTE 21
- >> 37 DELETE_FAST 1 (_[1])
-
- %-4d 40 LOAD_CONST 2 (1)
- 43 BINARY_ADD
- 44 RAISE_VARARGS 2
- >> 47 POP_TOP
+ 13 LOAD_FAST 0 (x)
+ 16 GET_ITER
+ >> 17 FOR_ITER 12 (to 32)
+ 20 STORE_FAST 1 (s)
+ 23 LOAD_FAST 1 (s)
+ 26 LIST_APPEND 2
+ 29 JUMP_ABSOLUTE 17
+
+ %-4d >> 32 LOAD_CONST 2 (1)
+ 35 BINARY_ADD
+ 36 RAISE_VARARGS 2
+ >> 39 POP_TOP
- %-4d 48 LOAD_CONST 0 (None)
- 51 RETURN_VALUE
+ %-4d 40 LOAD_CONST 0 (None)
+ 43 RETURN_VALUE
"""%(bug1333982.__code__.co_firstlineno + 1,
bug1333982.__code__.co_firstlineno + 2,
bug1333982.__code__.co_firstlineno + 3)
Modified: python/branches/py3k-issue1717/Lib/test/test_doctest.py
==============================================================================
--- python/branches/py3k-issue1717/Lib/test/test_doctest.py (original)
+++ python/branches/py3k-issue1717/Lib/test/test_doctest.py Sat Dec 20 14:43:13 2008
@@ -6,6 +6,9 @@
import doctest
import warnings
+# NOTE: There are some additional tests relating to interaction with
+# zipimport in the test_zipimport_support test module.
+
######################################################################
## Sample Objects (used by test cases)
######################################################################
@@ -369,7 +372,7 @@
>>> tests = finder.find(sample_func)
>>> print(tests) # doctest: +ELLIPSIS
- [<DocTest sample_func from ...:13 (1 example)>]
+ [<DocTest sample_func from ...:16 (1 example)>]
The exact name depends on how test_doctest was invoked, so allow for
leading path components.
Modified: python/branches/py3k-issue1717/Lib/test/test_httplib.py
==============================================================================
--- python/branches/py3k-issue1717/Lib/test/test_httplib.py (original)
+++ python/branches/py3k-issue1717/Lib/test/test_httplib.py Sat Dec 20 14:43:13 2008
@@ -188,7 +188,8 @@
resp.close()
def test_negative_content_length(self):
- sock = FakeSocket('HTTP/1.1 200 OK\r\nContent-Length: -1\r\n\r\nHello\r\n')
+ sock = FakeSocket(
+ 'HTTP/1.1 200 OK\r\nContent-Length: -1\r\n\r\nHello\r\n')
resp = httplib.HTTPResponse(sock, method="GET")
resp.begin()
self.assertEquals(resp.read(), b'Hello\r\n')
Modified: python/branches/py3k-issue1717/Lib/test/test_inspect.py
==============================================================================
--- python/branches/py3k-issue1717/Lib/test/test_inspect.py (original)
+++ python/branches/py3k-issue1717/Lib/test/test_inspect.py Sat Dec 20 14:43:13 2008
@@ -18,6 +18,9 @@
# getclasstree, getargspec, getargvalues, formatargspec, formatargvalues,
# currentframe, stack, trace, isdatadescriptor
+# NOTE: There are some additional tests relating to interaction with
+# zipimport in the test_zipimport_support test module.
+
modfile = mod.__file__
if modfile.endswith(('c', 'o')):
modfile = modfile[:-1]
Modified: python/branches/py3k-issue1717/Lib/test/test_io.py
==============================================================================
--- python/branches/py3k-issue1717/Lib/test/test_io.py (original)
+++ python/branches/py3k-issue1717/Lib/test/test_io.py Sat Dec 20 14:43:13 2008
@@ -679,8 +679,9 @@
@classmethod
def lookupTestDecoder(cls, name):
if cls.codecEnabled and name == 'test_decoder':
+ latin1 = codecs.lookup('latin-1')
return codecs.CodecInfo(
- name='test_decoder', encode=None, decode=None,
+ name='test_decoder', encode=latin1.encode, decode=None,
incrementalencoder=None,
streamreader=None, streamwriter=None,
incrementaldecoder=cls)
@@ -840,8 +841,11 @@
[ '\r\n', [ "unix\nwindows\r\n", "os9\rlast\nnonl" ] ],
[ '\r', [ "unix\nwindows\r", "\nos9\r", "last\nnonl" ] ],
]
-
- encodings = ('utf-8', 'latin-1')
+ encodings = (
+ 'utf-8', 'latin-1',
+ 'utf-16', 'utf-16-le', 'utf-16-be',
+ 'utf-32', 'utf-32-le', 'utf-32-be',
+ )
# Try a range of buffer sizes to test the case where \r is the last
# character in TextIOWrapper._pending_line.
@@ -1195,56 +1199,84 @@
self.assertEqual(buffer.seekable(), txt.seekable())
- def test_newline_decoder(self):
- import codecs
- decoder = codecs.getincrementaldecoder("utf-8")()
- decoder = io.IncrementalNewlineDecoder(decoder, translate=True)
-
- self.assertEquals(decoder.decode(b'\xe8\xa2\x88'), "\u8888")
-
- self.assertEquals(decoder.decode(b'\xe8'), "")
- self.assertEquals(decoder.decode(b'\xa2'), "")
- self.assertEquals(decoder.decode(b'\x88'), "\u8888")
+ def check_newline_decoder_utf8(self, decoder):
+ # UTF-8 specific tests for a newline decoder
+ def _check_decode(b, s, **kwargs):
+ # We exercise getstate() / setstate() as well as decode()
+ state = decoder.getstate()
+ self.assertEquals(decoder.decode(b, **kwargs), s)
+ decoder.setstate(state)
+ self.assertEquals(decoder.decode(b, **kwargs), s)
+
+ _check_decode(b'\xe8\xa2\x88', "\u8888")
+
+ _check_decode(b'\xe8', "")
+ _check_decode(b'\xa2', "")
+ _check_decode(b'\x88', "\u8888")
+
+ _check_decode(b'\xe8', "")
+ _check_decode(b'\xa2', "")
+ _check_decode(b'\x88', "\u8888")
- self.assertEquals(decoder.decode(b'\xe8'), "")
+ _check_decode(b'\xe8', "")
self.assertRaises(UnicodeDecodeError, decoder.decode, b'', final=True)
- decoder.setstate((b'', 0))
- self.assertEquals(decoder.decode(b'\n'), "\n")
- self.assertEquals(decoder.decode(b'\r'), "")
- self.assertEquals(decoder.decode(b'', final=True), "\n")
- self.assertEquals(decoder.decode(b'\r', final=True), "\n")
-
- self.assertEquals(decoder.decode(b'\r'), "")
- self.assertEquals(decoder.decode(b'a'), "\na")
-
- self.assertEquals(decoder.decode(b'\r\r\n'), "\n\n")
- self.assertEquals(decoder.decode(b'\r'), "")
- self.assertEquals(decoder.decode(b'\r'), "\n")
- self.assertEquals(decoder.decode(b'\na'), "\na")
-
- self.assertEquals(decoder.decode(b'\xe8\xa2\x88\r\n'), "\u8888\n")
- self.assertEquals(decoder.decode(b'\xe8\xa2\x88'), "\u8888")
- self.assertEquals(decoder.decode(b'\n'), "\n")
- self.assertEquals(decoder.decode(b'\xe8\xa2\x88\r'), "\u8888")
- self.assertEquals(decoder.decode(b'\n'), "\n")
-
- decoder = codecs.getincrementaldecoder("utf-8")()
- decoder = io.IncrementalNewlineDecoder(decoder, translate=True)
+ decoder.reset()
+ _check_decode(b'\n', "\n")
+ _check_decode(b'\r', "")
+ _check_decode(b'', "\n", final=True)
+ _check_decode(b'\r', "\n", final=True)
+
+ _check_decode(b'\r', "")
+ _check_decode(b'a', "\na")
+
+ _check_decode(b'\r\r\n', "\n\n")
+ _check_decode(b'\r', "")
+ _check_decode(b'\r', "\n")
+ _check_decode(b'\na', "\na")
+
+ _check_decode(b'\xe8\xa2\x88\r\n', "\u8888\n")
+ _check_decode(b'\xe8\xa2\x88', "\u8888")
+ _check_decode(b'\n', "\n")
+ _check_decode(b'\xe8\xa2\x88\r', "\u8888")
+ _check_decode(b'\n', "\n")
+
+ def check_newline_decoder(self, decoder, encoding):
+ result = []
+ encoder = codecs.getincrementalencoder(encoding)()
+ def _decode_bytewise(s):
+ for b in encoder.encode(s):
+ result.append(decoder.decode(bytes([b])))
self.assertEquals(decoder.newlines, None)
- decoder.decode(b"abc\n\r")
+ _decode_bytewise("abc\n\r")
self.assertEquals(decoder.newlines, '\n')
- decoder.decode(b"\nabc")
+ _decode_bytewise("\nabc")
self.assertEquals(decoder.newlines, ('\n', '\r\n'))
- decoder.decode(b"abc\r")
+ _decode_bytewise("abc\r")
self.assertEquals(decoder.newlines, ('\n', '\r\n'))
- decoder.decode(b"abc")
+ _decode_bytewise("abc")
self.assertEquals(decoder.newlines, ('\r', '\n', '\r\n'))
- decoder.decode(b"abc\r")
+ _decode_bytewise("abc\r")
+ self.assertEquals("".join(result), "abc\n\nabcabc\nabcabc")
decoder.reset()
- self.assertEquals(decoder.decode(b"abc"), "abc")
+ self.assertEquals(decoder.decode("abc".encode(encoding)), "abc")
self.assertEquals(decoder.newlines, None)
+ def test_newline_decoder(self):
+ encodings = (
+ 'utf-8', 'latin-1',
+ 'utf-16', 'utf-16-le', 'utf-16-be',
+ 'utf-32', 'utf-32-le', 'utf-32-be',
+ )
+ for enc in encodings:
+ decoder = codecs.getincrementaldecoder(enc)()
+ decoder = io.IncrementalNewlineDecoder(decoder, translate=True)
+ self.check_newline_decoder(decoder, enc)
+ decoder = codecs.getincrementaldecoder("utf-8")()
+ decoder = io.IncrementalNewlineDecoder(decoder, translate=True)
+ self.check_newline_decoder_utf8(decoder)
+
+
# XXX Tests for open()
class MiscIOTest(unittest.TestCase):
Modified: python/branches/py3k-issue1717/Lib/test/test_long.py
==============================================================================
--- python/branches/py3k-issue1717/Lib/test/test_long.py (original)
+++ python/branches/py3k-issue1717/Lib/test/test_long.py Sat Dec 20 14:43:13 2008
@@ -3,6 +3,7 @@
import sys
import random
+import math
# Used for lazy formatting of failure messages
class Frm(object):
@@ -837,6 +838,41 @@
self.assertTrue(i - i is 0)
self.assertTrue(0 * i is 0)
+ def test_bit_length(self):
+ tiny = 1e-10
+ for x in range(-65000, 65000):
+ k = x.bit_length()
+ # Check equivalence with Python version
+ self.assertEqual(k, len(bin(x).lstrip('-0b')))
+ # Behaviour as specified in the docs
+ if x != 0:
+ self.assert_(2**(k-1) <= abs(x) < 2**k)
+ else:
+ self.assertEqual(k, 0)
+ # Alternative definition: x.bit_length() == 1 + floor(log_2(x))
+ if x != 0:
+ # When x is an exact power of 2, numeric errors can
+ # cause floor(log(x)/log(2)) to be one too small; for
+ # small x this can be fixed by adding a small quantity
+ # to the quotient before taking the floor.
+ self.assertEqual(k, 1 + math.floor(
+ math.log(abs(x))/math.log(2) + tiny))
+
+ self.assertEqual((0).bit_length(), 0)
+ self.assertEqual((1).bit_length(), 1)
+ self.assertEqual((-1).bit_length(), 1)
+ self.assertEqual((2).bit_length(), 2)
+ self.assertEqual((-2).bit_length(), 2)
+ for i in [2, 3, 15, 16, 17, 31, 32, 33, 63, 64, 234]:
+ a = 2**i
+ self.assertEqual((a-1).bit_length(), i)
+ self.assertEqual((1-a).bit_length(), i)
+ self.assertEqual((a).bit_length(), i+1)
+ self.assertEqual((-a).bit_length(), i+1)
+ self.assertEqual((a+1).bit_length(), i+1)
+ self.assertEqual((-a-1).bit_length(), i+1)
+
+
def test_main():
support.run_unittest(LongTest)
Modified: python/branches/py3k-issue1717/Lib/test/test_normalization.py
==============================================================================
--- python/branches/py3k-issue1717/Lib/test/test_normalization.py (original)
+++ python/branches/py3k-issue1717/Lib/test/test_normalization.py Sat Dec 20 14:43:13 2008
@@ -9,7 +9,7 @@
TESTDATAURL = "http://www.unicode.org/Public/" + unidata_version + "/ucd/" + TESTDATAFILE
if os.path.exists(TESTDATAFILE):
- f = open(TESTDATAFILE)
+ f = open(TESTDATAFILE, encoding='utf-8')
l = f.readline()
f.close()
if not unidata_version in l:
Modified: python/branches/py3k-issue1717/Lib/test/test_tarfile.py
==============================================================================
--- python/branches/py3k-issue1717/Lib/test/test_tarfile.py (original)
+++ python/branches/py3k-issue1717/Lib/test/test_tarfile.py Sat Dec 20 14:43:13 2008
@@ -255,17 +255,14 @@
def test_extractall(self):
# Test if extractall() correctly restores directory permissions
# and times (see issue1735).
- if sys.platform == "win32":
- # Win32 has no support for utime() on directories or
- # fine grained permissions.
- return
-
tar = tarfile.open(tarname, encoding="iso8859-1")
directories = [t for t in tar if t.isdir()]
tar.extractall(TEMPDIR, directories)
for tarinfo in directories:
path = os.path.join(TEMPDIR, tarinfo.name)
- self.assertEqual(tarinfo.mode & 0o777, os.stat(path).st_mode & 0o777)
+ if sys.platform != "win32":
+ # Win32 has no support for fine grained permissions.
+ self.assertEqual(tarinfo.mode & 0o777, os.stat(path).st_mode & 0o777)
self.assertEqual(tarinfo.mtime, os.path.getmtime(path))
tar.close()
Modified: python/branches/py3k-issue1717/Lib/test/test_textwrap.py
==============================================================================
--- python/branches/py3k-issue1717/Lib/test/test_textwrap.py (original)
+++ python/branches/py3k-issue1717/Lib/test/test_textwrap.py Sat Dec 20 14:43:13 2008
@@ -365,6 +365,14 @@
self.assertRaises(ValueError, wrap, text, 0)
self.assertRaises(ValueError, wrap, text, -1)
+ def test_no_split_at_umlaut(self):
+ text = "Die Empf\xe4nger-Auswahl"
+ self.check_wrap(text, 13, ["Die", "Empf\xe4nger-", "Auswahl"])
+
+ def test_umlaut_followed_by_dash(self):
+ text = "aa \xe4\xe4-\xe4\xe4"
+ self.check_wrap(text, 7, ["aa \xe4\xe4-", "\xe4\xe4"])
+
class LongWordTestCase (BaseTestCase):
def setUp(self):
Modified: python/branches/py3k-issue1717/Lib/test/test_tokenize.py
==============================================================================
--- python/branches/py3k-issue1717/Lib/test/test_tokenize.py (original)
+++ python/branches/py3k-issue1717/Lib/test/test_tokenize.py Sat Dec 20 14:43:13 2008
@@ -795,6 +795,8 @@
self.assertEquals(encoding, 'utf-8')
self.assertEquals(consumed_lines, [])
+ readline = self.get_readline((b'# coding: bad\n',))
+ self.assertRaises(SyntaxError, detect_encoding, readline)
class TestTokenize(TestCase):
Modified: python/branches/py3k-issue1717/Lib/test/test_urllib2_localnet.py
==============================================================================
--- python/branches/py3k-issue1717/Lib/test/test_urllib2_localnet.py (original)
+++ python/branches/py3k-issue1717/Lib/test/test_urllib2_localnet.py Sat Dec 20 14:43:13 2008
@@ -449,7 +449,7 @@
# domain will be spared to serve its defined
# purpose.
urllib.request.urlopen,
- "http://www.python.invalid./")
+ "http://sadflkjsasf.i.nvali.d/")
def test_main():
support.run_unittest(ProxyAuthTests)
Modified: python/branches/py3k-issue1717/Lib/test/test_urllibnet.py
==============================================================================
--- python/branches/py3k-issue1717/Lib/test/test_urllibnet.py (original)
+++ python/branches/py3k-issue1717/Lib/test/test_urllibnet.py Sat Dec 20 14:43:13 2008
@@ -139,7 +139,7 @@
# purpose.
# urllib.urlopen, "http://www.sadflkjsasadf.com/")
urllib.request.urlopen,
- "http://www.python.invalid./")
+ "http://sadflkjsasf.i.nvali.d/")
class urlretrieveNetworkTests(unittest.TestCase):
"""Tests urllib.request.urlretrieve using the network."""
Modified: python/branches/py3k-issue1717/Lib/test/test_with.py
==============================================================================
--- python/branches/py3k-issue1717/Lib/test/test_with.py (original)
+++ python/branches/py3k-issue1717/Lib/test/test_with.py Sat Dec 20 14:43:13 2008
@@ -505,6 +505,36 @@
self.assertRaises(GeneratorExit, shouldThrow)
+ def testErrorsInBool(self):
+ # issue4589: __exit__ return code may raise an exception
+ # when looking at its truth value.
+
+ class cm(object):
+ def __init__(self, bool_conversion):
+ class Bool:
+ def __bool__(self):
+ return bool_conversion()
+ self.exit_result = Bool()
+ def __enter__(self):
+ return 3
+ def __exit__(self, a, b, c):
+ return self.exit_result
+
+ def trueAsBool():
+ with cm(lambda: True):
+ self.fail("Should NOT see this")
+ trueAsBool()
+
+ def falseAsBool():
+ with cm(lambda: False):
+ self.fail("Should raise")
+ self.assertRaises(AssertionError, falseAsBool)
+
+ def failAsBool():
+ with cm(lambda: 1//0):
+ self.fail("Should NOT see this")
+ self.assertRaises(ZeroDivisionError, failAsBool)
+
class NonLocalFlowControlTestCase(unittest.TestCase):
Modified: python/branches/py3k-issue1717/Lib/test/test_zipimport.py
==============================================================================
--- python/branches/py3k-issue1717/Lib/test/test_zipimport.py (original)
+++ python/branches/py3k-issue1717/Lib/test/test_zipimport.py Sat Dec 20 14:43:13 2008
@@ -211,16 +211,24 @@
zi = zipimport.zipimporter(TEMP_ZIP)
self.assertEquals(zi.archive, TEMP_ZIP)
self.assertEquals(zi.is_package(TESTPACK), True)
- zi.load_module(TESTPACK)
+ mod = zi.load_module(TESTPACK)
+ self.assertEquals(zi._get_filename(TESTPACK), mod.__file__)
self.assertEquals(zi.is_package(packdir + '__init__'), False)
self.assertEquals(zi.is_package(packdir + TESTPACK2), True)
self.assertEquals(zi.is_package(packdir2 + TESTMOD), False)
- mod_name = packdir2 + TESTMOD
- mod = __import__(module_path_to_dotted_name(mod_name))
+ mod_path = packdir2 + TESTMOD
+ mod_name = module_path_to_dotted_name(mod_path)
+ pkg = __import__(mod_name)
+ mod = sys.modules[mod_name]
self.assertEquals(zi.get_source(TESTPACK), None)
- self.assertEquals(zi.get_source(mod_name), None)
+ self.assertEquals(zi.get_source(mod_path), None)
+ self.assertEquals(zi._get_filename(mod_path), mod.__file__)
+ # To pass in the module name instead of the path, we must use the right importer
+ loader = mod.__loader__
+ self.assertEquals(loader.get_source(mod_name), None)
+ self.assertEquals(loader._get_filename(mod_name), mod.__file__)
# test prefix and archivepath members
zi2 = zipimport.zipimporter(TEMP_ZIP + os.sep + TESTPACK)
@@ -248,15 +256,23 @@
self.assertEquals(zi.archive, TEMP_ZIP)
self.assertEquals(zi.prefix, packdir)
self.assertEquals(zi.is_package(TESTPACK2), True)
- zi.load_module(TESTPACK2)
+ mod = zi.load_module(TESTPACK2)
+ self.assertEquals(zi._get_filename(TESTPACK2), mod.__file__)
self.assertEquals(zi.is_package(TESTPACK2 + os.sep + '__init__'), False)
self.assertEquals(zi.is_package(TESTPACK2 + os.sep + TESTMOD), False)
- mod_name = TESTPACK2 + os.sep + TESTMOD
- mod = __import__(module_path_to_dotted_name(mod_name))
+ mod_path = TESTPACK2 + os.sep + TESTMOD
+ mod_name = module_path_to_dotted_name(mod_path)
+ pkg = __import__(mod_name)
+ mod = sys.modules[mod_name]
self.assertEquals(zi.get_source(TESTPACK2), None)
- self.assertEquals(zi.get_source(mod_name), None)
+ self.assertEquals(zi.get_source(mod_path), None)
+ self.assertEquals(zi._get_filename(mod_path), mod.__file__)
+ # To pass in the module name instead of the path, we must use the right importer
+ loader = mod.__loader__
+ self.assertEquals(loader.get_source(mod_name), None)
+ self.assertEquals(loader._get_filename(mod_name), mod.__file__)
finally:
z.close()
os.remove(TEMP_ZIP)
Modified: python/branches/py3k-issue1717/Lib/textwrap.py
==============================================================================
--- python/branches/py3k-issue1717/Lib/textwrap.py (original)
+++ python/branches/py3k-issue1717/Lib/textwrap.py Sat Dec 20 14:43:13 2008
@@ -76,7 +76,7 @@
# (after stripping out empty strings).
wordsep_re = re.compile(
r'(\s+|' # any whitespace
- r'[^\s\w]*\w+[a-zA-Z]-(?=\w+[a-zA-Z])|' # hyphenated words
+ r'[^\s\w]*\w+[^0-9\W]-(?=\w+[^0-9\W])|' # hyphenated words
r'(?<=[\w\!\"\'\&\.\,\?])-{2,}(?=\w))') # em-dash
# This less funky little regex just split on recognized spaces. E.g.
Modified: python/branches/py3k-issue1717/Lib/tokenize.py
==============================================================================
--- python/branches/py3k-issue1717/Lib/tokenize.py (original)
+++ python/branches/py3k-issue1717/Lib/tokenize.py Sat Dec 20 14:43:13 2008
@@ -26,7 +26,7 @@
import re, string, sys
from token import *
-from codecs import lookup
+from codecs import lookup, BOM_UTF8
from itertools import chain, repeat
cookie_re = re.compile("coding[:=]\s*([-\w.]+)")
@@ -251,11 +251,11 @@
It detects the encoding from the presence of a utf-8 bom or an encoding
cookie as specified in pep-0263. If both a bom and a cookie are present,
- but disagree, a SyntaxError will be raised.
+ but disagree, a SyntaxError will be raised. If the encoding cookie is an
+ invalid charset, raise a SyntaxError.
If no encoding is specified, then the default of 'utf-8' will be returned.
"""
- utf8_bom = b'\xef\xbb\xbf'
bom_found = False
encoding = None
def read_or_stop():
@@ -268,18 +268,25 @@
try:
line_string = line.decode('ascii')
except UnicodeDecodeError:
- pass
- else:
- matches = cookie_re.findall(line_string)
- if matches:
- encoding = matches[0]
- if bom_found and lookup(encoding).name != 'utf-8':
- # This behaviour mimics the Python interpreter
- raise SyntaxError('encoding problem: utf-8')
- return encoding
+ return None
+
+ matches = cookie_re.findall(line_string)
+ if not matches:
+ return None
+ encoding = matches[0]
+ try:
+ codec = lookup(encoding)
+ except LookupError:
+ # This behaviour mimics the Python interpreter
+ raise SyntaxError("unknown encoding: " + encoding)
+
+ if bom_found and codec.name != 'utf-8':
+ # This behaviour mimics the Python interpreter
+ raise SyntaxError('encoding problem: utf-8')
+ return encoding
first = read_or_stop()
- if first.startswith(utf8_bom):
+ if first.startswith(BOM_UTF8):
bom_found = True
first = first[3:]
if not first:
Modified: python/branches/py3k-issue1717/Lib/webbrowser.py
==============================================================================
--- python/branches/py3k-issue1717/Lib/webbrowser.py (original)
+++ python/branches/py3k-issue1717/Lib/webbrowser.py Sat Dec 20 14:43:13 2008
@@ -2,6 +2,7 @@
"""Interfaces for launching and remotely controlling Web browsers."""
# Maintained by Georg Brandl.
+import io
import os
import shlex
import sys
@@ -223,7 +224,6 @@
cmdline = [self.name] + raise_opt + args
if remote or self.background:
- import io
inout = io.open(os.devnull, "r+")
else:
# for TTY browsers, we need stdin/out
@@ -348,7 +348,6 @@
else:
action = "openURL"
- import io
devnull = io.open(os.devnull, "r+")
# if possible, put browser in separate process group, so
# keyboard interrupts don't affect browser as well as Python
Modified: python/branches/py3k-issue1717/Makefile.pre.in
==============================================================================
--- python/branches/py3k-issue1717/Makefile.pre.in (original)
+++ python/branches/py3k-issue1717/Makefile.pre.in Sat Dec 20 14:43:13 2008
@@ -1050,7 +1050,7 @@
# This installs the Demos and Tools into the applications directory.
# It is not part of a normal frameworkinstall
frameworkinstallextras:
- cd Mac && Make installextras DESTDIR="$(DESTDIR)"
+ cd Mac && $(MAKE) installextras DESTDIR="$(DESTDIR)"
# This installs a few of the useful scripts in Tools/scripts
scriptsinstall:
Modified: python/branches/py3k-issue1717/Misc/ACKS
==============================================================================
--- python/branches/py3k-issue1717/Misc/ACKS (original)
+++ python/branches/py3k-issue1717/Misc/ACKS Sat Dec 20 14:43:13 2008
@@ -344,6 +344,7 @@
Flemming Kjær Jensen
Jiba
Orjan Johansen
+Fredrik Johansson
Gregory K. Johnson
Simon Johnston
Evan Jones
Modified: python/branches/py3k-issue1717/Misc/NEWS
==============================================================================
--- python/branches/py3k-issue1717/Misc/NEWS (original)
+++ python/branches/py3k-issue1717/Misc/NEWS Sat Dec 20 14:43:13 2008
@@ -7,7 +7,7 @@
What's New in Python 3.1 alpha 0
================================
-*Release date: XX-XXX-2008*
+*Release date: XX-XXX-200X*
Core and Builtins
-----------------
@@ -19,6 +19,27 @@
The tp_compare slot is reserved for future usage. A type definition
with an assigned tp_compare slot will raise a TypeError.
+- Issue #3106: Speedup some comparisons (str/str and int/int).
+
+- Issue #2183: Simplify and optimize bytecode for list, dict and set
+ comprehensions. Original patch for list comprehensions by Neal Norwitz.
+
+- Issue #2467: gc.DEBUG_STATS reported invalid elapsed times. Also, always
+ print elapsed times, not only when some objects are uncollectable /
+ unreachable. Original patch by Neil Schemenauer.
+
+- Issue #3439: Add a bit_length method to int.
+
+- Issue #2173: When getting device encoding, check that return value of
+ nl_langinfo is not the empty string. This was causing silent build
+ failures on OS X.
+
+- Issue #4597: Fixed several opcodes that weren't always propagating
+ exceptions.
+
+- Issue #4589: Fixed exception handling when the __exit__ function of a
+ context manager returns a value that cannot be converted to a bool.
+
- Issue #4445: Replace "sizeof(PyBytesObject)" with
"offsetof(PyBytesObject, ob_sval) + 1" when allocating memory for
bytes instances. On a typical machine this saves 3 bytes of memory
@@ -42,6 +63,40 @@
Library
-------
+- Issue #4574: reading an UTF16-encoded text file crashes if \r on 64-char
+ boundary.
+
+- Issue #4223: inspect.getsource() will now correctly display source code
+ for packages loaded via zipimport (or any other conformant PEP 302
+ loader). Original patch by Alexander Belopolsky.
+
+- Issue #4201: pdb can now access and display source code loaded via
+ zipimport (or any other conformant PEP 302 loader). Original patch by
+ Alexander Belopolsky.
+
+- Issue #4197: doctests in modules loaded via zipimport (or any other PEP
+ 302 conformant loader) will now work correctly in most cases (they
+ are still subject to the constraints that exist for all code running
+ from inside a module loaded via a PEP 302 loader and attempting to
+ perform IO operations based on __file__). Original patch by
+ Alexander Belopolsky.
+
+- Issues #4082 and #4512: Add runpy support to zipimport in a manner that
+ allows backporting to maintenance branches. Original patch by
+ Alexander Belopolsky.
+
+- Issue #4163: textwrap module: allow word splitting on a hyphen preceded by
+ a non-ASCII letter.
+
+- Issue #4616: TarFile.utime(): Restore directory times on Windows.
+
+- Issue #4021: tokenize.detect_encoding() now raises a SyntaxError when the
+ codec cannot be found. This is for compatibility with the builtin behavior.
+
+- Issue #4084: Fix max, min, max_mag and min_mag Decimal methods to
+ give correct results in the case where one argument is a quiet NaN
+ and the other is a finite number that requires rounding.
+
- Issue #4483: _dbm module now builds on systems with gdbm & gdbm_compat
libs.
@@ -80,6 +135,15 @@
support unusual filenames (such as those containing semi-colons) in
Content-Disposition headers.
+Extension Modules
+-----------------
+
+- Issues #3167, #3682: Fix test_math failures for log, log10 on Solaris,
+ OpenBSD.
+
+- Issue #4583: array.array would not always prohibit resizing when a buffer
+ has been exported, resulting in an interpreter crash when accessing the
+ buffer.
Build
-----
Modified: python/branches/py3k-issue1717/Modules/_fileio.c
==============================================================================
--- python/branches/py3k-issue1717/Modules/_fileio.c (original)
+++ python/branches/py3k-issue1717/Modules/_fileio.c Sat Dec 20 14:43:13 2008
@@ -55,20 +55,27 @@
#define PyFileIO_Check(op) (PyObject_TypeCheck((op), &PyFileIO_Type))
-/* Returns 0 on success, errno (which is < 0) on failure. */
+/* Returns 0 on success, -1 with exception set on failure. */
static int
internal_close(PyFileIOObject *self)
{
- int save_errno = 0;
+ int err = 0;
+ int save_errno;
if (self->fd >= 0) {
int fd = self->fd;
self->fd = -1;
Py_BEGIN_ALLOW_THREADS
- if (close(fd) < 0)
+ err = close(fd);
+ if (err < 0)
save_errno = errno;
Py_END_ALLOW_THREADS
}
- return save_errno;
+ if (err < 0) {
+ errno = save_errno;
+ PyErr_SetFromErrno(PyExc_IOError);
+ return -1;
+ }
+ return 0;
}
static PyObject *
@@ -78,11 +85,8 @@
self->fd = -1;
Py_RETURN_NONE;
}
- errno = internal_close(self);
- if (errno < 0) {
- PyErr_SetFromErrno(PyExc_IOError);
+ if (internal_close(self))
return NULL;
- }
Py_RETURN_NONE;
}
@@ -121,7 +125,8 @@
if (fstat(self->fd, &buf) == 0 && S_ISDIR(buf.st_mode)) {
char *msg = strerror(EISDIR);
PyObject *exc;
- internal_close(self);
+ if (internal_close(self))
+ return -1;
exc = PyObject_CallFunction(PyExc_IOError, "(is)",
EISDIR, msg);
@@ -306,11 +311,8 @@
PyObject_ClearWeakRefs((PyObject *) self);
if (self->fd >= 0 && self->closefd) {
- errno = internal_close(self);
- if (errno < 0) {
- PySys_WriteStderr("close failed: [Errno %d] %s\n",
- errno, strerror(errno));
- }
+ if(internal_close(self))
+ PyErr_WriteUnraisable((PyObject*)self);
}
Py_TYPE(self)->tp_free((PyObject *)self);
Modified: python/branches/py3k-issue1717/Modules/arraymodule.c
==============================================================================
--- python/branches/py3k-issue1717/Modules/arraymodule.c (original)
+++ python/branches/py3k-issue1717/Modules/arraymodule.c Sat Dec 20 14:43:13 2008
@@ -49,9 +49,15 @@
char *items;
size_t _new_size;
+ if (self->ob_exports > 0 && newsize != Py_SIZE(self)) {
+ PyErr_SetString(PyExc_BufferError,
+ "cannot resize an array that is exporting buffers");
+ return -1;
+ }
+
/* Bypass realloc() when a previous overallocation is large enough
to accommodate the newsize. If the newsize is 16 smaller than the
- current size, then proceed with the realloc() to shrink the list.
+ current size, then proceed with the realloc() to shrink the array.
*/
if (self->allocated >= newsize &&
@@ -61,11 +67,13 @@
return 0;
}
- if (self->ob_exports > 0) {
- PyErr_SetString(PyExc_BufferError,
- "cannot resize an array that is exporting data");
- return -1;
- }
+ if (newsize == 0) {
+ PyMem_FREE(self->ob_item);
+ self->ob_item = NULL;
+ Py_SIZE(self) = 0;
+ self->allocated = 0;
+ return 0;
+ }
/* This over-allocates proportional to the array size, making room
* for additional growth. The over-allocation is mild, but is
@@ -731,25 +739,15 @@
memmove(item + (ihigh+d)*a->ob_descr->itemsize,
item + ihigh*a->ob_descr->itemsize,
(Py_SIZE(a)-ihigh)*a->ob_descr->itemsize);
- Py_SIZE(a) += d;
- PyMem_RESIZE(item, char, Py_SIZE(a)*a->ob_descr->itemsize);
- /* Can't fail */
- a->ob_item = item;
- a->allocated = Py_SIZE(a);
+ if (array_resize(a, Py_SIZE(a) + d) == -1)
+ return -1;
}
else if (d > 0) { /* Insert d items */
- PyMem_RESIZE(item, char,
- (Py_SIZE(a) + d)*a->ob_descr->itemsize);
- if (item == NULL) {
- PyErr_NoMemory();
+ if (array_resize(a, Py_SIZE(a) + d))
return -1;
- }
memmove(item + (ihigh+d)*a->ob_descr->itemsize,
item + ihigh*a->ob_descr->itemsize,
(Py_SIZE(a)-ihigh)*a->ob_descr->itemsize);
- a->ob_item = item;
- Py_SIZE(a) += d;
- a->allocated = Py_SIZE(a);
}
if (n > 0)
memcpy(item + ilow*a->ob_descr->itemsize, b->ob_item,
@@ -804,8 +802,7 @@
static int
array_do_extend(arrayobject *self, PyObject *bb)
{
- Py_ssize_t size;
- char *old_item;
+ Py_ssize_t size, oldsize;
if (!array_Check(bb))
return array_iter_extend(self, bb);
@@ -820,18 +817,12 @@
PyErr_NoMemory();
return -1;
}
- size = Py_SIZE(self) + Py_SIZE(b);
- old_item = self->ob_item;
- PyMem_RESIZE(self->ob_item, char, size*self->ob_descr->itemsize);
- if (self->ob_item == NULL) {
- self->ob_item = old_item;
- PyErr_NoMemory();
+ oldsize = Py_SIZE(self);
+ size = oldsize + Py_SIZE(b);
+ if (array_resize(self, size) == -1)
return -1;
- }
- memcpy(self->ob_item + Py_SIZE(self)*self->ob_descr->itemsize,
- b->ob_item, Py_SIZE(b)*b->ob_descr->itemsize);
- Py_SIZE(self) = size;
- self->allocated = size;
+ memcpy(self->ob_item + oldsize * self->ob_descr->itemsize,
+ b->ob_item, Py_SIZE(b) * b->ob_descr->itemsize);
return 0;
#undef b
@@ -867,27 +858,15 @@
return PyErr_NoMemory();
}
size = Py_SIZE(self) * self->ob_descr->itemsize;
- if (n == 0) {
- PyMem_FREE(items);
- self->ob_item = NULL;
- Py_SIZE(self) = 0;
- self->allocated = 0;
+ if (n > 0 && size > PY_SSIZE_T_MAX / n) {
+ return PyErr_NoMemory();
}
- else {
- if (size > PY_SSIZE_T_MAX / n) {
- return PyErr_NoMemory();
- }
- PyMem_RESIZE(items, char, n * size);
- if (items == NULL)
- return PyErr_NoMemory();
- p = items;
- for (i = 1; i < n; i++) {
- p += size;
- memcpy(p, items, size);
- }
- self->ob_item = items;
- Py_SIZE(self) *= n;
- self->allocated = Py_SIZE(self);
+ if (array_resize(self, n * Py_SIZE(self)) == -1)
+ return NULL;
+ items = p = self->ob_item;
+ for (i = 1; i < n; i++) {
+ p += size;
+ memcpy(p, items, size);
}
}
Py_INCREF(self);
@@ -1312,7 +1291,6 @@
array_fromlist(arrayobject *self, PyObject *list)
{
Py_ssize_t n;
- Py_ssize_t itemsize = self->ob_descr->itemsize;
if (!PyList_Check(list)) {
PyErr_SetString(PyExc_TypeError, "arg must be list");
@@ -1320,28 +1298,15 @@
}
n = PyList_Size(list);
if (n > 0) {
- char *item = self->ob_item;
- Py_ssize_t i;
- PyMem_RESIZE(item, char, (Py_SIZE(self) + n) * itemsize);
- if (item == NULL) {
- PyErr_NoMemory();
+ Py_ssize_t i, old_size;
+ old_size = Py_SIZE(self);
+ if (array_resize(self, old_size + n) == -1)
return NULL;
- }
- self->ob_item = item;
- Py_SIZE(self) += n;
- self->allocated = Py_SIZE(self);
for (i = 0; i < n; i++) {
PyObject *v = PyList_GetItem(list, i);
if ((*self->ob_descr->setitem)(self,
Py_SIZE(self) - n + i, v) != 0) {
- Py_SIZE(self) -= n;
- if (itemsize && (Py_SIZE(self) > PY_SSIZE_T_MAX / itemsize)) {
- return PyErr_NoMemory();
- }
- PyMem_RESIZE(item, char,
- Py_SIZE(self) * itemsize);
- self->ob_item = item;
- self->allocated = Py_SIZE(self);
+ array_resize(self, old_size);
return NULL;
}
}
@@ -1395,21 +1360,15 @@
}
n = n / itemsize;
if (n > 0) {
- char *item = self->ob_item;
- if ((n > PY_SSIZE_T_MAX - Py_SIZE(self)) ||
- ((Py_SIZE(self) + n) > PY_SSIZE_T_MAX / itemsize)) {
+ Py_ssize_t old_size = Py_SIZE(self);
+ if ((n > PY_SSIZE_T_MAX - old_size) ||
+ ((old_size + n) > PY_SSIZE_T_MAX / itemsize)) {
return PyErr_NoMemory();
}
- PyMem_RESIZE(item, char, (Py_SIZE(self) + n) * itemsize);
- if (item == NULL) {
- PyErr_NoMemory();
+ if (array_resize(self, old_size + n) == -1)
return NULL;
- }
- self->ob_item = item;
- Py_SIZE(self) += n;
- self->allocated = Py_SIZE(self);
- memcpy(item + (Py_SIZE(self) - n) * itemsize,
- str, itemsize*n);
+ memcpy(self->ob_item + old_size * itemsize,
+ str, n * itemsize);
}
Py_INCREF(Py_None);
return Py_None;
@@ -1458,19 +1417,10 @@
return NULL;
}
if (n > 0) {
- Py_UNICODE *item = (Py_UNICODE *) self->ob_item;
- if (Py_SIZE(self) > PY_SSIZE_T_MAX - n) {
- return PyErr_NoMemory();
- }
- PyMem_RESIZE(item, Py_UNICODE, Py_SIZE(self) + n);
- if (item == NULL) {
- PyErr_NoMemory();
+ Py_ssize_t old_size = Py_SIZE(self);
+ if (array_resize(self, old_size + n) == -1)
return NULL;
- }
- self->ob_item = (char *) item;
- Py_SIZE(self) += n;
- self->allocated = Py_SIZE(self);
- memcpy(item + Py_SIZE(self) - n,
+ memcpy(self->ob_item + old_size * sizeof(Py_UNICODE),
ustr, n * sizeof(Py_UNICODE));
}
@@ -1740,12 +1690,12 @@
self->ob_item + stop * itemsize,
(Py_SIZE(self) - stop) * itemsize);
if (array_resize(self, Py_SIZE(self) +
- needed - slicelength) < 0)
+ needed - slicelength) < 0)
return -1;
}
else if (slicelength < needed) {
if (array_resize(self, Py_SIZE(self) +
- needed - slicelength) < 0)
+ needed - slicelength) < 0)
return -1;
memmove(self->ob_item + (start + needed) * itemsize,
self->ob_item + stop * itemsize,
Modified: python/branches/py3k-issue1717/Modules/gcmodule.c
==============================================================================
--- python/branches/py3k-issue1717/Modules/gcmodule.c (original)
+++ python/branches/py3k-issue1717/Modules/gcmodule.c Sat Dec 20 14:43:13 2008
@@ -709,6 +709,24 @@
(void)PyFloat_ClearFreeList();
}
+static double
+get_time(void)
+{
+ double result = 0;
+ if (tmod != NULL) {
+ PyObject *f = PyObject_CallMethod(tmod, "time", NULL);
+ if (f == NULL) {
+ PyErr_Clear();
+ }
+ else {
+ if (PyFloat_Check(f))
+ result = PyFloat_AsDouble(f);
+ Py_DECREF(f);
+ }
+ }
+ return result;
+}
+
/* This is the main function. Read this to understand how the
* collection process works. */
static Py_ssize_t
@@ -731,16 +749,7 @@
}
if (debug & DEBUG_STATS) {
- if (tmod != NULL) {
- PyObject *f = PyObject_CallMethod(tmod, "time", NULL);
- if (f == NULL) {
- PyErr_Clear();
- }
- else {
- t1 = PyFloat_AsDouble(f);
- Py_DECREF(f);
- }
- }
+ t1 = get_time();
PySys_WriteStderr("gc: collecting generation %d...\n",
generation);
PySys_WriteStderr("gc: objects in each generation:");
@@ -813,17 +822,6 @@
if (debug & DEBUG_COLLECTABLE) {
debug_cycle("collectable", FROM_GC(gc));
}
- if (tmod != NULL && (debug & DEBUG_STATS)) {
- PyObject *f = PyObject_CallMethod(tmod, "time", NULL);
- if (f == NULL) {
- PyErr_Clear();
- }
- else {
- t1 = PyFloat_AsDouble(f)-t1;
- Py_DECREF(f);
- PySys_WriteStderr("gc: %.4fs elapsed.\n", t1);
- }
- }
}
/* Clear weakrefs and invoke callbacks as necessary. */
@@ -845,14 +843,19 @@
debug_cycle("uncollectable", FROM_GC(gc));
}
if (debug & DEBUG_STATS) {
+ double t2 = get_time();
if (m == 0 && n == 0)
- PySys_WriteStderr("gc: done.\n");
+ PySys_WriteStderr("gc: done");
else
PySys_WriteStderr(
"gc: done, "
"%" PY_FORMAT_SIZE_T "d unreachable, "
- "%" PY_FORMAT_SIZE_T "d uncollectable.\n",
+ "%" PY_FORMAT_SIZE_T "d uncollectable",
n+m, n);
+ if (t1 && t2) {
+ PySys_WriteStderr(", %.4fs elapsed", t2-t1);
+ }
+ PySys_WriteStderr(".\n");
}
/* Append instances in the uncollectable set to a Python
Modified: python/branches/py3k-issue1717/Modules/mathmodule.c
==============================================================================
--- python/branches/py3k-issue1717/Modules/mathmodule.c (original)
+++ python/branches/py3k-issue1717/Modules/mathmodule.c Sat Dec 20 14:43:13 2008
@@ -137,6 +137,58 @@
}
/*
+ Various platforms (Solaris, OpenBSD) do nonstandard things for log(0),
+ log(-ve), log(NaN). Here are wrappers for log and log10 that deal with
+ special values directly, passing positive non-special values through to
+ the system log/log10.
+ */
+
+static double
+m_log(double x)
+{
+ if (Py_IS_FINITE(x)) {
+ if (x > 0.0)
+ return log(x);
+ errno = EDOM;
+ if (x == 0.0)
+ return -Py_HUGE_VAL; /* log(0) = -inf */
+ else
+ return Py_NAN; /* log(-ve) = nan */
+ }
+ else if (Py_IS_NAN(x))
+ return x; /* log(nan) = nan */
+ else if (x > 0.0)
+ return x; /* log(inf) = inf */
+ else {
+ errno = EDOM;
+ return Py_NAN; /* log(-inf) = nan */
+ }
+}
+
+static double
+m_log10(double x)
+{
+ if (Py_IS_FINITE(x)) {
+ if (x > 0.0)
+ return log10(x);
+ errno = EDOM;
+ if (x == 0.0)
+ return -Py_HUGE_VAL; /* log10(0) = -inf */
+ else
+ return Py_NAN; /* log10(-ve) = nan */
+ }
+ else if (Py_IS_NAN(x))
+ return x; /* log10(nan) = nan */
+ else if (x > 0.0)
+ return x; /* log10(inf) = inf */
+ else {
+ errno = EDOM;
+ return Py_NAN; /* log10(-inf) = nan */
+ }
+}
+
+
+/*
math_1 is used to wrap a libm function f that takes a double
arguments and returns a double.
@@ -831,11 +883,11 @@
if (!PyArg_UnpackTuple(args, "log", 1, 2, &arg, &base))
return NULL;
- num = loghelper(arg, log, "log");
+ num = loghelper(arg, m_log, "log");
if (num == NULL || base == NULL)
return num;
- den = loghelper(base, log, "log");
+ den = loghelper(base, m_log, "log");
if (den == NULL) {
Py_DECREF(num);
return NULL;
@@ -854,7 +906,7 @@
static PyObject *
math_log10(PyObject *self, PyObject *arg)
{
- return loghelper(arg, log10, "log10");
+ return loghelper(arg, m_log10, "log10");
}
PyDoc_STRVAR(math_log10_doc,
Modified: python/branches/py3k-issue1717/Modules/posixmodule.c
==============================================================================
--- python/branches/py3k-issue1717/Modules/posixmodule.c (original)
+++ python/branches/py3k-issue1717/Modules/posixmodule.c Sat Dec 20 14:43:13 2008
@@ -6724,7 +6724,7 @@
#elif defined(CODESET)
{
char *codeset = nl_langinfo(CODESET);
- if (codeset)
+ if (codeset != NULL && codeset[0] != 0)
return PyUnicode_FromString(codeset);
}
#endif
Modified: python/branches/py3k-issue1717/Modules/zipimport.c
==============================================================================
--- python/branches/py3k-issue1717/Modules/zipimport.c (original)
+++ python/branches/py3k-issue1717/Modules/zipimport.c Sat Dec 20 14:43:13 2008
@@ -354,6 +354,29 @@
return NULL;
}
+/* Return a string matching __file__ for the named module */
+static PyObject *
+zipimporter_get_filename(PyObject *obj, PyObject *args)
+{
+ ZipImporter *self = (ZipImporter *)obj;
+ PyObject *code;
+ char *fullname, *modpath;
+ int ispackage;
+
+ if (!PyArg_ParseTuple(args, "s:zipimporter._get_filename",
+ &fullname))
+ return NULL;
+
+ /* Deciding the filename requires working out where the code
+ would come from if the module was actually loaded */
+ code = get_module_code(self, fullname, &ispackage, &modpath);
+ if (code == NULL)
+ return NULL;
+ Py_DECREF(code); /* Only need the path info */
+
+ return PyUnicode_FromString(modpath);
+}
+
/* Return a bool signifying whether the module is a package or not. */
static PyObject *
zipimporter_is_package(PyObject *obj, PyObject *args)
@@ -518,6 +541,12 @@
is the module couldn't be found, return None if the archive does\n\
contain the module, but has no source for it.");
+
+PyDoc_STRVAR(doc_get_filename,
+"_get_filename(fullname) -> filename string.\n\
+\n\
+Return the filename for the specified module.");
+
static PyMethodDef zipimporter_methods[] = {
{"find_module", zipimporter_find_module, METH_VARARGS,
doc_find_module},
@@ -529,6 +558,8 @@
doc_get_code},
{"get_source", zipimporter_get_source, METH_VARARGS,
doc_get_source},
+ {"_get_filename", zipimporter_get_filename, METH_VARARGS,
+ doc_get_filename},
{"is_package", zipimporter_is_package, METH_VARARGS,
doc_is_package},
{NULL, NULL} /* sentinel */
Modified: python/branches/py3k-issue1717/Objects/floatobject.c
==============================================================================
--- python/branches/py3k-issue1717/Objects/floatobject.c (original)
+++ python/branches/py3k-issue1717/Objects/floatobject.c Sat Dec 20 14:43:13 2008
@@ -1238,7 +1238,7 @@
********************/
/* leading whitespace and optional sign */
- while (isspace(*s))
+ while (isspace(Py_CHARMASK(*s)))
s++;
if (*s == '-') {
s++;
@@ -1308,7 +1308,7 @@
exp = 0;
/* optional trailing whitespace leading to the end of the string */
- while (isspace(*s))
+ while (isspace(Py_CHARMASK(*s)))
s++;
if (s != s_end)
goto parse_error;
Modified: python/branches/py3k-issue1717/Objects/longobject.c
==============================================================================
--- python/branches/py3k-issue1717/Objects/longobject.c (original)
+++ python/branches/py3k-issue1717/Objects/longobject.c Sat Dec 20 14:43:13 2008
@@ -2232,14 +2232,45 @@
return sign < 0 ? -1 : sign > 0 ? 1 : 0;
}
+#define TEST_COND(cond) \
+ ((cond) ? Py_True : Py_False)
+
static PyObject *
long_richcompare(PyObject *self, PyObject *other, int op)
{
- PyObject *result;
+ int result;
+ PyObject *v;
CHECK_BINOP(self, other);
- result = Py_CmpToRich(op, long_compare((PyLongObject*)self,
- (PyLongObject*)other));
- return result;
+ if (self == other)
+ result = 0;
+ else
+ result = long_compare((PyLongObject*)self, (PyLongObject*)other);
+ /* Convert the return value to a Boolean */
+ switch (op) {
+ case Py_EQ:
+ v = TEST_COND(result == 0);
+ break;
+ case Py_NE:
+ v = TEST_COND(result != 0);
+ break;
+ case Py_LE:
+ v = TEST_COND(result <= 0);
+ break;
+ case Py_GE:
+ v = TEST_COND(result >= 0);
+ break;
+ case Py_LT:
+ v = TEST_COND(result == -1);
+ break;
+ case Py_GT:
+ v = TEST_COND(result == 1);
+ break;
+ default:
+ PyErr_BadArgument();
+ return NULL;
+ }
+ Py_INCREF(v);
+ return v;
}
static long
@@ -3650,6 +3681,75 @@
return PyLong_FromSsize_t(res);
}
+static const unsigned char BitLengthTable[32] = {
+ 0, 1, 2, 2, 3, 3, 3, 3, 4, 4, 4, 4, 4, 4, 4, 4,
+ 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5
+};
+
+static PyObject *
+long_bit_length(PyLongObject *v)
+{
+ PyLongObject *result, *x, *y;
+ Py_ssize_t ndigits, msd_bits = 0;
+ digit msd;
+
+ assert(v != NULL);
+ assert(PyLong_Check(v));
+
+ ndigits = ABS(Py_SIZE(v));
+ if (ndigits == 0)
+ return PyLong_FromLong(0);
+
+ msd = v->ob_digit[ndigits-1];
+ while (msd >= 32) {
+ msd_bits += 6;
+ msd >>= 6;
+ }
+ msd_bits += (long)(BitLengthTable[msd]);
+
+ if (ndigits <= PY_SSIZE_T_MAX/PyLong_SHIFT)
+ return PyLong_FromSsize_t((ndigits-1)*PyLong_SHIFT + msd_bits);
+
+ /* expression above may overflow; use Python integers instead */
+ result = (PyLongObject *)PyLong_FromSsize_t(ndigits - 1);
+ if (result == NULL)
+ return NULL;
+ x = (PyLongObject *)PyLong_FromLong(PyLong_SHIFT);
+ if (x == NULL)
+ goto error;
+ y = (PyLongObject *)long_mul(result, x);
+ Py_DECREF(x);
+ if (y == NULL)
+ goto error;
+ Py_DECREF(result);
+ result = y;
+
+ x = (PyLongObject *)PyLong_FromLong(msd_bits);
+ if (x == NULL)
+ goto error;
+ y = (PyLongObject *)long_add(result, x);
+ Py_DECREF(x);
+ if (y == NULL)
+ goto error;
+ Py_DECREF(result);
+ result = y;
+
+ return (PyObject *)result;
+
+error:
+ Py_DECREF(result);
+ return NULL;
+}
+
+PyDoc_STRVAR(long_bit_length_doc,
+"int.bit_length() -> int\n\
+\n\
+Number of bits necessary to represent self in binary.\n\
+>>> bin(37)\n\
+'0b100101'\n\
+>>> (37).bit_length()\n\
+6");
+
#if 0
static PyObject *
long_is_finite(PyObject *v)
@@ -3661,6 +3761,8 @@
static PyMethodDef long_methods[] = {
{"conjugate", (PyCFunction)long_long, METH_NOARGS,
"Returns self, the complex conjugate of any int."},
+ {"bit_length", (PyCFunction)long_bit_length, METH_NOARGS,
+ long_bit_length_doc},
#if 0
{"is_finite", (PyCFunction)long_is_finite, METH_NOARGS,
"Returns always True."},
Modified: python/branches/py3k-issue1717/Objects/unicodeobject.c
==============================================================================
--- python/branches/py3k-issue1717/Objects/unicodeobject.c (original)
+++ python/branches/py3k-issue1717/Objects/unicodeobject.c Sat Dec 20 14:43:13 2008
@@ -6493,42 +6493,65 @@
return 1;
}
+
+#define TEST_COND(cond) \
+ ((cond) ? Py_True : Py_False)
+
PyObject *PyUnicode_RichCompare(PyObject *left,
PyObject *right,
int op)
{
int result;
-
+
if (PyUnicode_Check(left) && PyUnicode_Check(right)) {
- result = unicode_compare((PyUnicodeObject *)left,
- (PyUnicodeObject *)right);
- } else {
- Py_INCREF(Py_NotImplemented);
- return Py_NotImplemented;
- }
-
- /* Convert the return value to a Boolean */
- switch (op) {
- case Py_EQ:
- result = (result == 0);
- break;
- case Py_NE:
- result = (result != 0);
- break;
- case Py_LE:
- result = (result <= 0);
- break;
- case Py_GE:
- result = (result >= 0);
- break;
- case Py_LT:
- result = (result == -1);
- break;
- case Py_GT:
- result = (result == 1);
- break;
+ PyObject *v;
+ if (((PyUnicodeObject *) left)->length !=
+ ((PyUnicodeObject *) right)->length) {
+ if (op == Py_EQ) {
+ Py_INCREF(Py_False);
+ return Py_False;
+ }
+ if (op == Py_NE) {
+ Py_INCREF(Py_True);
+ return Py_True;
+ }
+ }
+ if (left == right)
+ result = 0;
+ else
+ result = unicode_compare((PyUnicodeObject *)left,
+ (PyUnicodeObject *)right);
+
+ /* Convert the return value to a Boolean */
+ switch (op) {
+ case Py_EQ:
+ v = TEST_COND(result == 0);
+ break;
+ case Py_NE:
+ v = TEST_COND(result != 0);
+ break;
+ case Py_LE:
+ v = TEST_COND(result <= 0);
+ break;
+ case Py_GE:
+ v = TEST_COND(result >= 0);
+ break;
+ case Py_LT:
+ v = TEST_COND(result == -1);
+ break;
+ case Py_GT:
+ v = TEST_COND(result == 1);
+ break;
+ default:
+ PyErr_BadArgument();
+ return NULL;
+ }
+ Py_INCREF(v);
+ return v;
}
- return PyBool_FromLong(result);
+
+ Py_INCREF(Py_NotImplemented);
+ return Py_NotImplemented;
}
int PyUnicode_Contains(PyObject *container,
Modified: python/branches/py3k-issue1717/Parser/tokenizer.c
==============================================================================
--- python/branches/py3k-issue1717/Parser/tokenizer.c (original)
+++ python/branches/py3k-issue1717/Parser/tokenizer.c Sat Dec 20 14:43:13 2008
@@ -452,8 +452,8 @@
stream = PyObject_CallMethod(io, "open", "ssis",
tok->filename, "r", -1, enc);
else
- stream = PyObject_CallMethod(io, "open", "isis",
- fileno(tok->fp), "r", -1, enc);
+ stream = PyObject_CallMethod(io, "open", "isisOOO",
+ fileno(tok->fp), "r", -1, enc, Py_None, Py_None, Py_False);
if (stream == NULL)
goto cleanup;
Modified: python/branches/py3k-issue1717/Python/ceval.c
==============================================================================
--- python/branches/py3k-issue1717/Python/ceval.c (original)
+++ python/branches/py3k-issue1717/Python/ceval.c Sat Dec 20 14:43:13 2008
@@ -1120,6 +1120,7 @@
}
Py_FatalError("invalid argument to DUP_TOPX"
" (bytecode corruption?)");
+ /* Never returns, so don't bother to set why. */
break;
case UNARY_POSITIVE:
@@ -1305,9 +1306,8 @@
case LIST_APPEND:
w = POP();
- v = POP();
+ v = stack_pointer[-oparg];
err = PyList_Append(v, w);
- Py_DECREF(v);
Py_DECREF(w);
if (err == 0) {
PREDICT(JUMP_ABSOLUTE);
@@ -1317,9 +1317,8 @@
case SET_ADD:
w = POP();
- v = POP();
+ v = stack_pointer[-oparg];
err = PySet_Add(v, w);
- Py_DECREF(v);
Py_DECREF(w);
if (err == 0) {
PREDICT(JUMP_ABSOLUTE);
@@ -1736,6 +1735,7 @@
if ((v = f->f_locals) == NULL) {
PyErr_Format(PyExc_SystemError,
"no locals when loading %R", w);
+ why = WHY_EXCEPTION;
break;
}
if (PyDict_CheckExact(v)) {
@@ -1933,6 +1933,21 @@
if (err == 0) continue;
break;
+ case MAP_ADD:
+ w = TOP(); /* key */
+ u = SECOND(); /* value */
+ STACKADJ(-2);
+ v = stack_pointer[-oparg]; /* dict */
+ assert (PyDict_CheckExact(v));
+ err = PyDict_SetItem(v, w, u); /* v[w] = u */
+ Py_DECREF(u);
+ Py_DECREF(w);
+ if (err == 0) {
+ PREDICT(JUMP_ABSOLUTE);
+ continue;
+ }
+ break;
+
case LOAD_ATTR:
w = GETITEM(names, oparg);
v = TOP();
@@ -2189,7 +2204,17 @@
Py_DECREF(exit_func);
if (x == NULL)
break; /* Go to error exit */
- if (u != Py_None && PyObject_IsTrue(x)) {
+
+ if (u != Py_None)
+ err = PyObject_IsTrue(x);
+ else
+ err = 0;
+ Py_DECREF(x);
+
+ if (err < 0)
+ break; /* Go to error exit */
+ else if (err > 0) {
+ err = 0;
/* There was an exception and a True return */
STACKADJ(-2);
SET_TOP(PyLong_FromLong((long) WHY_SILENCED));
@@ -2197,7 +2222,6 @@
Py_DECREF(v);
Py_DECREF(w);
}
- Py_DECREF(x);
PREDICT(END_FINALLY);
break;
}
@@ -2278,7 +2302,10 @@
if (x != NULL && opcode == MAKE_CLOSURE) {
v = POP();
- err = PyFunction_SetClosure(x, v);
+ if (PyFunction_SetClosure(x, v) != 0) {
+ /* Can't happen unless bytecode is corrupt. */
+ why = WHY_EXCEPTION;
+ }
Py_DECREF(v);
}
@@ -2302,7 +2329,11 @@
Py_DECREF(w);
}
- err = PyFunction_SetAnnotations(x, v);
+ if (PyFunction_SetAnnotations(x, v) != 0) {
+ /* Can't happen unless
+ PyFunction_SetAnnotations changes. */
+ why = WHY_EXCEPTION;
+ }
Py_DECREF(v);
Py_DECREF(u);
}
@@ -2319,7 +2350,11 @@
w = POP();
PyTuple_SET_ITEM(v, posdefaults, w);
}
- err = PyFunction_SetDefaults(x, v);
+ if (PyFunction_SetDefaults(x, v) != 0) {
+ /* Can't happen unless
+ PyFunction_SetDefaults changes. */
+ why = WHY_EXCEPTION;
+ }
Py_DECREF(v);
}
if (x != NULL && kwdefaults > 0) {
@@ -2337,7 +2372,11 @@
Py_DECREF(w);
Py_DECREF(u);
}
- err = PyFunction_SetKwDefaults(x, v);
+ if (PyFunction_SetKwDefaults(x, v) != 0) {
+ /* Can't happen unless
+ PyFunction_SetKwDefaults changes. */
+ why = WHY_EXCEPTION;
+ }
Py_DECREF(v);
}
PUSH(x);
Modified: python/branches/py3k-issue1717/Python/compile.c
==============================================================================
--- python/branches/py3k-issue1717/Python/compile.c (original)
+++ python/branches/py3k-issue1717/Python/compile.c Sat Dec 20 14:43:13 2008
@@ -707,6 +707,8 @@
case SET_ADD:
case LIST_APPEND:
+ return -1;
+ case MAP_ADD:
return -2;
case BINARY_POWER:
@@ -2823,7 +2825,7 @@
*/
static int
-compiler_comprehension_generator(struct compiler *c, PyObject *tmpname,
+compiler_comprehension_generator(struct compiler *c,
asdl_seq *generators, int gen_index,
expr_ty elt, expr_ty val, int type)
{
@@ -2871,7 +2873,7 @@
}
if (++gen_index < asdl_seq_LEN(generators))
- if (!compiler_comprehension_generator(c, tmpname,
+ if (!compiler_comprehension_generator(c,
generators, gen_index,
elt, val, type))
return 0;
@@ -2886,27 +2888,19 @@
ADDOP(c, POP_TOP);
break;
case COMP_LISTCOMP:
- if (!compiler_nameop(c, tmpname, Load))
- return 0;
VISIT(c, expr, elt);
- ADDOP(c, LIST_APPEND);
+ ADDOP_I(c, LIST_APPEND, gen_index + 1);
break;
case COMP_SETCOMP:
- if (!compiler_nameop(c, tmpname, Load))
- return 0;
VISIT(c, expr, elt);
- ADDOP(c, SET_ADD);
+ ADDOP_I(c, SET_ADD, gen_index + 1);
break;
case COMP_DICTCOMP:
- if (!compiler_nameop(c, tmpname, Load))
- return 0;
/* With 'd[k] = v', v is evaluated before k, so we do
- the same. STORE_SUBSCR requires (item, map, key),
- so we still end up ROTing once. */
+ the same. */
VISIT(c, expr, val);
- ADDOP(c, ROT_TWO);
VISIT(c, expr, elt);
- ADDOP(c, STORE_SUBSCR);
+ ADDOP_I(c, MAP_ADD, gen_index + 1);
break;
default:
return 0;
@@ -2932,7 +2926,6 @@
asdl_seq *generators, expr_ty elt, expr_ty val)
{
PyCodeObject *co = NULL;
- identifier tmp = NULL;
expr_ty outermost_iter;
outermost_iter = ((comprehension_ty)
@@ -2943,9 +2936,6 @@
if (type != COMP_GENEXP) {
int op;
- tmp = compiler_new_tmpname(c);
- if (!tmp)
- goto error_in_scope;
switch (type) {
case COMP_LISTCOMP:
op = BUILD_LIST;
@@ -2963,12 +2953,9 @@
}
ADDOP_I(c, op, 0);
- ADDOP(c, DUP_TOP);
- if (!compiler_nameop(c, tmp, Store))
- goto error_in_scope;
}
- if (!compiler_comprehension_generator(c, tmp, generators, 0, elt,
+ if (!compiler_comprehension_generator(c, generators, 0, elt,
val, type))
goto error_in_scope;
@@ -2984,7 +2971,6 @@
if (!compiler_make_closure(c, co, 0))
goto error;
Py_DECREF(co);
- Py_XDECREF(tmp);
VISIT(c, expr, outermost_iter);
ADDOP(c, GET_ITER);
@@ -2994,7 +2980,6 @@
compiler_exit_scope(c);
error:
Py_XDECREF(co);
- Py_XDECREF(tmp);
return 0;
}
Modified: python/branches/py3k-issue1717/Python/import.c
==============================================================================
--- python/branches/py3k-issue1717/Python/import.c (original)
+++ python/branches/py3k-issue1717/Python/import.c Sat Dec 20 14:43:13 2008
@@ -87,8 +87,10 @@
3102 (__file__ points to source file)
Python 3.0a4: 3110 (WITH_CLEANUP optimization).
Python 3.0a5: 3130 (lexical exception stacking, including POP_EXCEPT)
+ Python 3.1a0: 3140 (optimize list, set and dict comprehensions:
+ change LIST_APPEND and SET_ADD, add MAP_ADD)
*/
-#define MAGIC (3130 | ((long)'\r'<<16) | ((long)'\n'<<24))
+#define MAGIC (3140 | ((long)'\r'<<16) | ((long)'\n'<<24))
/* Magic word as global; note that _PyImport_Init() can change the
value of this global to accommodate for alterations of how the
Modified: python/branches/py3k-issue1717/Tools/msi/crtlicense.txt
==============================================================================
--- python/branches/py3k-issue1717/Tools/msi/crtlicense.txt (original)
+++ python/branches/py3k-issue1717/Tools/msi/crtlicense.txt Sat Dec 20 14:43:13 2008
@@ -26,7 +26,7 @@
- alter any copyright, trademark or patent notice in Microsoft's
Distributable Code;
-- use Microsoft’s trademarks in your programs’ names or in a way that
+- use Microsoft's trademarks in your programs' names or in a way that
suggests your programs come from or are endorsed by Microsoft;
- distribute Microsoft's Distributable Code to run on a platform other
More information about the Python-checkins
mailing list