Python 2.5 schedule

A number of bugs are being squashed as Python 2.5 moves towards its next release. See PEP 356 for more details and the full schedule.

Contributing threads:

• Beta 1 schedule ? (Bug in stringobject?)
• Adding winerror module (Beta 1 schedule ?)
• current 2.5 issues
• TRUNK FREEZE IMMINENT FOR 2.5 BETA 1 - 00:00 UTC, 20-JUNE-2006
• beta1 coming real soon
• RELEASED Python 2.5 (beta 1)
• TRUNK is UNFROZEN, but in FEATURE FREEZE
• 2.5 and beyond

Checkins for betas and release candidates

Anthony Baxter announced some guidelines for checkins for the beta and release candidate releases. For all beta releases:

• All checkins must have an entry for Misc/NEWS, a test and docs
• All checkins that add features must have approval from a release manager
• All checkins must not break any of the buildbots

For all release candidates:

• All checkins must have approval from a release manager

Approval from a release manager (Anthony or Neal) should preferably be obtained in public (e.g. the python-dev list) and should be noted in the commit message.

Contributing threads:

• When to branch release25-maint?
• RFC: trunk checkins between now and 2.5 final

FishEye on the Python Subversion Repository

FishEye is once again available for the Python repository.

Contributing thread:

• FishEye on Python CVS Repository

PEP 3103: A Switch/Case Statement

After Thomas Lee provided a simple patch implementing a switch statement for Python, there was a massive discussion about it and how PEP 275 should best be implemented. After much discussion, basically three camps arose:

• School I: The switch statement should just be syntactic sugar for the corresponding if/elif chain.
• School II: The switch statement should dispatch on a precomputed dict of values.
• School III: The switch statement should correspond to an if/elif chain but require all expressions to be hashable (to allow for better optimizations).

School I was primarily concerned with the repetition of the x == in something like:

if x == ...:
   ...
elif x == ...:
   ...
elif x == ...:
   ...
else:
   ...

School II seemed to feel that just aiding DRY was not enough to introduce a new construct, and that the switch statement should also be able to avoid the function definitions in dispatching code like:

def f(...):
    ...
def g(...):
    ...
def h(...):
    ...
dispatch_dict = {x:f, y:g, z:h}
dispatch_dict[value](*args)

In order to optimize this kind of code, School II wanted to be able to compute the dispatch dict ahead of time, so that it wouldn't have be recomputed each time the switch statement was executed. There was a lot of discussion as to exactly when this freezing should occur, with some voting for module compilation time (allowing only constants in the cases), some voting for function definition time (allowing only constants and non-local names in the cases) and some voting for the first time the switch statement is executed (allowing only constants and both local and non-local names). Guido put together a thorough summary of the options in PEP 3103.

There was some discussion of introducing a static keyword which would cause an expression to be evaluated at function definition time, so that, for example, the following code would create a list of functions returning each of 0, 1, ... 9:

funcs = [lambda: (static i) for i in xrange(10)]

The intention was that switch statement cases would then allow only constants or static expressions. Guido requested a separate PEP on the idea, and Fredrik Lundh posted a proto-PEP, but at the time of this summary, no official PEP had been submitted.

In the end, it looked like Guido was leaning towards the switch statement as syntactic sugar for a dispatching dict, with the dict frozen at function definition time (which would mean compile-time for module-level switch statements). However, the introduction of the statement seemed likely to be postponed at least until Python 3.0.

Contributing threads:

• Switch statement
• An obscene computed goto bytecode hack for "switch" :)
• Simple Switch statement
• Alternatives to switch?
• Temporary Constantification
• Simple Switch statementZ
• PEP 3103: A Switch/Case Statement
• School IIb?
• Switch statement - handling errors
• Split switch statement
• once [was: Simple Switch statementZ]

Restricted execution in Python

For his Ph.D. thesis, Brett Cannon is looking into adding facilities for restricted execution to Python, partly with the goal of getting Python into Firefox alongside Javascript. His restricted execution specifications aimed to take advantage of the C-to-Python language barrier to enforce security restrictions. Though there's no real way to get private attributes in pure Python, objects coded in C and exposed to Python can select which attributes are exposed, thus making the non-exposed attributes truly private to Python-level code.

His initial draft aimed to hide as many "dangerous" objects as possible, and then cripple objects like file that would be difficult to hide. A number of people seemed to prefer a hiding-only approach, but comments from Armin Rigo seemed to suggest that plugging all the introspection holes that give access to file objects might be quite difficult.

The discussion continued on into the next fortnight.

Contributing threads:

• doc for new restricted execution design for Python
• Is Lib/test/crashers/recursive_call.py really a crasher?
• For sandboxing: alternative to crippling file()

NaN and infinities in Python float calculations

Nick Maclaren asked about trying to get more platform-independent behavior in Python's floats, so that IEEE 754 values as in PEP 754 would be produced more consistently. Currently, different OSes produce different results when these values are involved:

Python 2.4.2 (#1, May  2 2006, 08:28:01)
[GCC 4.1.0 (SUSE Linux)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> a = "NaN"
>>> b = float(a)
>>> c = int(b)
>>> d = (b == b)
>>> print a, b, c, d
NaN nan 0 False

Python 2.3.3 (#1, Feb 18 2004, 11:58:04)
[GCC 2.8.1] on sunos5
Type "help", "copyright", "credits" or "license" for more information.
>>> a = "NaN"
>>> b = float(a)
>>> c = int(b)
>>> d = (b == b)
>>> print a, b, c, d
NaN NaN 0 True

Nick Maclaren suggested either raising an exception for all ambiguous or invalid operations, or returning NaN or infinity as appropriate and then raising exceptions whenever an operation that would lose the error indication was performed. Nick Coghlan explained that the decimal module already does most of this:

>>> from decimal import Decimal as d
>>> nan = d('NaN')
>>> int(nan)
Traceback (most recent call last):
  ...
decimal.InvalidOperation
>>>
>>> from decimal import getcontext, Overflow
>>> ctx = getcontext()
>>> ctx.traps[Overflow] = False
>>> d('1e999999999') * 10
Decimal("Infinity")

Nick Maclaren seemed to suggest that he would be working on a PEP and an implementation that would bring some of the decimal module consistencies to Python's floats as well.

Contributing threads:

• Numerical robustness, IEEE etc.
• Numerical robustness, IEEE etc.
• Python memory model (low level)

ImportWarnings for directories without __init__

Ralf W. Grosse-Kunstleve complained that with Python 2.5 he started getting tons of "ImportWarning: Not importing directory" messages. James Y Knight pointed out that running Python in your home directory is quite likely to issue such warnings if you have any directories in your home directory that have the same name as a python module (e.g. readline). A number of options for silencing the errors were discussed, including invoking Python like python -W'ignore:Not importing directory' and including warnings.filterwarnings('ignore', 'Not importing directory', ImportWarning) in site.py or .pythonrc.py. Two patches were provided that introduce the warning only if the import fails, one by Shane Hathaway and one by Sergey A. Lipnevich. No final decision had been made at the time of this summary.

Contributing threads:

• Dropping __init__.py requirement for subpackages
• ImportWarning flood

Updating turtle.py

Gregor Lingl proposed replacing turtle.py in the Python standard library with his new xturtle.py module. The xturtle module is backwards compatible with the turtle module and adds a number of enhancements. However, Gregor's request came after Python 2.5's feature freeze, so he was told to propose it again in Python 2.6. There was some discussion about this -- as the stdlib turtle module is poorly tested, some contended that introducing the new APIs of xturtle would not make things any worse. A couple of compromises were offered: mentioning xturtle in the turtle module docs, and putting xturtle in the Tools directory.

Contributing threads:

• xturtle.py a replacement for turtle.py(!?)
• xturtle.py - a replacement for turtle.py
• xturtle.py a replacement for turtle.py(!?) ATTENTION PLEASE!
• xturtle.py

Relative imports and PEP 338: Executing Modules as Scripts

Relative imports, as described in PEP 328, introduced problems for PEP 338 which allows modules within packages and zipfiles to be run with the -m command-line switch. The -m switch sets the __name__ of the module to '__main__' so that if __name__ == '__main__' blocks will get executed. However, relative imports use __name__ to determine the parent package, so if a module that has a relative import is executed using the -m switch, the relative import will fail. Nick Coghlan suggested adding a __module_name__ attribute that would not be clobbered by the -m switch, but people generally seemed to think that it would be simpler to just require absolute imports in main modules.

Contributing threads:

• PEP 338 vs PEP 328 - a limitation of the -m switch
• PEP 328 and PEP 338, redux
• [Python-checkins] r47142 - in python/trunk: Doc/lib/librunpy.tex Lib/runpy.py Lib/test/test_runpy.py

Importing modules within unicode directories

Kristján V. Jónsson pointed out that currently, Python on Windows cannot import modules from directories with unicode names, even if the module names themselves are plain ASCII. Nick Coghlan suggested that this was likely because import.c was doing something like u'c:/tmp/\u814c'.encode('mbcs'), getting back 'c:/tmp/?' and being unable to do anything useful with that. Martin v. Löwis suggested using the 8.3 simplified filename used by DOS, at least until the import machinery gets reworked to better handle encodings, hopefully for Python 2.6. Thomas Heller had provided a patch for reworking import.c in this manner a while back, but it was large enough that no one had reviewed it.

Contributing thread:

• unicode imports

MS VC++ 2003 toolkit no longer available

Bill Janssen pointed out that Python 2.4 on Windows expects to be compiled with the MS Visual C++ compiler version 7.1, and that the corresponding MS VC++ 2003 toolkit is no longer available. Fredrik Lundh explained that the compiler is still available in the .net SDK as well as being available to MSDN subscribers. There was again some discussion about moving to the VS 2005 toolkit for compiling Python. It would have made compiling for 64bit architectures somewhat easier, but would have meant that extension writers would have to install three different compilers just to compile extensions for Python 2.3, 2.4 and 2.5, and would also have given problems for MinGW users as MinGW does not yet easily support linking to the msvcr80 runtime library.

Contributing threads:

• Python 2.4 extensions require VC 7.1?
• Documentation enhancement: "MS free compiler"?
• Documentation enhancement: "MS free compiler"?

Keeping interned strings in a set

Alexander Belopolsky tried out the new set C API by replacing the dict of interned strings with a set instead. He had to make two changes to get this to work: there's currently no way to retrieve a single object from a set, and Py_Finalize() needed to be changed to finalize sets after strings (instead of the other way around as it used to be). There was some discussion about trying to get rid of PySet_Fini() so the latter problem wouldn't be an issue at all, but with all the other Py*Fini() functions already existing, it didn't seem worth it.

The patch had no slowdown and reduced the memory consumption of the interning structure slightly.

Contributing threads:

• Keeping interned strings in a set
• Keeping interned strings in a set
• setobject code
• Proposal to eliminate PySet_Fini

Allowing empty subscripts

Guido finally vetoed the proposal to allow x[()] to be written as x[]. The use-cases were weak, and in most cases the functionality seemed better expressed as attribute access.

Contributing threads:

• Pre-PEP: Allow Empty Subscript List Without Parentheses
• Empty Subscript PEP on Wiki - keep or toss?

Creating range objects at the C level

Ralf W. Grosse-Kunstleve asked about the removal of the C function PyRange_New() which had been deprecated in Python 2.4. The right way to create ranges is to call PyRange_Type with the appropriate parameters, e.g. something like PyObject_CallFunction((PyObject*) &PyRange_Type, "lll", start, stop, step). Ralf was nervous about this alternative because it also appeared to be undocumented, and requested that something like the above be at least put into the What's New document.

Contributing threads:

• PyRange_New() alternative?
• PyObject* aliasing (Was: PyRange_New() alternative?)

type(), __class__ and isinstance()

Martin Maly pointed out that you can't fool isinstance() into thinking your object is not a subclass of its true base class:

>>> class C(object):
...     pass
...
>>> class D(object):
...     __class__ = property(lambda self: C)
...
>>> isinstance(D(), D)
True
>>> isinstance(D(), C)
True

Phillip J. Eby explained that isinstance() checks both the type() of the object and the __class__ attribute. In essence, you can lie about your __class__ to make isinstance() return True, but you can't lie to make it return False. Guido suggested that these issues, as well as lying about an object's __bases__, should be revisited for Python 3000.

Contributing thread:

• Semantic of isinstance

Requiring backward compatibility in the standard library modules

Ka-Ping Yee's uuid module, newly added for Python 2.5, contained a comment "This module works with Python 2.3 or higher". George Yoshida asked if that comment should be interpreted as requiring Python 2.3 compatibility. People generally felt like the list of backwards compatible modules in PEP 291 should be as small as possible so as to keep maintenance as simple as possible. Ka-Ping removed the comment, and submitted the module to PyPI for Python 2.3 and 2.4 users.

Contributing thread:

• uuid backward compatibility

Figleaf code coverage

Titus Brown offered some reports from his figleaf code coverage utility. People seemed particularly interested in trying to get coverage across multiple platforms, perhaps using a BuildBot extension, and Titus said he'd try to look into it. Walter Dörwald also pointed to his own code coverage module.

Contributing threads:

• Code coverage reporting.
• Code coverage reporting.

Improving error messages

Georg Brandl proposed going through abstract.c and modifying error messages like "object does not support item assignment" to also include the type of the object. He got little feedback, mainly because everyone seemed to think it was such an obviously good idea that there was no need for any. Python 2.5 now incorporates Georg's better error messages.

Contributing threads:

• Improve error msgs?
• Improve error msgs?

Allowing assignments in global statements

Talin proposed allowing a global statement to be combined with an assignment statement, e.g.:

global badger = 42

Guido suggested that such a desire was a sure indicator of overuse of global.

Contributing thread:

• Allow assignments in 'global' statements?

Splitting Python tests from CPython tests

Frank Wierzbicki volunteered some time into splitting out CPython specific test from Python-the-language tests. Armin Rigo pointed him to PyPy's tests modified to be more implementation independent.

Contributing thread:

• Cleanup of test harness for Python

A multi-dimensional array type for Python

For Google's Summer of Code, Karol Langner will be working on implementing a basic multi-dimensional array type for Python core, based on the numpy array struct. He asked for any comments or suggestions that people had for the project.

Contributing thread:

• basearray

Commenting on Topics

To comment on anything mentioned here, just post to comp.lang.python (or email python-list@python.org which is a gateway to the newsgroup) with a subject line mentioning what you are discussing. All python-dev members are interested in seeing ideas discussed by the community, so don't hesitate to take a stance on something. And if all of this really interests you then get involved and join python-dev!

How to Read the Summaries

This summary is written using reStructuredText. Any unfamiliar punctuation is probably markup for reST (otherwise it is probably regular expression syntax or a typo :); you can safely ignore it. We do suggest learning reST, though; it's simple and is accepted for PEP markup and can be turned into many different formats like HTML and LaTeX.