Python 2.5 schedule

After inserting a third beta release to allow some more time for testing the new features, Python continues to make progress towards the final Python 2.5 release. See PEP 356 for more details and the full schedule.

Contributing threads:

• outstanding bugs to fix for 2.5
• Py2.5 release schedule

How to submit a patch to Python

Just a few reminders for all those still new to python-dev. When submitting a new patch to SourceForge, don't assign it to anyone. Most python developers get email notifications for new patches and will assign it to themselves if appropriate. If you feel like the approach the patch takes might need discussion, it's alright to present it to python-dev and ask for some feedback. If you do, be sure to put the patch number and url (e.g. http://bugs.python.org/<sourceforge_id>) near the top of the message, so that developers can easily find it.

And if you don't want to wait for your patch to be looked at (which may take some time as all developers are volunteers), a few of the folks here, including Martin v. Löwis, have offered a five-for-one deal. Simply find five other patches, and check them for things like:

• Does the code look okay?
• Does Python build with it applied?
• Do all unit tests pass?
• Does the patch have tests?
• Does the patch have documentation?

Then post your notes to the five patch trackers and post a final message to python-dev indicating the patches you reviewed and the patch which you'd like to have someone look at for you.

Contributing threads:

• new guy
• first draft of bug guidelines for www.python.org/dev/
• Patch submitted, now what?

Demos of trackers to replace SourceForge

There are currently three potential trackers that have successfully imported the SourceForge data with demos online: roundup, jira and launchpad. Try 'em out, and send your discussions and comments to infrastructure@python.org and put your reports and reviews on the wiki.

Contributing thread:

• More tracker demos online

Restricted execution in Python

Brett Cannon decided this fortnight to go for an all-out capabilities based restricted execution design, and posted a new design document. As part of this work, Brett planned to rewrite most of the import machinery in pure Python code, hopefully cleaning up some of the idiosyncrasies of the current import.c mechanisms, and allowing him to do things like restrict imports to only .py files, not .pyc files. Armin Rigo pointed out that a good place to start would be the PyPy import implementation.

On the restricted execution front, one of the things that is now likely to happen in Brett's sandboxing branch is that object.__subclasses__() and dangerous constructors like the one for the code object will be completely removed from the Python level. This means a few backwards incompatible changes, but Brett suggested that they should only break pretty advanced and esoteric Python code. Since it's for his Ph.D. dissertation, he didn't want to tie his hands by requiring full backwards compatibility, and he was fine with waiting to merge his branch until Python 3000.

Contributing threads:

• Capabilities / Restricted Execution
• new security doc using object-capabilities

Character case and locales

Mihai Ibanescu asked about a bug in the logging module due to the fact that 'INFO'.lower() != 'info' in some locales. Marc-Andre Lemburg and Martin v. Löwis explained that since in Unicode, nearly all case-conversions are only script-dependent, not language-dependent, u'INFO'.lower() == u'info' should always be true.

Contributing thread:

• logging module broken because of locale

Progress on the C version of the decimal module

After looking at the current progress in converting the decimal module to C, Raymond Hettinger suggested that rather than using the Python implementation as an outline of the C implementation, a separate C implementation should be developed and then later wrapped as necessary to provide the Python APIs. Tim Peters explained their incremental approach: leaving most of the module written in Python, and converting methods to C code one at a time. Raymond had originally supported this approach, but after viewing the current C code, thought that it would result in C code that was too complex and convoluted.

There was some extended discussion on the mechanism in the current decimal module for holding flags, which uses a dict mapping error types to the counts of their occurrences. Raymond in particular wanted the C decimal module to be able to change this API if it was too complex to implement. A number of others agreed that the API had been a bad decision, and it looked like there would at least be a note in the documentation for Python 2.5 suggesting that users should not rely on the counting feature.

Contributing thread:

• Strategy for converting the decimal module to C

PEP 357: Integer clipping and __index__

Armin Rigo pointed out that the current implementation of __index__() was incorrectly truncating long integers:

>>> (2**100).__index__()
2147483647

As the original __index__() method was intended only to allow things other than plain Python ints as slice indices, truncating to the maximum value was fine. However, when __index__() also became the "can you faithfully act like an integer" check, this truncation was no longer acceptable. Nick Coghlan spent some time reworking the PEP 357 C API so that all the use cases of __index__() were covered. His patch for fixing __index__ changes the nb_index slot to return a PyInt or PyLong instead of a C int, and introduces the C API functions PyNumber_Index, PyNumber_AsSsize_t and PyNumber_AsClippedSsize_t(), all of which have an output variable signifying whether or not the object they received had an __index__ method.

Contributing thread:

• Bad interaction of __index__ and sequence repeat

PEP 302: Non-importer objects on sys.path_importer_cache

Phillip J. Eby asked about how to best fix the non-PEP-302 compliant changes to the import machinery made by the Need for Speed Sprint. PEP 302 indicates that everything on sys.path_importer_cache should be either None or a valid importer object, but the Need for Speed changes added True and False values to that as well. After getting approval to make the appropriate changes necessary to stay PEP-302-compliant, Phillip added imp.NullImporter to replace False values, and kept None to mean that the builtin import machinery should be used.

Contributing threads:

• Undocumented PEP 302 protocol change by need-for-speed sprint
• Release manager pronouncement needed: PEP 302 Fix

Running the test suites of user projects when Python is updated

Grig Gheorghiu volunteered to do some of the work to get community buildbots running, that is, buildbots running the test suites of Python user projects whenever the Python core repository was updated. People were quite enthusiastic and Martin v. Löwis offered to set up a post-commit hook on the python repository to trigger a build on Grig's buildbots if necessary.

Contributing threads:

• Community buildbots
• Community buildbots (was Re: User's complaints)
• Community buildbots -- reprise

Safe dumper/loader using Python syntax

Sylvain Fourmanoit presented his miniconf module which is a safe and cross-version dumper/loader for simple objects using the Python syntax. People generally liked the module, and Phillip J. Eby helped Sylvain clean up the implementation a bit. There was some discussion of including it in the Python 2.6 stdlib and perhaps Bob Ippolito's simplejson module alongside it.

Contributing threads:

• New miniconf module
• JSON implementation in Python 2.6

Programmatically sending Ctrl-C

In testing his patch to make sockets and Ctrl-C play nicely, Tony Nelson found that he needed a portable way to send a Ctrl-C-like signal. For Unix, he was using signal.alarm, but was wondering if there was a way to get something similar on Windows. Martin v. Löwis pointed out GenerateConsoleCtrlEvent, but also noted that this would send the Ctrl-C to all processes. In the end, Tony decided to punt on Windows, and just stick with the Unix tests.

Contributing threads:

• Socket Timeouts patch 1519025
• Testing Socket Timeouts patch 1519025

Documenting performance of container types

Neal Becker asked about documentation the performance of the basic Python container types, e.g. that lookup in a dict is O(1) and deletion from the beginning of a list is O(N). A number of people agreed that having such information would be helpful, but there was some concern that Guido should be the one to decide what performances guarantees were made by the language, and not just the CPython implementation. The discussion trailed off before any final decisions on how to update the documentation were made.

Contributing thread:

• Document performance requirements?

Running the uuid test suite

Georg Brandl fixed a bug that was causing the new uuid module's test suite not to run at all. The resulting tests indicated a number of problems in determining a MAC address. Neal Norwitz patched the uuid module so that it should work at least on Linux, Tru64, Solaris, and HP-UX, and Tim Peters patched it so that test_uuid no longer thinks that the uuid module knows multiple ways of getting a well-defined MAC address (which it doesn't on Windows).

Contributing threads:

• uuid test suite failing
• how about adding ping's uuid module to the standard lib ?
• Another uuid problem
• test_uuid

CPython and checking for NULL values

Neal Norwitz took a look at some of the issues raised by the automatic analysis of Python's source code offered by Klocwork. There was a fairly long discussion around a Py_XINCREF of a variable that was required by the documentation to be non-NULL (and thus the "X" is unnecessary). There was some suggestion of trying to check for NULL values anyway, but the rest of Python doesn't do such checks.

Contributing thread:

• remaining issues from Klocwork static analysis

Excluding certain constructs from Python code

Boris Borcic suggested that to make changing version of Python easier, style sheets should be introduced such that you could allow or disallow particular constructs that you liked or didn't like. People thought this was generally a very bad idea as it would essentially introduce a bunch of language variants, and coders might not be able to read each others' source code without first applying the appropriate transformation.

Contributing threads:

• Python Style Sheets ? Re: User's complaints
• Python Style Sheets ? Re: User's complaints

Making attributes with leading single underscores private

David Hopwood proposed enforcing the PEP 8 convention that attributes with a single underscore are private to that object. His approach revolved around allowing only the first argument of a function to access attributes starting with '_', but Armin Rigo and others felt that this was not likely to be enforceable, giving an example where subclassing allowed access to supposedly private attributes. People generally felt that without an implementation to back up the proposal, there wasn't much to discuss.

Contributing thread:

• Internal namespace proposal

Loading module attributes on demand

In order to reduce the memory consumption of GTK+ applications written in Python, Johan Dahlin was looking into dynamically generating module attributes so that they would only be loaded when the application actually accessed them. He was able to produce this behavior by subclassing ModuleType, overridding __getattribute__, and then putting this object onto sys.path, but he felt like this was kind of a hackish solution. Phillip J. Eby pointed out the importing package and said that the __getattribute__ approach was generally okay, though it would cause problems for pydoc and inspect which don't handle subclasses of ModuleType well. Andrew Bennetts pointed out mercurial's demandload which allows modules to be imported on demand, but this didn't really solve Johan's problem because all attributes of the modules themselves were still imported at the same time.

Contributing thread:

• Dynamic module namspaces

Improving the Python test suite

Matt Fleming has put together a wiki page indicating the tests that are currently missing from Python's test suite, as well as the tests that are incomplete. He plans on working his way through the list when he gets some time, but help for any of the tests is quite welcome.

Contributing thread:

• Improving unit tests for the standard library

Improving the Python documentation

Georg Brandl, referring to bug 469773, suggested that python gain a "Using Python" page containing the man page and how to invoke the interpreter. He also suggested that creating a list of frequently needed documentation sections that are hard to find for newbies could go a long way towards making the Python documentation more user-friendly.

Contributing thread:

• Using Python docs

Commenting on Topics

To comment on anything mentioned here, just post to comp.lang.python (or email python-list@python.org which is a gateway to the newsgroup) with a subject line mentioning what you are discussing. All python-dev members are interested in seeing ideas discussed by the community, so don't hesitate to take a stance on something. And if all of this really interests you then get involved and join python-dev!

How to Read the Summaries

This summary is written using reStructuredText. Any unfamiliar punctuation is probably markup for reST (otherwise it is probably regular expression syntax or a typo :); you can safely ignore it. We do suggest learning reST, though; it's simple and is accepted for PEP markup and can be turned into many different formats like HTML and LaTeX.