python-dev Summary for 2003-01-01 through 2003-01-15
Brett C.
drifty@bigfoot.com
17 Jan 2003 14:43:03 -0800
+++++++++++++++++++++++++++++++++++++++++++++++++++++
python-dev Summary for 2003-01-01 through 2002-01-15
+++++++++++++++++++++++++++++++++++++++++++++++++++++
This is a summary of traffic on the `python-dev mailing list`_ between
January 1, 2003 and January 15, 2003 (inclusive). It is intended to
inform the wider Python community of on-going developments on the list
that might interest the wider Python community. To comment on
anything mentioned here, just post to python-list@python.org or
comp.lang.python in the usual way; give your posting a meaningful
subject line, and if it's about a PEP, include the PEP number (e.g.
Subject: PEP 201 - Lockstep iteration). All python-dev members are
interested in seeing ideas discussed by the community, so don't
hesitate to take a stance on something. And if all of this really
interests you then get involved and join python-dev!
This is the ninth summary written by Brett Cannon (and fashionably
late, no less!).
All summaries are archived at http://www.python.org/dev/summary/ .
Please note that this summary is written using reStructuredText_ which
can be found at http://docutils.sf.net/rst.html . Any unfamiliar
punctuation is probably markup for reST_; you can safely ignore it
(although I suggest learning reST; its simple and is accepted for PEP
markup). Also, because of the wonders of programs that like to
reformat text, I cannot guarantee you will be able to run the text
version of this summary through Docutils_ as-is unless it is from the
original itself.
.. _python-dev mailing list:
http://mail.python.org/mailman/listinfo/python-dev
.. _Docutils: http://docutils.sf.net/
.. _reST:
.. _reStructuredText: http://docutils.sf.net/rst.html
.. contents::
======================
Summary Announcements
======================
This summary has been written in the first-person. Aahz had suggested
I write in third-person, but I just can't stand it. =)
A new project named `Minimal Python`_ whose goal is to write a Python
interpreter using a minimal amount of C and maximizing the amount of
code written in Python. As of now it is just a mailing list but some
interesting points have already been made and it has some major Python
programmers involved with it so it will more than likely succeed.
Leaving out `Broken strptime in Python 2.3a1 & CVS`_ for two reasons.
One, the only lesson in that thread is that when C does not give you a
good standard, rewrite the code in Python and come up with one. Two,
since the whole thing revolves around my code I would be more biased
than I normally am. =)
Also, some of the threads have notes with them mentioning how I could
not find them in the Mailman archive.
And go to PyCon_ (especially since I am going to be giving a tutorial
on reST_)!
.. _Minimal Python: http://codespeak.net/mailman/listinfo/pypy-dev
.. _Broken strptime in Python 2.3a1 & CVS:
http://mail.python.org/pipermail/python-dev/2003-January/031983.html
.. _PyCon: http://www.python.org/pycon/
===================================================
`PEP 303: Extend divmod() for Multiple Divisors`__
===================================================
__ http://mail.python.org/pipermail/python-dev/2002-December/031511.html
Splinter threads:
- map, filter, reduce, lambda (can't find thread in Mailman archive)
As mentioned in the `last summary`_, this thread sparked a huge
response to the built-ins in Python (if you don't remember what all
the built-ins are, just execute ``dir(__builtins__)`` in the
interpreter). It started with Christian Tismer calling for the
deprecation of ``divmod()`` since he "found that divmod compared to /
and % is at least 50 percent *slower* to compute", primarily because
``divmod()`` is a function and the overhead that comes from being
that. Guido responded and agree that, in hindsight, "Maybe it wasn't
such a good idea". But since it isn't broken he is not for replacing
it since "At this point, any change causes waste". Besides, Tim
Peters came up with the great compromise of removing " divmod() when
[Tim's] dead. Before then, it stays."
Guido would rather have the energy spent on adding the ability for
the language to recognize that it "is a built-in so it can generate
more efficient code". This led to Christian suggesting changing
Python so that built-ins like ``divmod()`` that can easily be written
in Python be done so and thus remove the C version without any
backwards-compatibility issues. Guido's response was to "first make
some steps towards the better compiler [Guido] alluded to. Then we can
start cutting."
Tim in an email said that "If we have to drop a builtin, I never liked
reduce <wink>". This then led to Barry Warsaw saying he could stand
to lose ``apply()``. Then Raymond Hettinger nominated ``buffer()``
and ``intern()`` as things that could stand to go away. Michael
Hudson suggested ``map()`` and ``filter()``.
Guido said ``apply()`` could stand getting a PendingDeprecationWarning
since the new calling conventions (``fxn(*args, **kwargs)``) replace
``apply()``'s use. He almost mentioned how he wanted something like
``foo(1, 2, 3, *args, 5, 6, 7, **kw)`` since the ``5, 6, 7`` is
currently not supported.
But the call for the removal of ``lambda`` is what really set things
off. People in support of the removal said that nested scoping
removed a huge reason for having ``lambda``. But functional
programmers said that they wouldn't let ``lambda`` be taken away. It
was agreed it had its uses, but the name was not relevant to its use
and its syntax is non-standard and should be changed in Python 3000.
Someone suggested renaming it ``def``, but Guido said no since
"Python's parser is intentionally simple-minded and doesn't like
having to look ahead more than one token" and using ``def`` for
``lambda`` would break that design. Barry suggested ``anon`` which
garnered some support.
It was also suggested to have ``lambda`` use parentheses to hold its
argument list like a good little function.
And then all of this led to a discussion over features in Python that
are not necessarily easy to read but are powerful. Some said that
features such as list comprehensions, the ``*/**`` calling syntax,
etc., were not good since they are not syntactically obvious. This
was argued against by saying they are quite useful and are not needed
for basic programming.
.. _last summary: http://www.python.org/dev/summary/2002-12-16_2002-12-31.html
==================
`Holes in time`__
==================
__ http://mail.python.org/pipermail/python-dev/2003-January/031836.html
Tim Peters brought up an inherent problem with the new datetime_ type
and dealing with timezones and daylight savings. Imagine when DST
starts; the clock jumps from 1:59 to 3:00. Now what happens if
someone enters a time that does not use DST? The implementation has
to handle the possibility of something inputting 2:00 and instantly
pushing forward an hour. But what is worse is when DST ends. You go
from 1:59 back to 1:00! That means you pass over everything from 1:00
to 1:59 twice during that day. Since it is flat-out impossible to
tell, what should be done in this case?
Originally ``datetime`` raised a ValueError. It has now been changed
to "go through 1:00am to 1:59am twice" according to Aahz.
If you want to read a paper on the history of time go to
http://www.naggum.no/lugm-time.html as recommended by Neil
Schemenauer.
.. _datetime: http://www.python.org/dev/doc/devel/lib/module-datetime.html
===========================================
`no expected test output for test_sort?`__
===========================================
__ http://mail.python.org/pipermail/python-dev/2003-January/031889.html
This thread was originally started by Skip Montanaro to try to figure
out why ``test_sort`` was failing to catch an error when run through
``regrtest.py``. But Skip quickly realized ``regrtest.py`` was using
the wrong directories.
What makes this thread worth mentioning is a question I posed about
whether it was worth converting tests over to PyUnit. Guido said not
for the sake of converting, but if you were going to be improving upon
the tests, then moving a testing suite over to PyUnit was a good idea.
It was also said that all new testing suites should use PyUnit
(although doctest is still acceptable, just not preferred). Walter
Dörwald started `patch #662807`_ to use for converting tests to
PyUnit. So give a hand if you care to.
.. _patch #662807: http://www.python.org/sf/662807
============
binutils
============
(Can't find in Mailman archive)
Andrew Koenig told python-dev of a problem with binutils_ 2.13 way
back in the `2002-089-16 through 2002-09-01 summary`_. Well, it
appears that 2.13.2 fixes the issues.
.. _2002-089-16 through 2002-09-01 summary:
http://www.python.org/dev/summary/2002-08-16-2002-09-01.html
===========================
The meaning of __all__
===========================
(Can't find in Mailman archive)
Jack Jansen asked what the proper use of ``__all__``; a discussion on
the PyObjC_ mailing list sparked this question. The answer is that
``__all__`` is used for the public API of a module.
.. _PyObjC: http://pyobjc.sourceforge.net/
======================================
PEP 301 implementation checked in
======================================
(Can't find in Mailman archive;
http://mail.python.org/pipermail/python-list/2003-January/137270.html
goes to python-list discussion)
AM Kuchling announced that the `PEP 301`_ implementation by Richard
Jones. The web server handling everything is at `amk.ca`_ off of a
DSL line so don't hit it too hard. An example of how to change a
Distutils ``setup.py`` file is given in the e-mail.
.. _PEP 301: http://www.python.org/peps/pep-0301.html
.. _amk.ca: http://www.amk.ca/cgi-bin/pypi.cgi
=======================================
bz2 problem deriving subclass in C
=======================================
(Can't find in Mailman archive)
This thread was originally about a patch by Neal Norwitz and how to
properly deal with deallocating in C code. Apparently this has nipped
several people in the bum and has changed since the introduction of
new-style classes. It appears that if you are deallocating a base
type, you call its ``tp_dealloc()`` function; if it is a new-style
class, you call ``self->ob_type->tp_free()``.
====================
`Cross compiling`__
====================
__ http://mail.python.org/pipermail/python-dev/2003-January/031848.html
(Link only part of thread; rest missing)
Splinter threads:
- `Bastion too <http://mail.python.org/pipermail/python-dev/2003-January/031851.html>`__
- `What attempts at security should/can Python implement?
<http://mail.python.org/pipermail/python-dev/2003-January/031871.html>`__
- `Whither rexec? <http://mail.python.org/pipermail/python-dev/2003-January/031842.html>`__
- `tainting <http://mail.python.org/pipermail/python-dev/2003-January/031960.html>`__
Originally a thread about cross-compilation on OS X to Windows by
Timothy Wood, this ended up changing into a thread about security in
Python.
As has come up multiple times before, rexec_'s lack of actual security
for new-style classes came up. This then led to a question of whether
Bastion_ was secure as well. The answer was no and have now been
subsequently sabotaged on purpose by Guido; they now require you to
manually edit the code to allow them to work.
A discussion on how to make Python secure came up. Various points
were made such as how to deal with introspection and how people might
try to get around being locked out of features. Tainting was also
mentioned for strings.
If you need some security in your app, take a look at mxProxy_. Its
design is similar to how Zope3_ handles security.
.. _rexec: http://www.python.org/doc/current/lib/module-rexec.html
.. _Bastion: http://www.python.org/doc/current/lib/module-Bastion.html
.. _mxProxy: http://www.egenix.com/files/python/mxProxy.html
.. _Zope3: http://dev.zope.org/Wikis/DevSite/Projects/ComponentArchitecture/FrontPage
===========================
`tutor(function|module)`__
===========================
__ http://mail.python.org/pipermail/python-dev/2003-January/031893.html
(Only part of thread; rest missing from archive)
Christopher Blunk proposed the idea of having a built-in
``examples()`` function or something similar that would spit out
example code on how to use something. There was a discussion as to
whether it was reasonable for Python to try to keep up a group of
examples in the distribution. It was agreed, though, that more code
examples would be good to have *somewhere*.
One place is in the Demo_ directory of the source distribution.
Problem with that is it is only available in the source distribution
and it is not vigorously maintained. Both of these are solvable, but
it didn't look like anyone was taking the reigns to cause this to
happen.
Another was to push people to give to an online repository (such as
the `Python Cookbook`_) to keep it centralized and accessible by
anyone. Problem with that is it creates a dependency on something
outside of PythonLab's control.
The last option was adding it to the module documentation. This is
the safest since it means it becomes permanent and everyone will have
access to it. But it has the same drawback as all of these other
suggestions; you can't access it in the interpreter as easily as you
can ``help()``.
The thread ended with no conclusion beyond cleaning up the ``Demos``
directory and adding more examples to the official documentation would
be nice.
.. _Demo: http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/python/python/dist/src/Demo/
.. _Python Cookbook: http://aspn.activestate.com/ASPN/Python/Cookbook/
=========================================
`PEP 297: Support for System Upgrades`__
=========================================
__ http://mail.python.org/pipermail/python-dev/2003-January/031838.html
MA Lemburg asked if Guido would be willing to let an implementation of
`PEP 297`_ into Python 2.3. He said yes and would also be willing to
add it to 2.2.3 and even 2.1.4 if that version ever comes about.
A discussion of how to handle the implementation began. The agreed
upon directory name became ``site-package-<Python-version>`` with
``<Python version>`` being the full version: major.minor.micro.
``site.py`` will be modified so that if the current version of Python
is different from the version of the directory that it won't add it to
``sys.path``.
There was the brief idea of deleting the directory as soon as the
version did not match, but that was thought of being harsh and
unneeded.
.. _PEP 297: http://www.python.org/peps/pep-0297.html
================================
`PyBuffer* vs. array.array()`__
================================
__ http://mail.python.org/pipermail/python-dev/2003-January/031841.html
(Only part of thread; rest missing from archive)
Splinter threads:
- `Slow String Repeat
<http://mail.python.org/pipermail/python-dev/2003-January/031895.html>`__
Bill Bumgarner asked whether ``PyBuffer*`` or ``array.array()`` was
the proper thing to use because one is like a huge string and the
other is unsigned ints. Guido basically said to just use strings like
everyone else for byte storage. Bill said he would just live with the
dichotomy.
This thread also brought up a question of performance for the code
``array.array('B').fromlist([0 for x in range(0, width*height*3)])``.
Was cool to watch various people take stabs as speeding it up.
Written in Python, the fast way was to change it to ``array.array('B',
[0])*width*height*3``; doing several decent-sized calls minimized
``memcpy()`` calls in the underlying C code. Raymond Hettinger came
up with a fast way to do it in C.
==========================
`new features for 2.3?`__
==========================
__ http://mail.python.org/pipermail/python-dev/2003-January/031837.html
Neal Norwitz wanted to get the tarfile_ module into Python 2.3. This
led to Guido saying he wanted to get Docutils_ and better pickling for
new-style classes in as well.
The Docutils idea was shot down because it was stated it currently is
not ready for inclusion. 2.4 is probably the version being shot for.
The pickle idea raised security ideas (this is becoming a trend for
some reason on the list lately). In case you didn't know, you
shouldn't run pickled code you don't trust.
.. _tarfile: http://python.org/sf/651082
================
`sys.path[0]`__
================
__ http://mail.python.org/pipermail/python-dev/2003-January/031896.html
Thomas Heller had a question about the documented handling of
``sys.path[0]``. This quickly led to the point that the value, when
specified and not the empty string, is a relative path; not
necessarily the desired result. And so `patch #664376`_ was opened to
make it be absolute.
.. _patch #664376: http://www.python.org/sf/664376
===================
`Misc. warnings`__
===================
__ http://mail.python.org/pipermail/python-dev/2003-January/031884.html
MA Lemburg noticed that ``test_ossaudiodev`` and ``test_bsddb3`` were
not expected skips on his Linux box. He wondered why they were not
exepected to be skipped on his platform. This led to a discussion
over the expected skip set and how it is decided what is going to be
skipped (experts on the various OSs make the call).
==============================
`Raising string exceptions`__
==============================
__ http://mail.python.org/pipermail/python-dev/2003-January/031907.html
It looks like string exceptions are going to raise
PendingDeprecationWarning; you have been warned.
======================
`PEP 290 revisited`__
======================
__ http://mail.python.org/pipermail/python-dev/2003-January/032003.html
Kevin Altis was cleaning up wxPython_ and all the references to the
``string`` module and pointed out that the stdlib probably could stand
to have a similar cleaning as well. Guido said, though, that it is
better to choose a module you are familiar with and do a complete
style clean-up instead of doing a library-wide clean-up of a single
thing.
.. _wxPython: http://www.wxpython.org/
============================
`Assignment to __class__`__
============================
__ http://mail.python.org/pipermail/python-dev/2003-January/032016.html
Kevin Jacobs got bitten by the change to non-heap objects that
prevents the assigning to ``__class__`` (it was recently backported to
the 2.2 branch). He asked if there was any way to bring it back;
Guido said no because of various layout issues in the C struct and
such.
=================================
`Parallel pyc construction`__
=================================
__ http://mail.python.org/pipermail/python-dev/2003-January/032060.html
Paul Dubois ran into an problem on a 384 processor system with .pyc
file corruption. He wanted a way to just skip the creation of .pyc
files entirely since his programs run for so long that initial
start-up times are inconsequential and that is the point of .pyc
files. Neal Norwitz is working on `patch #602345`_ to implement a
command-line option.
.. _patch #602345: http://www.python.org/sf/602345
==============================================
`Extension modules, Threading, and the GIL`__
==============================================
__ http://mail.python.org/pipermail/python-dev/2003-January/032017.html
This thread was briefly mentioned in the `last summary`_. But it had
changed into a discussion of Python external thread API. Mark Hammond
commented how he had bumped up against its limitations in multiple
contexts. Mark thought it would be useful to define an API where one
wanted the ability to say "I'm calling out to/in from an external
API". This would be a large project, though, and so Mark was not up
to doing it on his own. David Abrahams, though, stepped up and said
he was willing to help. So did Anthony Baxter.
Tim Peters had his own twist by wanting to be able to call from a
thread into a Python thread without knowing a single thing about that
state of Python at that point, e.g. don't even know that that
interpreter is done initializing. Mark saw the way to go about
solving all of this by writing a PEP, write a new C API to solve this
issue that is optional and used by extension modules wanting to use
the new features, and then define a new TLS (Thread Local Storage;
thank you acronymfinder.com). Mark thought the TLS would only be
needed for thread-state. Someone suggested ACE_ as a TSS (Thread
State Storage) implementation which can cover all the platforms that
Python needs covered. But Mark suggested coming up with a "pluggable
TLS" design.
.. _ACE: http://doc.ece.uci.edu/Doxygen/Beta/html/ace/classACE__TSS.html
============================================
`Interop between datetime and mxDateTime`__
============================================
__ http://mail.python.org/pipermail/python-dev/2003-January/032100.html
MA Lemburg expressed interest in trying to get mxDateTime_ to play
nice with datetime_. The idea of having a base type like the new
basestring in Python 2.3 was suggested. But there was pickling issues
with datetime, MA wanting to keep mxDateTime compatible with Python
1.5.2, and getting everyone to work in a consistent way through the
base type. Another instance where interfaces would be handy.
.. _mxDateTime: http://www.lemburg.com/files/python/mxDateTime.html
===========================
`properties on modules?`__
===========================
__ http://mail.python.org/pipermail/python-dev/2003-January/032106.html
Neil Schemenauer wanted to be able to make module variables be
properties. Samuele Pedroni stepped up and said the issue for this to
work was "about instance-level property apart the module thing
details". He also pointed out that it would mean you could directly
import a property which would cause unexpected results, thus killing
the idea. Guido was disappointed he didn't get to wield his BDFL
pronouncement stick and smack the idea down <wink>.
======================================
`Fwd: Re: PEP 267 improvement idea`__
======================================
__ http://mail.python.org/pipermail/python-dev/2003-January/032165.html
Oren Tirosh forwarded an e-mail from python-list@python.org to
python-dev about `PEP 267`_ and various strategies. One was using
negative keys for signaling guaranteed key lookup failure. Oren also
played with inlining. He also said that since interned strings are
now out of Python as of 2.3 (thanks to Oren) that the namespace could
"have only interned strings as entry keys and avoid the need for a
second search pass on first failed lookup". He even considered
creating a special dict type for namespaces.
.. _PEP 267: http://www.python.org/peps/pep-0267.html