From python3now at gmail.com  Mon Jan  1 17:31:59 2007
From: python3now at gmail.com (James Thiele)
Date: Mon, 1 Jan 2007 08:31:59 -0800
Subject: [Python-3000] Status of True, False, bool type
Message-ID: <8f01efd00701010831s3303f667pcd8eb27b382209eb@mail.gmail.com>

PEP 3100 states:
None becomes a keyword [4] (What about True, False?)

Has a decision been made?
--
Also a bool acts like an int in various contexts. For example:
>>> True == 1
True
>>> False == 0
True
>>> 16 + (0 == 0)
17

Will this behavior continue?

From guido at python.org  Mon Jan  1 17:50:11 2007
From: guido at python.org (Guido van Rossum)
Date: Mon, 1 Jan 2007 08:50:11 -0800
Subject: [Python-3000] Status of True, False, bool type
In-Reply-To: <8f01efd00701010831s3303f667pcd8eb27b382209eb@mail.gmail.com>
References: <8f01efd00701010831s3303f667pcd8eb27b382209eb@mail.gmail.com>
Message-ID: <ca471dc20701010850w1f482455jfd6b3c1c04c86ff0@mail.gmail.com>

On 1/1/07, James Thiele <python3now at gmail.com> wrote:
> PEP 3100 states:
> None becomes a keyword [4] (What about True, False?)
>
> Has a decision been made?

No, but I think the argument for making True/False reserved words is
pretty weak.

> Also a bool acts like an int in various contexts. For example:
> >>> True == 1
> True
> >>> False == 0
> True
> >>> 16 + (0 == 0)
> 17
>
> Will this behavior continue?

Yes. Don't confuse Python with Java.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From sluggoster at gmail.com  Mon Jan  1 20:13:57 2007
From: sluggoster at gmail.com (Mike Orr)
Date: Mon, 1 Jan 2007 11:13:57 -0800
Subject: [Python-3000] Status of True, False, bool type
In-Reply-To: <ca471dc20701010850w1f482455jfd6b3c1c04c86ff0@mail.gmail.com>
References: <8f01efd00701010831s3303f667pcd8eb27b382209eb@mail.gmail.com>
	<ca471dc20701010850w1f482455jfd6b3c1c04c86ff0@mail.gmail.com>
Message-ID: <6e9196d20701011113u7fbbbb43mb24e6635b9edc7c5@mail.gmail.com>

On 1/1/07, Guido van Rossum <guido at python.org> wrote:
> On 1/1/07, James Thiele <python3now at gmail.com> wrote:
> > PEP 3100 states:
> > None becomes a keyword [4] (What about True, False?)
> >
> > Has a decision been made?
>
> No, but I think the argument for making True/False reserved words is
> pretty weak.
>
> > Also a bool acts like an int in various contexts. For example:
> > >>> True == 1
> > True
> > >>> False == 0
> > True
> > >>> 16 + (0 == 0)
> > 17
> >
> > Will this behavior continue?
>
> Yes. Don't confuse Python with Java.

Huh?  'True == 1' is a "feature"?  '16 + (0 == 0)' being illegal is a
"Javaism"?  Would somebody care to explain this?  It's acceptable that
2 is true but not True?  Why do we need 1 for True at all if we have
True?

-- 
Mike Orr <sluggoster at gmail.com>

From tjreedy at udel.edu  Mon Jan  1 20:52:25 2007
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 1 Jan 2007 14:52:25 -0500
Subject: [Python-3000] Status of True, False, bool type
References: <8f01efd00701010831s3303f667pcd8eb27b382209eb@mail.gmail.com><ca471dc20701010850w1f482455jfd6b3c1c04c86ff0@mail.gmail.com>
	<6e9196d20701011113u7fbbbb43mb24e6635b9edc7c5@mail.gmail.com>
Message-ID: <enbopp$2eg$1@sea.gmane.org>


"Mike Orr" <sluggoster at gmail.com> wrote in message 
news:6e9196d20701011113u7fbbbb43mb24e6635b9edc7c5 at mail.gmail.com...
| Huh?  'True == 1' is a "feature"?  '16 + (0 == 0)' being illegal is a
| "Javaism"?  Would somebody care to explain this?  It's acceptable that
| 2 is true but not True?  Why do we need 1 for True at all if we have
| True?

Start with http://www.python.org/dev/peps/pep-0285/
There was also discussion on PyDev.

tjr




From jcarlson at uci.edu  Mon Jan  1 21:00:24 2007
From: jcarlson at uci.edu (Josiah Carlson)
Date: Mon, 01 Jan 2007 12:00:24 -0800
Subject: [Python-3000] Status of True, False, bool type
In-Reply-To: <6e9196d20701011113u7fbbbb43mb24e6635b9edc7c5@mail.gmail.com>
References: <ca471dc20701010850w1f482455jfd6b3c1c04c86ff0@mail.gmail.com>
	<6e9196d20701011113u7fbbbb43mb24e6635b9edc7c5@mail.gmail.com>
Message-ID: <20070101115005.82F2.JCARLSON@uci.edu>


"Mike Orr" <sluggoster at gmail.com> wrote:
> Huh?  'True == 1' is a "feature"?  '16 + (0 == 0)' being illegal is a
> "Javaism"?  Would somebody care to explain this?  It's acceptable that
> 2 is true but not True?  Why do we need 1 for True at all if we have
> True?

    >>> 1 is not True
    True
    >>> 1 == True
    True
    >>> isinstance(True, int)
    True

If you are suggesting that we change this behavior, stop, it is not
going to happen. If you are asking why this is the case, I would imagine,
is based in C/C++/many other sane and insane languages' handling of
boolean true.

 - Josiah


From tony at PageDNA.com  Mon Jan  1 21:45:04 2007
From: tony at PageDNA.com (Tony Lownds)
Date: Mon, 1 Jan 2007 12:45:04 -0800
Subject: [Python-3000] PEP 3107 Function Annotations: overloadable ->
References: <E3F0C6CC-C09E-4084-9C7C-3115C1BD5F37@PageDNA.com>
Message-ID: <06435893-735C-4DC3-BE48-B792B4BCC367@PageDNA.com>


> From: Tony Lownds <tony at pagedna.com>
>
> On Jan 1, 2007, at 9:48 AM, Kay Schluehr wrote:
>> Good. There is still one issue. I understand that you don't want to
>> fix
>> the semantics of function annotations but to be usefull some
>> annotations are needed to express function types. Using those
>> consistently with the notation of the enhanced function statement I
>> suggest introducing an arrow expression and an __arrow__ special
>> function:
>>
>> expr: arrow_expr ('->' arrow_expr)*
>> arrow_expr: xor_expr ('|' xor_expr)*
>>
>
> I agree with the use case and I am in favor of this addition  
> despite the
> drawbacks below. While overloading __eq__ is a decent alternative,
> the -> operator is so much nicer (IMO).
>
> The precedence seems right:
>
> Function(A) -> B | C    <=>  Function(A) -> (B | C)
>
> Most operators special method names refer to the action or operation
> rather that the symbol, eg __or__, not __vbar__. Also, since the
> token is called RARROW, __arrow__ / __rarrow__ would be potentially
> easy to mix up.
>
> There might be opposition to adding an operator whose meaning in C
> is very different. Also the operator as suggested does not have any
> meaning on any built in objects, which is odd. We could add a meaning
> for ints/bools:
>
> http://mathworld.wolfram.com/Implies.html
>
> If that is reasonable I would suggest calling the special method
> __implies__ and putting the slot on PyNumberMethods. If that's just
> silly, I  suggest calling the special method __returns__.
>
> We'll see what others say :) Thanks for the suggestion!
>

What do people here think?

Thanks
-Tony

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20070101/bbcd1da4/attachment.html 

From talin at acm.org  Mon Jan  1 22:11:59 2007
From: talin at acm.org (Talin)
Date: Mon, 01 Jan 2007 13:11:59 -0800
Subject: [Python-3000] PEP 3107 Function Annotations: overloadable ->
In-Reply-To: <06435893-735C-4DC3-BE48-B792B4BCC367@PageDNA.com>
References: <E3F0C6CC-C09E-4084-9C7C-3115C1BD5F37@PageDNA.com>
	<06435893-735C-4DC3-BE48-B792B4BCC367@PageDNA.com>
Message-ID: <4599791F.9020505@acm.org>

Tony Lownds wrote:

>> From: Tony Lownds <tony at pagedna.com>
> What do people here think?

I've been wanting something like this, however the name I wanted to use 
was "__becomes__" and "__rbecomes__". My reasoning is as follows:

1) Normally, we don't name operators based on their shape - we don't 
call '/' the __slash__ operator, for example, nor do we call '|' the 
"__vbar__" operator.

2) I think that all operators should have a "suggested semantic". When 
someone overloads the '+' operator, its a good bet that the meaning of 
the overload has something to do with addition or accumulation in a 
general sense. This won't *always* be true, but it will be true often 
enough.

But an arbitrary operator with no guidelines as to what it means is 
anyone's guess; It means that when we see a '->' operator embedded in 
the code, we have no idea what is being said.

 From an HCI perspective, punctuation symbols improve code readability 
only if their meanings are familiar to the reader; An operator whose 
meaning is constantly changing is a hindrance to readability rather than 
a help.

(This is why, although I have lobbied for new operators in the past, I 
have lobbied against a generic operator declaration system a la Logix. 
This was recently discussed on the python-ideas list.)

3) I suggest that the expression:

   a -> b

should be read as:

   "a becomes b"

which is why I propose the name "becomes". (An alternative is 
"produces", but that's longer.)

-- Talin

From talin at acm.org  Mon Jan  1 22:17:34 2007
From: talin at acm.org (Talin)
Date: Mon, 01 Jan 2007 13:17:34 -0800
Subject: [Python-3000] Status of True, False, bool type
In-Reply-To: <20070101115005.82F2.JCARLSON@uci.edu>
References: <ca471dc20701010850w1f482455jfd6b3c1c04c86ff0@mail.gmail.com>	<6e9196d20701011113u7fbbbb43mb24e6635b9edc7c5@mail.gmail.com>
	<20070101115005.82F2.JCARLSON@uci.edu>
Message-ID: <45997A6E.6040503@acm.org>

Josiah Carlson wrote:
>     >>> 1 is not True
>     True
>     >>> 1 == True
>     True
>     >>> isinstance(True, int)
>     True
> 

This makes perfect sense to me. 'True' is implicitly convertible to, but 
not identical to, 1.

(I was about to type a long-winded explanation of why this makes sense 
from a historical and philosophical perspective, but I think I'll just 
skip it for now.)

-- Talin

From guido at python.org  Mon Jan  1 22:20:04 2007
From: guido at python.org (Guido van Rossum)
Date: Mon, 1 Jan 2007 13:20:04 -0800
Subject: [Python-3000] PEP 3107 Function Annotations: overloadable ->
In-Reply-To: <4599791F.9020505@acm.org>
References: <E3F0C6CC-C09E-4084-9C7C-3115C1BD5F37@PageDNA.com>
	<06435893-735C-4DC3-BE48-B792B4BCC367@PageDNA.com>
	<4599791F.9020505@acm.org>
Message-ID: <ca471dc20701011320t41e538c8yf217a50fdc9900bd@mail.gmail.com>

This is sufficiently controversial that I believe it ought to go to
python-ideas first. If it comes to a PEP it should be a separate one
from PEP 3107.

On 1/1/07, Talin <talin at acm.org> wrote:
> Tony Lownds wrote:
>
> >> From: Tony Lownds <tony at pagedna.com>
> > What do people here think?
>
> I've been wanting something like this, however the name I wanted to use
> was "__becomes__" and "__rbecomes__". My reasoning is as follows:
>
> 1) Normally, we don't name operators based on their shape - we don't
> call '/' the __slash__ operator, for example, nor do we call '|' the
> "__vbar__" operator.
>
> 2) I think that all operators should have a "suggested semantic". When
> someone overloads the '+' operator, its a good bet that the meaning of
> the overload has something to do with addition or accumulation in a
> general sense. This won't *always* be true, but it will be true often
> enough.
>
> But an arbitrary operator with no guidelines as to what it means is
> anyone's guess; It means that when we see a '->' operator embedded in
> the code, we have no idea what is being said.
>
>  From an HCI perspective, punctuation symbols improve code readability
> only if their meanings are familiar to the reader; An operator whose
> meaning is constantly changing is a hindrance to readability rather than
> a help.
>
> (This is why, although I have lobbied for new operators in the past, I
> have lobbied against a generic operator declaration system a la Logix.
> This was recently discussed on the python-ideas list.)
>
> 3) I suggest that the expression:
>
>    a -> b
>
> should be read as:
>
>    "a becomes b"
>
> which is why I propose the name "becomes". (An alternative is
> "produces", but that's longer.)
>
> -- Talin
> _______________________________________________
> Python-3000 mailing list
> Python-3000 at python.org
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From tony at pagedna.com  Mon Jan  1 22:54:33 2007
From: tony at pagedna.com (Tony Lownds)
Date: Mon, 1 Jan 2007 13:54:33 -0800
Subject: [Python-3000] PEP 3107 Function Annotations: overloadable ->
In-Reply-To: <ca471dc20701011320t41e538c8yf217a50fdc9900bd@mail.gmail.com>
References: <E3F0C6CC-C09E-4084-9C7C-3115C1BD5F37@PageDNA.com>
	<06435893-735C-4DC3-BE48-B792B4BCC367@PageDNA.com>
	<4599791F.9020505@acm.org>
	<ca471dc20701011320t41e538c8yf217a50fdc9900bd@mail.gmail.com>
Message-ID: <44CB651A-DDC1-4B3F-A054-4829020E5A13@pagedna.com>


On Jan 1, 2007, at 1:20 PM, Guido van Rossum wrote:

> This is sufficiently controversial that I believe it ought to go to
> python-ideas first. If it comes to a PEP it should be a separate one
> from PEP 3107.
>

Ok. Some other discussion on PEP 3107 (attribute annotations)
clearly should follow suit, and be a seperate PEP from python-ideas.

One last issue: interoperability came up again. This issue came up
before in a rather long thread starting here:

http://mail.python.org/pipermail/python-3000/2006-August/002796.html

I'm going to see if resolution can't be reached on python-ideas as  
well, to
avoid huge threads here. Then the PEP will be ready for pronouncement.

Thanks
-Tony



From rrr at ronadam.com  Mon Jan  1 22:57:00 2007
From: rrr at ronadam.com (Ron Adam)
Date: Mon, 01 Jan 2007 15:57:00 -0600
Subject: [Python-3000] Status of True, False, bool type
In-Reply-To: <6e9196d20701011113u7fbbbb43mb24e6635b9edc7c5@mail.gmail.com>
References: <8f01efd00701010831s3303f667pcd8eb27b382209eb@mail.gmail.com>	<ca471dc20701010850w1f482455jfd6b3c1c04c86ff0@mail.gmail.com>
	<6e9196d20701011113u7fbbbb43mb24e6635b9edc7c5@mail.gmail.com>
Message-ID: <enc0bh$mbs$1@sea.gmane.org>

Mike Orr wrote:
> On 1/1/07, Guido van Rossum <guido at python.org> wrote:
>> On 1/1/07, James Thiele <python3now at gmail.com> wrote:
>>> PEP 3100 states:
>>> None becomes a keyword [4] (What about True, False?)
>>>
>>> Has a decision been made?
>> No, but I think the argument for making True/False reserved words is
>> pretty weak.
>>
>>> Also a bool acts like an int in various contexts. For example:
>>>>>> True == 1
>>> True
>>>>>> False == 0
>>> True
>>>>>> 16 + (0 == 0)
>>> 17
>>>
>>> Will this behavior continue?
>> Yes. Don't confuse Python with Java.
> 
> Huh?  'True == 1' is a "feature"?  '16 + (0 == 0)' being illegal is a
> "Javaism"?  Would somebody care to explain this?  It's acceptable that
> 2 is true but not True?  Why do we need 1 for True at all if we have
> True?

None of the above examples are inconsistent with boolean algebra combined with 
pythons practice of promoting types in various situations.  Since bool is a 
subclass of int, it doesn't need promoting to int in these cases, but the 
results would be the same.

    http://www.allaboutcircuits.com/vol_4/chpt_7/2.html


The only place (I know of) where it seems out of order would be examples like...

    >>> True + True
    2
    >>> True == (True + True)
    False

Here the result of the addition two bools is an int instead of being a bool.


The AbstractBaseClass wiki has questions concerning where bool is in the class 
tree.  It shows bool as *not* being a subclass of int.

     http://wiki.python.org/moin/AbstractBaseClasses


So would adding two bools return an int or a bool if this is changed?

Should Bool be a subclass of Numeric instead of object?


The way python uses bools makes me think maybe the relationship between integers 
and bools should be reversed.  An integer could be a subclass of bool.  Then all 
positive integers, would also be equal to True.  But that would change the 
behavior of adding integers and bools, because the result would be either a True 
or False instead of an int because the int would be promoted to bool instead of 
the bool being (like promoted to) an int.

Cheers,
    Ron




From tomerfiliba at gmail.com  Mon Jan  1 23:38:30 2007
From: tomerfiliba at gmail.com (tomer filiba)
Date: Tue, 2 Jan 2007 00:38:30 +0200
Subject: [Python-3000] self-contained exceptions
Message-ID: <1d85506f0701011438q6d8b65bfi4ef1e8f9fa509650@mail.gmail.com>

a while back some people suggested having the traceback
as an attribute of the exception object, which allows for
nested exceptions.

i would also like to note that having the traceback part of
the exception object allows the exception to print itself,
i.e., its repr would format the traceback text on it's own.

this makes sys.excepthook unnecessary, as well as
traceback.format_exception (which i greatly dislike).

even if the exception's repr would raise an exception
(recursion limit, etc.), that exception would be a builtin
one, so we won't end up in a repr-exception-loop.

with this approach, sys.excepthook is needless, or at least
simplified greatly (just print repr(exc)). for example, exceptions
raised by RPyC have a special attribute to them -- remote_traceback,
which is the text of the remote traceback. currently i have to
replace excepthook to have it displayed with the local exception
traceback, which is means just importing RPyC messes with
your interpreter's internals, and may cause problems when
another library installs its own hook.

apart from self-repr'ing, nested exceptions are very useful too:

try:
    foo()
except Exception, ex:
    raise MyException("something happened", nested = ex)

so that when i pdb.pm() my code, i can inspect the inner
exception too (which is desirable of course).

the major argument against having the traceback part of
the exception object was increasing memory consumption,
so that exceptions are no longer "light weight" (as used by
StopIteration, etc.)

i don't see why this argument stands. the traceback object
exists either way, the only question is in what namespace
it exists, and the fact it may "live longer" than necessary,
i.e., the traceback exists for as long as the exception object
exists.

i don't think it's really that bad. first, for StopIteration, the exception
object is decref()ed immediately after being raised. moreover,
since no one is expected to see that traceback, it can be set
directly to None, i.e., a PyErr_SetStopIter() function which sets
exc.traceback to None internally.

other than that, you *want* to keep the traceback alive. therefore,
i suggest adding two new attributes to the exception class:
* traceback - the traceback object, or None
* nested - nested ("inner") exception, or None

Exception.__repr__ will be extended to formatting the
traceback (if not None) and formatting all the nested
exceptions (+ detect recursion).

this may be an expensive operation, but as it only happens
when the exceptions is printed to stderr, it's negligible.


-tomer

From brett at python.org  Tue Jan  2 01:14:45 2007
From: brett at python.org (Brett Cannon)
Date: Mon, 1 Jan 2007 16:14:45 -0800
Subject: [Python-3000] PEP 3108: Standard Library Reorganization
Message-ID: <bbaeab100701011614l27a87f45pb7e782c1d78a0725@mail.gmail.com>

As part of my New Years resolution to get all of my current and planned PEPs
actually written, accepted, and implemented for 2007, here is the stdlib
reorg PEP.  I have already checked it into svn but I have inlined it below
for comments.

The PEP is separated into two parts: removal and renaming.  For removal I
tried to keep the list to what I thought was very reasonable.  If you look
in Open Issues there are some possible removals that I am not so sure on so
I would like feedback on those.  I also have not touched the various Mac
modules since I never use them and a bunch of them are undocumented.  If Bob
or Donald (or anyone else for that matter) could help me identify Mac
modules that could go I would really appreciate it.

For the renaming I listed all of the modules that either violate PEP 8, have
both a Python and C implementation, or do not have a public API and thus
should be hidden.  The only possibly controversial renaming is of _winreg,
dummy_threading, and repr.  _winreg should be renamed since the higher-level
version never materialized.  I personally want dummy_threading renamed
mockthreading since dummythreading just looks bad to me, it really is more
of a mock implementation, and I wrote the module to begin with.  =)  and
repr should be renamed reprlib (if it is not flagged for removal of its
public interface) so it doesn't shadow a built-in.

The Open Issues section also has a bunch of suggested groupings of modules
that seemed extremely obvious to me or that other people pointed out.  I
know Guido didn't OK this but they were just staring me in the face so I
just went ahead and listed what I thought could be reasonable packages to
create or modules that really don't need to be separate.  Since we are doing
renames anyway I figured it wouldn't hurt to at least consider these.  They
are also not extensive in terms of putting the entire stdlib into a handful
of packages, just stuff that is obviously the same theme (e.g., collections)
or are tightly coupled (e.g., urllib/urllib2).  I leave it to Guido and the
rest of you guys to either tell me to ignore these possibilities or to
actually move forward with some of them.

Anyway, have at it!

-Brett


-----------------------------------------

Abstract
========

Just like the language itself, Python's standard library (stdlib) has
grown over the years to be very rich.  But over time some modules
have lost their need to be included with Python.  There has also been
an introduction of a naming convention for modules since Python's
inception that not all modules follow.

With Python 3.0 a chance to remove modules that do not have long term
usefulness has presented itself.  This chance also allows for the
renaming of modules so that they follow the Python style guide
[#pep-0008]_.  This PEP lists modules that should not be included in
Python 3.0 and what modules need to be renamed.


Modules to Remove
=================

Guido pronounced that "silly old stuff" is to be deleted from the
stdlib for Py3K [#silly-old-stuff]_.  This is open-ended on purpose.
Each module to be removed needs to have a justification as to why it
should no longer be distributed with Python.  This can range from the
module being deprecated in Python 2.x to being for a platform that is
no longer widely used.

This section of the PEP lists the various modules to be removed. Each
subsection represents a different reason for modules to be
removed.  Each module must have a specific justification on top of
being listed in a specific subsection so as to make sure only modules
that truly deserve to be removed are in fact removed.

When a reason mentions how long it has been since a module has been
"uniquely edited", it is in reference to how long it has been since a
checkin was done specifically for the module and not for a change that
applied universally across the entire stdlib.  If an edit time is not
denoted as "unique" then it is the last time the file was edited,
period.


Previously deprecated
---------------------

Modules in this section have been deprecated at some point in the
Python 2.x release series but are currently still distributed with
Python.  Deprecation information is gathered either from PEP 4 or the
Global Module Index [#pep-0004]_, [#module-index]_.  Each module is
listed with the Python version that the deprecation started in.

* buildtools
    2.3
* cfmfile
    2.4
* gopherlib
    2.5
* macfs
    2.3
* md5
    2.5
* mimetools
    2.3
* MimeWriter
    2.3
* mimefy
    2.3
* multifile
    2.3
* posixfile
    1.5
* rfc822
    2.3
* rgbimg
    2.5
* sha
    2.5


Platform-specific with minimal use
----------------------------------

Python supports many platforms, some of which are not widely held.
And on some of these platforms there are modules that have limited use
to people on those platforms.  Because of their limited usefulness it
would be better to no longer burden the Python development team with
their maintenance.

* IRIX (which is no longer produced [#irix-retirement]_)
    + AL/al
        - Provides sound support on Indy and Indigo workstations.
            * Both workstations are no longer available.
        - Code has not been uniquely edited in three years.
    + cd
        - CD drive control for SGI systems.
            * SGI no longer sells machines with IRIX on them.
        - Code has not been uniquely edited in 14 years.
    + DEVICE/GL/gl/cgen/cgensuport
        - OpenGL access.
        - Has not been edited in at least eight years.
        - Third-party libraries provide better support.
            * PyOpenGL [#pyopengl]_
    + FL/fl/flp
        - Wrapper for the FORMS library [#irix-forms]_
            * FORMS has not been edited in 12 years.
        - Library is not widely used.
            * First eight hits on Google are for Python docs for fl.
    + fm
        - Wrapper to the IRIS Font Manager library.
            * Only available on SGI machines which no longer come with
              IRIX.
    + imgfile
        - Wrapper for SGI libimage library for imglib image files
          (``.rgb`` files).
        - Python Imaging Library provdes read-only support [#pil]_.
        - Not uniquely edited in 13 years.
    + jpeg
        - Wrapper for JPEG (de)compressor.
        - Code not uniquely edited in nine years.
        - Third-party libraries provide better support.
            * Python Imaging Library [#pil]_
    + sv
        - Wrapper for Indigo video card.
            * Harware is no longer manufactured.
        - Undocumented.
        - Code not uniquely edited in 13 years.
* Solaris
    + SUNAUDIODEV/sunaudiodev
        - Access to the sound card on Sun machines.
        - Code not uniquely edited in over eight years.
* Mac
    + applesingle
        - Undocumented.
            * AppleSingle is a binary file format for A/UX.
                + A/UX no longer distributed.
* UNIX
    + nis
        - Wrapper for NIS.
            * NIS has been replaced by LDAP, DNS, and Kerberos.


Minimal usage
-------------

Some modules that are platform-independent have minimal usage.  This
can be from how easy it is to implement the functionality from scratch
or because the audience for the code is small.

* audiodev
    + Undocumented.
    + Not edited in five years.
    + If removed sunaudio should go as well.
        - Undocumented.
        - Not edited in over seven years.
* fileinput
    + Basic functionality handled by ``itertools.chain``.
    + Using ``enumerate`` (for the line number in the file),
      ``itertools.repeat`` (for returning the filename with each
      line), and ``zip`` (for connecting the ``enumerate`` object and
      ``itertools.repeat`` object)  provides 95% of other unique
      abilities of fileinput.
* imputil
    + Undocumented.
    + Never updated to support absolute imports.
* mutex
    + Easy to implement using a semaphore and a queue.
    + Cannot block on a lock attempt.
    + Not uniquely edited since its addition 15 years ago.
* repr
    + Controls output of the repr of objects.
        - String slicing and string interpolation can do similar work.
    + Used by pdb, but do not need to expose API.
* symtable/_symtable
    + Undocumented.
* telnetlib
    + Telnet is not used very much anymore.
        - Telnet is unsafe.
        - Most people use SSH instead.
* toaiff
    + Undocumented.
    + Requires ``sox`` library to be installed on the system.
* user
    + Easily handled by allowing the application specify its own
      module name, check for existence, and import if found.
* new
    + Just a rebinding of names from the 'types' module.
    + Can also call ``type`` built-in to get most types easily.
* pure
    + Written before Pure Atria was bought by Rational which was then
      bought by IBM (in other words, very old).

Obsolete
--------

Becoming obsolete signifies that either another module in the stdlib
or a widely distributed third-party library provides a better solution
for what the module is meant for.

* base64/quopri/uu
    + Support exists in the codecs module.
    + If removed (along with binhex), also remove binascii.
        - C implementation of base64, binhex, and uu modules.
* asynchat/asyncore
    + Third-party libraries provide better solutions.
        - twisted [#twisted]_
    + Deprecation previously supported [#py-dev-summary-2004-11-01]_
* Bastion/rexec
    + Restricted execution / security.
    + Turned off in Python 2.3.
    + Modules deemed unsafe.
* dl
    + ctypes provides better support for same functionality.
* fpformat
    + All functionality is supported by string interpolation.
* getopt
    + optparse provides better functionality.
* ihooks
    + Documented except for saying that module might be obsolete.
    + For use with rexec which has been turned off since Python 2.3.
* imageop
    + Better support by third-party libraries.
        - Python Imaging Library [#pil]_.
* linuxaudiodev
    + Replaced by ossaudiodev.
* stat
    + ``os.stat`` now returns a tuple with attributes.
* statvfs
    + ``os.statvfs`` now returns a tuple with attributes.
* strop
    + Implements functions used by 'string' module that have now
      become methods on the str type.
* thread
    + People should use 'threading' instead.
        - Rename 'thread' to _thread.
        - Deprecate dummy_thread.
            * Rename to _mockthread.
            * Change of name better reflects modules purpose.
    + Guido has previously supported the deprecation
      [#thread-deprecation]_.
* timing
    + Use timeit or time.
    + Documentation says the module is obsolete [#timing-module]_.


Modules to Rename
=================

Along with the stdlib gaining some modules that are no longer
relevant, there is also the issue of naming.  Many modules existed in
the stdlib before PEP 8 came into existence [#pep-0008]_.  This has
led to some naming inconsistencies that should be addressed.

Any module that has been suggested for removal and does not meet the
required naming scheme is *not* listed below.


PEP 8 violations
----------------

PEP 8 specifies that modules "should have short, lowercase names,
without underscores" [#pep-0008]_.  There is no mention, though, if
this rule extends to modules contained within a package.  The
assumption is that underscores are acceptable in module names when
they are contained within a package but that any uppercase letters is
not.

* _winreg
    winreg (rename also because module has a public interface).
* autoGIL
    autogil
* BaseHTTPServer
    basehttpserver
* Carbon
    carbon
* CGIHTTPServer
    cgihttpserver
* ColorPicker
    colorpicker
* ConfigParser
    configparser
* Cookie
    cookie
* copy_reg
    copyreg
* cProfile
* DocXMLRPCServer
    docxmlrpcserver
* dummy_threading
    mockthreading (rename because "mock" makes more sense than
    "dummy" and rename already required).
* EasyDialogs
    easydialogs
* FrameWork
    framework
* HTMLParser
    htmlparser
* MacOS
    macos
* MiniAEFrame
    miniaeframe
* Nav
    nav
* PixMapWrapper
    pixmapwrapper
* py_compile
    pycompile
* Queue
    queue
* repr
    reprlib (rename because module name shadows a built-in).
* ScrolledText
    scrolledtext
* SimpleHTTPServer
    simplehttpserver
* SimpleXMLRPCServer
    simplexmlrpcserver
* SocketServer
    socketserver
* StringIO
    stringio
* Tix
    tix
* Tkinter
    tkinter
* UserDict
    userdict
* UserList
    userlist
* UserString
    userstring
* W
    w


Merging C and Python implementations of the same interface
----------------------------------------------------------

Several interfaces have both a Python and C implementation.  While it
is great to have a C implementation for speed with a Python
implementation as fallback, there is no need to expose the two
implementations independently in the stdlib.  For Python 3.0 all
interfaces with two implementations will be merged into a single
public interface.

The C module is to be given a leading underscore to delineate the fact
that it is not the reference implementation (the Python implementation
is).  This means that any semantic difference between the C and Python
versions must be dealt with before Python 3.0 or else the C
implementation will be removed until it can be fixed.

One interface that is not listed below is xml.etree.ElementTree.  This
is an externally maintained module and thus is not under the direct
control of the Python development team for renaming.  See `Open
Issues`_ for a discussion on this.

* pickle/cPickle
    + Rename cPickle to _pickle.
    + Semantic completeness of C implementation *not* verified.
* profile/cProfile
    + Rename cProfile to profile.
    + Semantic completeness of C implementation *not* verified.
* StringIO/cStringIO
    + Rename StringIO to stringio.
    + Rename cStringIO to _stringio.
    + Semantic completeness of C implementation *not* verified.


No public, documented interface
-------------------------------

There are several modules in the stdlib that have no defined public
interface.  These modules exist as support code for other modules that
are exposed.  Because they are not meant to be used directly they
should be renamed to reflect this fact.

* bdb
    _bdb
* markupbase
    _markupbase
* opcode
    _opcode
* dummythread
    _mockthread (assuming the deprecation of 'thread' occurs).


Transition Plan
===============

For modules to be removed
-------------------------

A PendingDeprecationWarning will be set in Python 2.6 for all modules
slated to be removed in Python 3.0.  This will allow people to know
which modules will not exist but without being overly noisy since
PendingDeprecationWarning is by default silenced.


For modules to be renamed
-------------------------

Modules will be renamed in Python 2.6 .  The original names of the
modules will still work but will raise ImportWarning upon import.  The
refactoring tool for transitioning to Python 3.0 will refactor imports
that use the original names to the new names.


Open Issues
===========

Consolidate dependent modules together into a single module or package?
-----------------------------------------------------------------------

The stdlib has several modules that have a level of dependency between
them (e.g., urllib and urllib2).  Usually one is a low-level module
that provides basic abilities with a corresponding higher-level API is
given in another module for simple use-cases.  In Python 3.0 we could
group these dependent modules together into a single module or package
to better reflect their relationship.

Keep in mind when looking at the groupings that deprecation or removal
is also a possibility if there is enough overlap or a module is
obsolete.

* Cookie/cookielib
* urllib/urllib2
    + urlparse?
    + httplib?
* cgi/cgitb
* Tix/Tkinter
* getpass/pwd/spwd/grp
* mailbox/mhlib
* anydbm/whichdbm
* bsddb/dbhash
* pickle/pickletools
* HTMLParser/htmllib
* ftplib/netrc
* parser/symbol


Consolidate certain modules with similar themes together in a package?
----------------------------------------------------------------------

Packages are often used to group together modules that have a similar
theme but do not have any direct relationship or dependency upon each
other.  For Python 3.0 obvious groupings could be done since renaming
of various modules is already occurring.

* collections
    + heapq
    + Queue
    + sets
    + UserDist
    + UserList
    + What to do with UserString?
        - Have a package for Python implementations of built-in types
          instead of putting the User* modules into 'collections'?
* mac
    + Various Mac-specific modules.
    + Same can be done for other platform-specific code.
* Profiling
    + cProfile
    + profile
    + hotshot
    + pstats
* email
    + mailbox
    + mhlib
* Databases
    + anydbm
    + dbhash
    + dbm
    + bsddb
    + dumbdbm
    + gdbm
    + whichdb
* Audio
    + aifc
    + audioop
    + chunk
    + ossaudiodev
    + sunau
    + wave
    + winsound
* Servers
    + BaseHTTPServer
    + CGIHTTPServer
    + DocXMLRPCServer
    + SimpleHTTPServer
    + SimpleXMLRPCServer
    + SocketServer


Modules reliant on obsolete/rarely used file formats?
-----------------------------------------------------

Several modules in the stdlib work on a specific file format.  It is
possible some of these formats are no longer used and thus the stdlib
modules for them can go.  Below is a list of some modules which rely
on a file format that may be obsolete.

* aifc
    AIFF and AIFF-C audio files.  Appears to be only user of the cl
    module (which is undocumented).
* audioop
    Raw (8|16|32)-bit wide audio files (as generated by the al and
    sunaudiodev modules).
* binhex
    binhex4 encoding.
* chunk
    AIFF, AIFF-C, and RMFF audio files.
* sunau
    Sun AU audio files [#sun-au]_.


Renaming of modules maintained outside of the stdlib
----------------------------------------------------

xml.etree.ElementTree not only does not meet PEP 8 naming standards
but it also has an exposed C implementation [#pep-0008]_.  It is an
externally maintained package, though [#pep-0360]_.  A request will be
made for the maintainer to change the name so that it matches PEP 8
and hides the C implementation.


References
==========

.. [#pep-0004] PEP 4: Deprecation of Standard Modules
    (http://www.python.org/dev/pep-0004/)

.. [#pep-0008] PEP 8: Style Guide for Python Code
    (http://www.python.org/dev/pep-0008/)

.. [#pep-0360] PEP 360: Externally Maintained Packages
    (http://www.python.org/peps/pep-0360/)

.. [#module-index] Python Documentation: Global Module Index
    (http://docs.python.org/modindex.html)

.. [#timing-module] Python Library Reference: Obsolete
    (http://docs.python.org/lib/obsolete-modules.html)

.. [#silly-old-stuff] Python-Dev email: "Py3k release schedule worries"
    (http://mail.python.org/pipermail/python-3000/2006-December/005130.html)

.. [#thread-deprecation] Python-Dev email: Autoloading?
    (http://mail.python.org/pipermail/python-dev/2005-October/057244.html)

.. [#py-dev-summary-2004-11-01] Python-Dev Summary: 2004-11-01
    (http://www.python.org/dev/summary/2004-11-01_2004-11-15/#id10)

.. [#pyopengl] PyOpenGL
    (http://pyopengl.sourceforge.net/)

.. [#pil] Python Imaging Library (PIL)
    (http://www.pythonware.com/products/pil/)

.. [#twisted] Twisted
    (http://twistedmatrix.com/trac/)

.. [#irix-retirement] SGI Press Release:
    End of General Availability for MIPS IRIX Products -- December 2006
    (http://www.sgi.com/support/mips_irix.html)

.. [#irix-forms] FORMS Library by Mark Overmars
    (ftp://ftp.cs.ruu.nl/pub/SGI/FORMS)

.. [#sun-au] Wikipedia: Au file format
    (http://en.wikipedia.org/wiki/Au_file_format)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20070101/f38b0f15/attachment-0001.htm 

From rasky at develer.com  Tue Jan  2 01:23:32 2007
From: rasky at develer.com (Giovanni Bajo)
Date: Tue, 02 Jan 2007 01:23:32 +0100
Subject: [Python-3000] self-contained exceptions
In-Reply-To: <1d85506f0701011438q6d8b65bfi4ef1e8f9fa509650@mail.gmail.com>
References: <1d85506f0701011438q6d8b65bfi4ef1e8f9fa509650@mail.gmail.com>
Message-ID: <enc8lv$8uu$1@sea.gmane.org>

tomer filiba wrote:

> this makes sys.excepthook unnecessary

Maybe I'm stating the obvious here, but I override sys.excepthook in 
real-world code. If you want it go away for some reason, I guess you should 
propose a replacement with similar semantic.
-- 
Giovanni Bajo


From guido at python.org  Tue Jan  2 01:30:22 2007
From: guido at python.org (Guido van Rossum)
Date: Mon, 1 Jan 2007 16:30:22 -0800
Subject: [Python-3000] self-contained exceptions
In-Reply-To: <1d85506f0701011438q6d8b65bfi4ef1e8f9fa509650@mail.gmail.com>
References: <1d85506f0701011438q6d8b65bfi4ef1e8f9fa509650@mail.gmail.com>
Message-ID: <ca471dc20701011630l633130e0n356eb9b176f1d610@mail.gmail.com>

On 1/1/07, tomer filiba <tomerfiliba at gmail.com> wrote:
> a while back some people suggested having the traceback
> as an attribute of the exception object, which allows for
> nested exceptions.

PEP 344, to be precise.

> i would also like to note that having the traceback part of
> the exception object allows the exception to print itself,
> i.e., its repr would format the traceback text on it's own.

I'm not sure that's an improvement. I like to b able to have different
ways of formatting exceptions. And a repr() that's as long as a
typical traceback isn't a great idea either. Note that currently
there's nothing to prevent tracebacks from having a repr() that prints
the exception, and yet this isn't what it does.

> this makes sys.excepthook unnecessary,

Not at all. sys.excepthook can do other things besides formatting the
exception, and when it does format the exception, it can do so in any
number of interesting alternative ways (e.g. cgitb.py).

> as well as
> traceback.format_exception (which i greatly dislike).

Why? Because it once insulted you?

> even if the exception's repr would raise an exception
> (recursion limit, etc.), that exception would be a builtin
> one, so we won't end up in a repr-exception-loop.
>
> with this approach, sys.excepthook is needless, or at least
> simplified greatly (just print repr(exc)). for example, exceptions
> raised by RPyC have a special attribute to them -- remote_traceback,
> which is the text of the remote traceback. currently i have to
> replace excepthook to have it displayed with the local exception
> traceback, which is means just importing RPyC messes with
> your interpreter's internals, and may cause problems when
> another library installs its own hook.
>
> apart from self-repr'ing, nested exceptions are very useful too:
>
> try:
>     foo()
> except Exception, ex:
>     raise MyException("something happened", nested = ex)

It really looks like you've never read PEP 344, or you wouldn't be
introducing a different way to spell what it already proposes.

> so that when i pdb.pm() my code, i can inspect the inner
> exception too (which is desirable of course).
>
> the major argument against having the traceback part of
> the exception object was increasing memory consumption,
> so that exceptions are no longer "light weight" (as used by
> StopIteration, etc.)

Where did you hear this? It is not *my* objection.

> i don't see why this argument stands. the traceback object
> exists either way, the only question is in what namespace
> it exists, and the fact it may "live longer" than necessary,
> i.e., the traceback exists for as long as the exception object
> exists.
>
> i don't think it's really that bad. first, for StopIteration, the exception
> object is decref()ed immediately after being raised. moreover,
> since no one is expected to see that traceback, it can be set
> directly to None, i.e., a PyErr_SetStopIter() function which sets
> exc.traceback to None internally.
>
> other than that, you *want* to keep the traceback alive. therefore,
> i suggest adding two new attributes to the exception class:
> * traceback - the traceback object, or None
> * nested - nested ("inner") exception, or None
>
> Exception.__repr__ will be extended to formatting the
> traceback (if not None) and formatting all the nested
> exceptions (+ detect recursion).
>
> this may be an expensive operation, but as it only happens
> when the exceptions is printed to stderr, it's negligible.

I believe the main argument *against* having the tb included in the
exception is that it would greatly increase the cyclic nature of
tracebacks and stack frames, which in turn would cause lots of code to
break due to late GC'ing of unclosed files, etc. For example, this
code:

try:
  1/0
except Exception, err:
  pass

will introduce a cycle from err -> traceback -> stack frame -> err,
keeping all locals in the same scope alive until the next GC happens.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From rasky at develer.com  Tue Jan  2 01:52:00 2007
From: rasky at develer.com (Giovanni Bajo)
Date: Tue, 02 Jan 2007 01:52:00 +0100
Subject: [Python-3000] PEP 3108: Standard Library Reorganization
In-Reply-To: <bbaeab100701011614l27a87f45pb7e782c1d78a0725@mail.gmail.com>
References: <bbaeab100701011614l27a87f45pb7e782c1d78a0725@mail.gmail.com>
Message-ID: <encabb$e5s$1@sea.gmane.org>

Brett Cannon wrote:

> * fileinput
>     + Basic functionality handled by ``itertools.chain``.
>     + Using ``enumerate`` (for the line number in the file),
>       ``itertools.repeat`` (for returning the filename with each
>       line), and ``zip`` (for connecting the ``enumerate`` object and
>       ``itertools.repeat`` object)  provides 95% of other unique
>       abilities of fileinput.

How would you rewrite this without fileinput:

====== cat.py ======
import sys
import fileinput

for L in fileinput.input():
     sys.stdout.write(L)
====================

The very concise call to input() lets you handle many different issues at once 
and implement a natural interface for a text-mungling program (accepts one or 
more file names, default to stdin, if "-" is specified it means stdin, and so on).

I think this module should stay. Its most important feature is that it lets 
you implement a de-facto standard interface with a single call.


> The C module is to be given a leading underscore to delineate the fact
> that it is not the reference implementation (the Python implementation
> is).  This means that any semantic difference between the C and Python
> versions must be dealt with before Python 3.0 or else the C
> implementation will be removed until it can be fixed.

Why should always the Python implementation be the reference one? Of the three 
examples you did, in two cases (cPickle and cStringIO) the C implementation is 
much faster. In fact, I would suggest to remove the Python implementation if 
any...

> * profile/cProfile
>     + Rename cProfile to profile.
>     + Semantic completeness of C implementation *not* verified.

Is this an editor mistake? Where does the current "profile" go? Are you 
suggesting to remove it?

> Modules will be renamed in Python 2.6 .  The original names of the
> modules will still work but will raise ImportWarning upon import.  The
> refactoring tool for transitioning to Python 3.0 will refactor imports
> that use the original names to the new names.

Does that mean that a very large amount of existing Python code will raise an 
ImportWarning under Python 2.6? I don't think this is acceptable. Why should 
the users be bothered with deprectation warnings related to Python 3.0? Surely 
the *developers* are the only ones interested in this.

A PendingDeprecationWarning is fine in this case too I guess. Or you could 
introduce a specific Py3kDeprecationWarning, turned on with a mystical -3 
command line option (in which we can join all the features to help porting 
source code to Py3k).


>     + UserDist
>     + UserList
>     + What to do with UserString?

BTW, aren't the User* modules a relic of when builtin types weren't 
subclassable? Are they still useful now? Really a genuine question...
-- 
Giovanni Bajo


From jcarlson at uci.edu  Tue Jan  2 03:13:20 2007
From: jcarlson at uci.edu (Josiah Carlson)
Date: Mon, 01 Jan 2007 18:13:20 -0800
Subject: [Python-3000] PEP 3108: Standard Library Reorganization
In-Reply-To: <bbaeab100701011614l27a87f45pb7e782c1d78a0725@mail.gmail.com>
References: <bbaeab100701011614l27a87f45pb7e782c1d78a0725@mail.gmail.com>
Message-ID: <20070101174925.82FB.JCARLSON@uci.edu>


"Brett Cannon" <brett at python.org> wrote:
> * fileinput
>     + Basic functionality handled by ``itertools.chain``.
>     + Using ``enumerate`` (for the line number in the file),
>       ``itertools.repeat`` (for returning the filename with each
>       line), and ``zip`` (for connecting the ``enumerate`` object and
>       ``itertools.repeat`` object)  provides 95% of other unique
>       abilities of fileinput.

I agree with Giovanni on the above.

> * telnetlib
>     + Telnet is not used very much anymore.
>         - Telnet is unsafe.
>         - Most people use SSH instead.

I don't know how common telnet lib use is currently, but I have found it
useful in writing plaintext line-based clients (like SMTP, POP, HTTP,
IMAP, etc.), before moving on to writing async* derived clients.

> * base64/quopri/uu
>     + Support exists in the codecs module.
>     + If removed (along with binhex), also remove binascii.
>         - C implementation of base64, binhex, and uu modules.

The binascii module has become a catch all module for various plaintext
<-> binary conversions.  I'm not sure if using .encode()/.decode() are
necessarily desireable, especially in the case of certain binascii
conversions that involve long/int <-> binary (that didn't make it into
Python 2.5).

> * asynchat/asyncore
>     + Third-party libraries provide better solutions.
>         - twisted [#twisted]_
>     + Deprecation previously supported [#py-dev-summary-2004-11-01]_

There is quite a bit of user code that depends on asynchat/asyncore
modules.  While there are many people who end up using twisted, forcing
all asynchronous socket users to "drink the twisted kool-aid" would set
a bad precident for any commonly used stdlib Python library with a more
fully-featured 3rd party module.


> * stat
>     + ``os.stat`` now returns a tuple with attributes.

The stat module also offers useful functions like stat.IS_DIR() for
determining if an object is a directory.

> * thread
>     + People should use 'threading' instead.
>         - Rename 'thread' to _thread.
>         - Deprecate dummy_thread.
>             * Rename to _mockthread.
>             * Change of name better reflects modules purpose.
>     + Guido has previously supported the deprecation
>       [#thread-deprecation]_.

Sounds good.  Can we get this particular change into the 2.x series? 
I've seen too much code that mixes and matches threading and thread use.


> * SocketServer
>     socketserver

SocketServer and its derivatives (*HTTPServer) are tricky (I've been
seeing them used quite a bit)...

> * Servers
>     + BaseHTTPServer
>     + CGIHTTPServer
>     + DocXMLRPCServer
>     + SimpleHTTPServer
>     + SimpleXMLRPCServer
>     + SocketServer

Would it be a reasonable semantic to say that 'no top level modules or
packages can violate PEP 8 module/package naming guidelines', but
sub-modules or packages may use CamelCase by historical precident or
when doing so would make understanding the purpose of the module/package
easier?


 - Josiah


From mike.klaas at gmail.com  Tue Jan  2 04:03:33 2007
From: mike.klaas at gmail.com (Mike Klaas)
Date: Mon, 1 Jan 2007 19:03:33 -0800
Subject: [Python-3000] PEP 3108: Standard Library Reorganization
In-Reply-To: <bbaeab100701011614l27a87f45pb7e782c1d78a0725@mail.gmail.com>
References: <bbaeab100701011614l27a87f45pb7e782c1d78a0725@mail.gmail.com>
Message-ID: <3d2ce8cb0701011903h694d7a5g58e008f79f48e00b@mail.gmail.com>

On 1/1/07, Brett Cannon <brett at python.org> wrote:
> * new
>     + Just a rebinding of names from the 'types' module.
>     + Can also call ``type`` built-in to get most types easily.

Perhaps 'types' should be deprecated instead?

obj.x = new.instancemethod(...

has a very pleasing ring to it (perhaps too pleasing--"new" feels like
a language construct, not a module name).

Much nicer than

obj.x = types.MethodType(...

-Mike

From brett at python.org  Tue Jan  2 04:37:01 2007
From: brett at python.org (Brett Cannon)
Date: Mon, 1 Jan 2007 19:37:01 -0800
Subject: [Python-3000] PEP 3108: Standard Library Reorganization
In-Reply-To: <encabb$e5s$1@sea.gmane.org>
References: <bbaeab100701011614l27a87f45pb7e782c1d78a0725@mail.gmail.com>
	<encabb$e5s$1@sea.gmane.org>
Message-ID: <bbaeab100701011937r1cb30f15h435c4a2f37d86e23@mail.gmail.com>

On 1/1/07, Giovanni Bajo <rasky at develer.com> wrote:
>
> Brett Cannon wrote:
>
> > * fileinput
> >     + Basic functionality handled by ``itertools.chain``.
> >     + Using ``enumerate`` (for the line number in the file),
> >       ``itertools.repeat`` (for returning the filename with each
> >       line), and ``zip`` (for connecting the ``enumerate`` object and
> >       ``itertools.repeat`` object)  provides 95% of other unique
> >       abilities of fileinput.
>
> How would you rewrite this without fileinput:
>
> ====== cat.py ======
> import sys
> import fileinput
>
> for L in fileinput.input():
>      sys.stdout.write(L)
> ====================
>
> The very concise call to input() lets you handle many different issues at
> once
> and implement a natural interface for a text-mungling program (accepts one
> or
> more file names, default to stdin, if "-" is specified it means stdin, and
> so on).
>
> I think this module should stay. Its most important feature is that it
> lets
> you implement a de-facto standard interface with a single call.


Fair enough.  If more people say it should stay it can stay.

> The C module is to be given a leading underscore to delineate the fact
> > that it is not the reference implementation (the Python implementation
> > is).  This means that any semantic difference between the C and Python
> > versions must be dealt with before Python 3.0 or else the C
> > implementation will be removed until it can be fixed.
>
> Why should always the Python implementation be the reference one? Of the
> three
> examples you did, in two cases (cPickle and cStringIO) the C
> implementation is
> much faster. In fact, I would suggest to remove the Python implementation
> if
> any...


The Python version is the reference implementation because all
implementations of Python can use it.  The C implementation is just for
speed.  Besides the Python version almost always comes first and is what
people use as a design reference.  No one is saying the C version goes away,
just that the Python version stays as a backup and is used to define the
semantics.

And Guido has already decided this whole thing anyway.

> * profile/cProfile
> >     + Rename cProfile to profile.
> >     + Semantic completeness of C implementation *not* verified.
>
> Is this an editor mistake? Where does the current "profile" go? Are you
> suggesting to remove it?


It's a typo.

> Modules will be renamed in Python 2.6 .  The original names of the
> > modules will still work but will raise ImportWarning upon import.  The
> > refactoring tool for transitioning to Python 3.0 will refactor imports
> > that use the original names to the new names.
>
> Does that mean that a very large amount of existing Python code will raise
> an
> ImportWarning under Python 2.6? I don't think this is acceptable. Why
> should
> the users be bothered with deprectation warnings related to Python 3.0?
> Surely
> the *developers* are the only ones interested in this.


Yeah, but DeprecationWarnngs are loud as well.  It is up to the developer to
change the code still.

A PendingDeprecationWarning is fine in this case too I guess. Or you could
> introduce a specific Py3kDeprecationWarning, turned on with a mystical -3
> command line option (in which we can join all the features to help porting
> source code to Py3k).


PendingDeprecationWarning could work.  I will wait and see what other people
have to say.

>     + UserDist
> >     + UserList
> >     + What to do with UserString?
>
> BTW, aren't the User* modules a relic of when builtin types weren't
> subclassable? Are they still useful now? Really a genuine question...


I think some people use them for mixin purposes.  But I don't know since I
never use them.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20070101/a29a8f8b/attachment-0001.htm 

From brett at python.org  Tue Jan  2 04:44:08 2007
From: brett at python.org (Brett Cannon)
Date: Mon, 1 Jan 2007 19:44:08 -0800
Subject: [Python-3000] PEP 3108: Standard Library Reorganization
In-Reply-To: <20070101174925.82FB.JCARLSON@uci.edu>
References: <bbaeab100701011614l27a87f45pb7e782c1d78a0725@mail.gmail.com>
	<20070101174925.82FB.JCARLSON@uci.edu>
Message-ID: <bbaeab100701011944n7b73a33aq68b3ac0f50b7b8f3@mail.gmail.com>

On 1/1/07, Josiah Carlson <jcarlson at uci.edu> wrote:
>
>
> "Brett Cannon" <brett at python.org> wrote:
> > * fileinput
> >     + Basic functionality handled by ``itertools.chain``.
> >     + Using ``enumerate`` (for the line number in the file),
> >       ``itertools.repeat`` (for returning the filename with each
> >       line), and ``zip`` (for connecting the ``enumerate`` object and
> >       ``itertools.repeat`` object)  provides 95% of other unique
> >       abilities of fileinput.
>
> I agree with Giovanni on the above.


OK.  Starting to sound like fileinput should stay.

> * telnetlib
> >     + Telnet is not used very much anymore.
> >         - Telnet is unsafe.
> >         - Most people use SSH instead.
>
> I don't know how common telnet lib use is currently, but I have found it
> useful in writing plaintext line-based clients (like SMTP, POP, HTTP,
> IMAP, etc.), before moving on to writing async* derived clients.


Fair enough.  If more people step up it can stay.

> * base64/quopri/uu
> >     + Support exists in the codecs module.
> >     + If removed (along with binhex), also remove binascii.
> >         - C implementation of base64, binhex, and uu modules.
>
> The binascii module has become a catch all module for various plaintext
> <-> binary conversions.


Right, but is it necessarily the best thing that it has become a catch-all?

  I'm not sure if using .encode()/.decode() are
> necessarily desireable, especially in the case of certain binascii
> conversions that involve long/int <-> binary (that didn't make it into
> Python 2.5).


 Anyone else want to weigh in to save binascii?

> * asynchat/asyncore
> >     + Third-party libraries provide better solutions.
> >         - twisted [#twisted]_
> >     + Deprecation previously supported [#py-dev-summary-2004-11-01]_
>
> There is quite a bit of user code that depends on asynchat/asyncore
> modules.  While there are many people who end up using twisted, forcing
> all asynchronous socket users to "drink the twisted kool-aid" would set
> a bad precident for any commonly used stdlib Python library with a more
> fully-featured 3rd party module.


Well, I just remember the thread that led to the suggestion the two modules
go.  It can be saved from the chopping block but I remember part of the
issue was no one was wanting to maintain the modules anymore and they were
starting to collect dust.

> * stat
> >     + ``os.stat`` now returns a tuple with attributes.
>
> The stat module also offers useful functions like stat.IS_DIR() for
> determining if an object is a directory.


Perhaps the functions should move to os or be part of the object returned by
os.stat.

> * thread
> >     + People should use 'threading' instead.
> >         - Rename 'thread' to _thread.
> >         - Deprecate dummy_thread.
> >             * Rename to _mockthread.
> >             * Change of name better reflects modules purpose.
> >     + Guido has previously supported the deprecation
> >       [#thread-deprecation]_.
>
> Sounds good.  Can we get this particular change into the 2.x series?
> I've seen too much code that mixes and matches threading and thread use.


Probably if people want it.

> * SocketServer
> >     socketserver
>
> SocketServer and its derivatives (*HTTPServer) are tricky (I've been
> seeing them used quite a bit)...


Well, I am not proposing to remove them although I would not cry if the
HTTPServer children disappeared, but that's just me.  We will have to see if
anyone else is up for letting them go.

> * Servers
> >     + BaseHTTPServer
> >     + CGIHTTPServer
> >     + DocXMLRPCServer
> >     + SimpleHTTPServer
> >     + SimpleXMLRPCServer
> >     + SocketServer
>
> Would it be a reasonable semantic to say that 'no top level modules or
> packages can violate PEP 8 module/package naming guidelines', but
> sub-modules or packages may use CamelCase by historical precident or
> when doing so would make understanding the purpose of the module/package
> easier?



Could.  The renaming as of right now only enforces it on the package name
stringently.  But the style guide is not my call.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20070101/58eef67b/attachment.html 

From brett at python.org  Tue Jan  2 04:46:01 2007
From: brett at python.org (Brett Cannon)
Date: Mon, 1 Jan 2007 19:46:01 -0800
Subject: [Python-3000] PEP 3108: Standard Library Reorganization
In-Reply-To: <3d2ce8cb0701011903h694d7a5g58e008f79f48e00b@mail.gmail.com>
References: <bbaeab100701011614l27a87f45pb7e782c1d78a0725@mail.gmail.com>
	<3d2ce8cb0701011903h694d7a5g58e008f79f48e00b@mail.gmail.com>
Message-ID: <bbaeab100701011946h6ab58dfeo70ed58104a7a6c3c@mail.gmail.com>

On 1/1/07, Mike Klaas <mike.klaas at gmail.com> wrote:
>
> On 1/1/07, Brett Cannon <brett at python.org> wrote:
> > * new
> >     + Just a rebinding of names from the 'types' module.
> >     + Can also call ``type`` built-in to get most types easily.
>
> Perhaps 'types' should be deprecated instead?
>
> obj.x = new.instancemethod(...
>
> has a very pleasing ring to it (perhaps too pleasing--"new" feels like
> a language construct, not a module name).
>
> Much nicer than
>
> obj.x = types.MethodType(...


Well the 'types' module does do a little bit more than just give fancy name
to stuff.  And the feeling of 'new' as a construct only occurs if you come
from certain other languages.  We could just ditch both, but I don't know if
people want to go that far.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20070101/c413e8b0/attachment.htm 

From anthony at interlink.com.au  Tue Jan  2 04:46:58 2007
From: anthony at interlink.com.au (Anthony Baxter)
Date: Tue, 2 Jan 2007 14:46:58 +1100
Subject: [Python-3000] PEP 3108: Standard Library Reorganization
In-Reply-To: <encabb$e5s$1@sea.gmane.org>
References: <bbaeab100701011614l27a87f45pb7e782c1d78a0725@mail.gmail.com>
	<encabb$e5s$1@sea.gmane.org>
Message-ID: <200701021447.01263.anthony@interlink.com.au>

On Tuesday 02 January 2007 11:52, Giovanni Bajo wrote:
> Brett Cannon wrote:
> > Modules will be renamed in Python 2.6 .  The original names of
> > the modules will still work but will raise ImportWarning upon
> > import.  The refactoring tool for transitioning to Python 3.0
> > will refactor imports that use the original names to the new
> > names.
>
> Does that mean that a very large amount of existing Python code
> will raise an ImportWarning under Python 2.6? I don't think this
> is acceptable. Why should the users be bothered with deprectation
> warnings related to Python 3.0? Surely the *developers* are the
> only ones interested in this.
>
> A PendingDeprecationWarning is fine in this case too I guess. Or
> you could introduce a specific Py3kDeprecationWarning, turned on
> with a mystical -3 command line option (in which we can join all
> the features to help porting source code to Py3k).

I could not agree more. The list of modules Brett gave covers enough 
that pretty much every single piece of code I have will need to be 
edited. Worse yet, I _can't_ edit it and change it to the new names 
unless I want to abandon pre-2.6 compatibility. This is completely 
unacceptable. Living with a huge spew of warnings is not an 
acceptable outcome - for those of us running production systems in 
Python, a warning should be an indication of something to be fixed, 
not pointless log-filler.

I'd like to suggest that any warnings about 3.0 compat _not_ be 
issued by default, if the fix results in code that fails with 
earlier 2.x releases. 
Even if there's a way to fix a compat issue for pre-2.6, if it's a 
very common construct then the warning should also be suppressed by 
default. A -3 flag (or whatever it ends up being) should be used to 
enable these warnings.

Making 2.6 "the release where everyone's code starts spewing pages 
of warnings" is a bad bad thing. It will convey a terrible 
perception of Python amongst people who use it. Vendors who 
distribute code to other users will also be most unimpressed. You 
and I might know that a warning is just a warning, but many other 
people will not realise that, and will think "ohmygod - 2.6 broke 
my code!"

Anthony
-- 
Anthony Baxter     <anthony at interlink.com.au>
It's never too late to have a happy childhood.

From brett at python.org  Tue Jan  2 05:37:25 2007
From: brett at python.org (Brett Cannon)
Date: Mon, 1 Jan 2007 20:37:25 -0800
Subject: [Python-3000] PEP 3108: Standard Library Reorganization
In-Reply-To: <200701021447.01263.anthony@interlink.com.au>
References: <bbaeab100701011614l27a87f45pb7e782c1d78a0725@mail.gmail.com>
	<encabb$e5s$1@sea.gmane.org>
	<200701021447.01263.anthony@interlink.com.au>
Message-ID: <bbaeab100701012037l8964cefna0ece900ad193ef@mail.gmail.com>

On 1/1/07, Anthony Baxter <anthony at interlink.com.au> wrote:
>
> On Tuesday 02 January 2007 11:52, Giovanni Bajo wrote:
> > Brett Cannon wrote:
> > > Modules will be renamed in Python 2.6 .  The original names of
> > > the modules will still work but will raise ImportWarning upon
> > > import.  The refactoring tool for transitioning to Python 3.0
> > > will refactor imports that use the original names to the new
> > > names.
> >
> > Does that mean that a very large amount of existing Python code
> > will raise an ImportWarning under Python 2.6? I don't think this
> > is acceptable. Why should the users be bothered with deprectation
> > warnings related to Python 3.0? Surely the *developers* are the
> > only ones interested in this.
> >
> > A PendingDeprecationWarning is fine in this case too I guess. Or
> > you could introduce a specific Py3kDeprecationWarning, turned on
> > with a mystical -3 command line option (in which we can join all
> > the features to help porting source code to Py3k).
>
> I could not agree more. The list of modules Brett gave covers enough
> that pretty much every single piece of code I have will need to be
> edited. Worse yet, I _can't_ edit it and change it to the new names
> unless I want to abandon pre-2.6 compatibility. This is completely
> unacceptable. Living with a huge spew of warnings is not an
> acceptable outcome - for those of us running production systems in
> Python, a warning should be an indication of something to be fixed,
> not pointless log-filler.
>
> I'd like to suggest that any warnings about 3.0 compat _not_ be
> issued by default, if the fix results in code that fails with
> earlier 2.x releases.
> Even if there's a way to fix a compat issue for pre-2.6, if it's a
> very common construct then the warning should also be suppressed by
> default. A -3 flag (or whatever it ends up being) should be used to
> enable these warnings.
>
> Making 2.6 "the release where everyone's code starts spewing pages
> of warnings" is a bad bad thing. It will convey a terrible
> perception of Python amongst people who use it. Vendors who
> distribute code to other users will also be most unimpressed. You
> and I might know that a warning is just a warning, but many other
> people will not realise that, and will think "ohmygod - 2.6 broke
> my code!"


OK, the argument works for me.  Would you prefer a Py3K-specific warning
that is like PendingDeprecationWarning and thus normally silenced or just
stick with PendingDeprecationWarning?

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20070101/9928eb57/attachment.htm 

From ncoghlan at gmail.com  Tue Jan  2 05:52:55 2007
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 02 Jan 2007 14:52:55 +1000
Subject: [Python-3000] PEP 3108: Standard Library Reorganization
In-Reply-To: <bbaeab100701011614l27a87f45pb7e782c1d78a0725@mail.gmail.com>
References: <bbaeab100701011614l27a87f45pb7e782c1d78a0725@mail.gmail.com>
Message-ID: <4599E527.6090403@gmail.com>

Brett Cannon wrote:
> * telnetlib
>     + Telnet is not used very much anymore.
>         - Telnet is unsafe.
>         - Most people use SSH instead.

SSH may be common in the PC/server world, but it's still much easier 
(and less demanding on the host's resources) to embed a telnet server 
rather than an SSH one.

Embedded HTTP servers seem to be slowly taking over this role (where 
SNMP hasn't already done so), but telnet-based equipment remote control 
interfaces are certainly still quite common.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From sluggoster at gmail.com  Tue Jan  2 07:13:11 2007
From: sluggoster at gmail.com (Mike Orr)
Date: Mon, 1 Jan 2007 22:13:11 -0800
Subject: [Python-3000] PEP 3108: Standard Library Reorganization
In-Reply-To: <bbaeab100701011614l27a87f45pb7e782c1d78a0725@mail.gmail.com>
References: <bbaeab100701011614l27a87f45pb7e782c1d78a0725@mail.gmail.com>
Message-ID: <6e9196d20701012213v4e65da9xd77d1bc0ea2656cf@mail.gmail.com>

On 1/1/07, Brett Cannon <brett at python.org> wrote:
> * thread
>     + People should use 'threading' instead.

'threading' has no equivalent to 'thread.get_ident()'.  If this
function is ported, I'd support renaming 'thread'.  You need it to
print "Thread #123 doing bla bla bla" messages, to distinguish threads
in a thread pool, to keep track of thread-local data, etc.  The
threading.Thread name isn't the same thing because it's an arbitrary
string, not a guaranteed-unique integer, and the routine creating the
threads may be third-party and not setting unique identifiers.

> * stat
>     + ``os.stat`` now returns a tuple with attributes.

Certain things in 'stat' do not exist elsewhere:  S_ISCHR, S_ISBLK,
S_ISFIFO, S_ISSOCK.  These could be made into more friendly
os.path.is*() functions that take the pathname rather than the
st_mode.  I've never used them, but it would be a pity to make people
calculate the bitmasks manually.

> * fileinput

+1 to leave it.

-- 
Mike Orr <sluggoster at gmail.com>

From barry at python.org  Tue Jan  2 07:26:35 2007
From: barry at python.org (Barry Warsaw)
Date: Tue, 2 Jan 2007 01:26:35 -0500
Subject: [Python-3000] PEP 3108: Standard Library Reorganization
In-Reply-To: <3d2ce8cb0701011903h694d7a5g58e008f79f48e00b@mail.gmail.com>
References: <bbaeab100701011614l27a87f45pb7e782c1d78a0725@mail.gmail.com>
	<3d2ce8cb0701011903h694d7a5g58e008f79f48e00b@mail.gmail.com>
Message-ID: <8B3F3B90-9C0C-4A95-B8B2-3D3DE5DEEF0B@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Jan 1, 2007, at 10:03 PM, Mike Klaas wrote:

> On 1/1/07, Brett Cannon <brett at python.org> wrote:
>> * new
>>     + Just a rebinding of names from the 'types' module.
>>     + Can also call ``type`` built-in to get most types easily.
>
> Perhaps 'types' should be deprecated instead?

Don't forget that the types module contains some types that aren't  
readily available elsewhere.  It shouldn't be deprecated so quickly.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRZn7IHEjvBPtnXfVAQLTtQP+NW3qa7Egurd3+ly5ZQa0Qqe1i2Lnj+uZ
RkZ17KgRg3XuugpWBOX+tY4rfPW+jClHy6sKaZe44bCaIjb163IsjFcCNVwtMU+8
J9VFBKBgZ7j2ZWT0n5bsyE9Z3PujSXCIWevTblUXv+xSgHgPwR0NJDQvFQ8p+cGE
XURJ6PX+iQs=
=2L59
-----END PGP SIGNATURE-----

From sluggoster at gmail.com  Tue Jan  2 07:29:25 2007
From: sluggoster at gmail.com (Mike Orr)
Date: Mon, 1 Jan 2007 22:29:25 -0800
Subject: [Python-3000] Status of True, False, bool type
In-Reply-To: <20070101115005.82F2.JCARLSON@uci.edu>
References: <ca471dc20701010850w1f482455jfd6b3c1c04c86ff0@mail.gmail.com>
	<6e9196d20701011113u7fbbbb43mb24e6635b9edc7c5@mail.gmail.com>
	<20070101115005.82F2.JCARLSON@uci.edu>
Message-ID: <6e9196d20701012229p1fd6b59exd8c283a74c67a7df@mail.gmail.com>

On 1/1/07, Josiah Carlson <jcarlson at uci.edu> wrote:
>
> "Mike Orr" <sluggoster at gmail.com> wrote:
> > Huh?  'True == 1' is a "feature"?  '16 + (0 == 0)' being illegal is a
> > "Javaism"?  Would somebody care to explain this?  It's acceptable that
> > 2 is true but not True?  Why do we need 1 for True at all if we have
> > True?
>
>     >>> 1 is not True
>     True
>     >>> 1 == True
>     True
>     >>> isinstance(True, int)
>     True
>
> If you are suggesting that we change this behavior, stop, it is not
> going to happen. If you are asking why this is the case, I would imagine,
> is based in C/C++/many other sane and insane languages' handling of
> boolean true.

It doesn't matter to me if it stays.  I thought 1 meaning True was
just a concession to backward compatibility because Python didn't have
True and False from the beginning.  So this would be a good time to
ditch it.  Having 1 mean two different things means you can't tell
immediately whether a variable is numeric or boolean; e.g., is it a
line count or just a statement that some lines exist?  Fortunately I
rarely see Python code nowadays that uses 1 this way.

-- 
Mike Orr <sluggoster at gmail.com>

From jcarlson at uci.edu  Tue Jan  2 07:48:26 2007
From: jcarlson at uci.edu (Josiah Carlson)
Date: Mon, 01 Jan 2007 22:48:26 -0800
Subject: [Python-3000] Status of True, False, bool type
In-Reply-To: <6e9196d20701012229p1fd6b59exd8c283a74c67a7df@mail.gmail.com>
References: <20070101115005.82F2.JCARLSON@uci.edu>
	<6e9196d20701012229p1fd6b59exd8c283a74c67a7df@mail.gmail.com>
Message-ID: <20070101224349.830F.JCARLSON@uci.edu>


"Mike Orr" <sluggoster at gmail.com> wrote:
> 
> On 1/1/07, Josiah Carlson <jcarlson at uci.edu> wrote:
> >
> > "Mike Orr" <sluggoster at gmail.com> wrote:
> > > Huh?  'True == 1' is a "feature"?  '16 + (0 == 0)' being illegal is a
> > > "Javaism"?  Would somebody care to explain this?  It's acceptable that
> > > 2 is true but not True?  Why do we need 1 for True at all if we have
> > > True?
> >
> >     >>> 1 is not True
> >     True
> >     >>> 1 == True
> >     True
> >     >>> isinstance(True, int)
> >     True
> >
> > If you are suggesting that we change this behavior, stop, it is not
> > going to happen. If you are asking why this is the case, I would imagine,
> > is based in C/C++/many other sane and insane languages' handling of
> > boolean true.
> 
> It doesn't matter to me if it stays.  I thought 1 meaning True was
> just a concession to backward compatibility because Python didn't have
> True and False from the beginning.  So this would be a good time to
> ditch it.  Having 1 mean two different things means you can't tell
> immediately whether a variable is numeric or boolean; e.g., is it a
> line count or just a statement that some lines exist?  Fortunately I
> rarely see Python code nowadays that uses 1 this way.

But 1 doesn't mean True.  1 stands for the value 1.  True happens to be,
by long familliar semantics to many users, equal to 1, just as...

    bool c = true;
    printf("%i", c);

... will print 1 in C.  Don't read more into it than necessary.  As
Terry Reedy suggested, read PEP 285, which offers better reasoning than
I have or will provide.

 - Josiah


From barry at python.org  Tue Jan  2 07:48:55 2007
From: barry at python.org (Barry Warsaw)
Date: Tue, 2 Jan 2007 01:48:55 -0500
Subject: [Python-3000] PEP 3108: Standard Library Reorganization
In-Reply-To: <200701021447.01263.anthony@interlink.com.au>
References: <bbaeab100701011614l27a87f45pb7e782c1d78a0725@mail.gmail.com>
	<encabb$e5s$1@sea.gmane.org>
	<200701021447.01263.anthony@interlink.com.au>
Message-ID: <5F1ED8EA-061B-499E-B052-2924E190A903@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Jan 1, 2007, at 10:46 PM, Anthony Baxter wrote:

> I could not agree more. The list of modules Brett gave covers enough
> that pretty much every single piece of code I have will need to be
> edited. Worse yet, I _can't_ edit it and change it to the new names
> unless I want to abandon pre-2.6 compatibility. This is completely
> unacceptable. Living with a huge spew of warnings is not an
> acceptable outcome - for those of us running production systems in
> Python, a warning should be an indication of something to be fixed,
> not pointless log-filler.

I agree with everything Anthony just said.  One of the things I tried  
to do with the email package in Python 2.5 was transition to PEP 8  
names, but hack in enough backward compatibility that the old module  
names would Just Work.  I think I was largely successful at that --  
the only disconnect is that code that failed in Python 2.4 would fail  
slightly differently in Python 2.5, but nothing that worked in 2.4  
would fail in 2.5.

To me, a standard library reorganization would proceed like this:  
figure out the new hierarchy and package names now and implement them  
for Python 2.6.  Make sure every old package name continues to work,  
but encourage people (through documentation and social coercion, not  
through warnings) to start using the new hierarchy.  Issue no  
warnings in Python 2.6 unless the magical -3 flag were given, but  
never make the old hierarchy issue errors in Python 2.x.  I  
personally wouldn't care if the old hierarchy just disappears in  
Python 3.0 if the Magical Code Tranmogrifier did most of the  
conversion work well enough.

As for the mass extinction being planned, I'm not so sure.  I have a  
vague unease about it.  For one thing I think that the str/unicode ->  
unicode/bytearray change will be disruptive enough that we may not  
fully understand what interfaces we really want until that happens.   
I also have this feeling that by ditching so much that's widely used,  
we're setting Python 3.0 up for a lot of criticism that the batteries  
were removed.  For example, as icky as all those server modules are,  
they're darn handy and a lot of code has been written for them.   
Maybe I'm totally off base here, but it seems like it's one thing to  
restructure the hierarchy, and pepify the names, but it's another to  
just remove code unless there are compelling enough alternatives that  
folks are willing to rewrite everything to use them.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRZoAWHEjvBPtnXfVAQKcqwP/aVuZfWW6XWSlIZkFejndNscTbwrBc3Bz
qFEudQ4fcvZVGLa70u+q6oVLHnmINZ3rcYKogzsBC5VgZYcxkOC54BP9EhxZ8N1A
Byxmcb/pd4bmx1llaFZmEtNZFWnTRW9W+JK6PHkuRkcizH3sTlQLZ9dIKgKKsck1
iVao7uVvX2Y=
=Sjeg
-----END PGP SIGNATURE-----

From jcarlson at uci.edu  Tue Jan  2 08:20:55 2007
From: jcarlson at uci.edu (Josiah Carlson)
Date: Mon, 01 Jan 2007 23:20:55 -0800
Subject: [Python-3000] PEP 3108: Standard Library Reorganization
In-Reply-To: <bbaeab100701011944n7b73a33aq68b3ac0f50b7b8f3@mail.gmail.com>
References: <20070101174925.82FB.JCARLSON@uci.edu>
	<bbaeab100701011944n7b73a33aq68b3ac0f50b7b8f3@mail.gmail.com>
Message-ID: <20070101225107.8315.JCARLSON@uci.edu>


"Brett Cannon" <brett at python.org> wrote:
> On 1/1/07, Josiah Carlson <jcarlson at uci.edu> wrote:
> > "Brett Cannon" <brett at python.org> wrote:
> > * telnetlib
> > >     + Telnet is not used very much anymore.
> > >         - Telnet is unsafe.
> > >         - Most people use SSH instead.
> >
> > I don't know how common telnet lib use is currently, but I have found it
> > useful in writing plaintext line-based clients (like SMTP, POP, HTTP,
> > IMAP, etc.), before moving on to writing async* derived clients.
> 
> Fair enough.  If more people step up it can stay.
> 
> > * base64/quopri/uu
> > >     + Support exists in the codecs module.
> > >     + If removed (along with binhex), also remove binascii.
> > >         - C implementation of base64, binhex, and uu modules.
> >
> > The binascii module has become a catch all module for various plaintext
> > <-> binary conversions.
> 
> 
> Right, but is it necessarily the best thing that it has become a catch-all?

I would guess that Raymond Hettinger would have something to say about
this (being that he was going to add the long/int <-> binary conversion
to binascii). I don't know if he has been paying attention and/or
reading py3k.


> > * asynchat/asyncore
> > >     + Third-party libraries provide better solutions.
> > >         - twisted [#twisted]_
> > >     + Deprecation previously supported [#py-dev-summary-2004-11-01]_
> >
> > There is quite a bit of user code that depends on asynchat/asyncore
> > modules.  While there are many people who end up using twisted, forcing
> > all asynchronous socket users to "drink the twisted kool-aid" would set
> > a bad precident for any commonly used stdlib Python library with a more
> > fully-featured 3rd party module.
> 
> Well, I just remember the thread that led to the suggestion the two modules
> go.  It can be saved from the chopping block but I remember part of the
> issue was no one was wanting to maintain the modules anymore and they were
> starting to collect dust.

Removing asynchat/asyncore would also necessitate removing smtpd, which
is a subclass of asynchat.async_chat .

Dennis Allison offers the following in that thread,

    It might be worthwhile to list what the existing problems are with
    the asyncore and asynchat libraries.  There is also the problem that
    many applications, Zope among them, depend upon them.

http://mail.python.org/pipermail/python-dev/2004-November/049832.html

A later post points off to the Zope mailing list for reasons why
asyncore/asynchat are a problem, but Martin points out that he doesn't
see any "problems" specified by the poster as applicable.

http://mail.python.org/pipermail/python-dev/2004-November/049834.html

Guido states,

    IMO (after having worked with it extensively in ZEO/zrpc) asyncore
    is a really small piece of functionality that you'll need no matter
    how you slice it, if you don't want to do all your I/O synchronously
    (which would mean one thread per connection).

http://mail.zope.org/pipermail/zope3-dev/2002-October/003232.html

If Martin and Guido think that asyncore is fine, and one would need to
completely switch paradigms to use the only alternate, then it really
doesn't make sense to remove it.


If asyncore/asynchat/smtpd needs a maintainer, then I volunteer.  Assign
all bugs, patches, etc., to me and I'll handle them.


> > * stat
> > >     + ``os.stat`` now returns a tuple with attributes.
> >
> > The stat module also offers useful functions like stat.IS_DIR() for
> > determining if an object is a directory.
> 
> 
> Perhaps the functions should move to os or be part of the object returned by
> os.stat.

Methods sound better (to me) than functions in the os module.  If for
the reason that os.stat(fn).ST_ISDIR() would be easier to remember than
os.ST_ISDIR(os.stat(fn).st_mode) .

> > * SocketServer
> > >     socketserver
> >
> > SocketServer and its derivatives (*HTTPServer) are tricky (I've been
> > seeing them used quite a bit)...
> 
> 
> Well, I am not proposing to remove them although I would not cry if the
> HTTPServer children disappeared, but that's just me.  We will have to see if
> anyone else is up for letting them go.

I'm not voting for their removal, just stating that they are trickly to
handle, both in a naming, and in an understanding perspective.  Honestly,
which is easier to read and/or understand to you: basehttpserver or
BaseHTTPServer?  I prefer CamelCase.


> > * Servers
> > >     + BaseHTTPServer
> > >     + CGIHTTPServer
> > >     + DocXMLRPCServer
> > >     + SimpleHTTPServer
> > >     + SimpleXMLRPCServer
> > >     + SocketServer
> >
> > Would it be a reasonable semantic to say that 'no top level modules or
> > packages can violate PEP 8 module/package naming guidelines', but
> > sub-modules or packages may use CamelCase by historical precident or
> > when doing so would make understanding the purpose of the module/package
> > easier?
> 
> Could.  The renaming as of right now only enforces it on the package name
> stringently.  But the style guide is not my call.

xml.etree.ElementTree could be used as an example of what could/should
or shouldn't be done.

 - Josiah


From phd at phd.pp.ru  Tue Jan  2 11:32:34 2007
From: phd at phd.pp.ru (Oleg Broytmann)
Date: Tue, 2 Jan 2007 13:32:34 +0300
Subject: [Python-3000] PEP 3108: Standard Library Reorganization
In-Reply-To: <bbaeab100701011944n7b73a33aq68b3ac0f50b7b8f3@mail.gmail.com>
References: <bbaeab100701011614l27a87f45pb7e782c1d78a0725@mail.gmail.com>
	<20070101174925.82FB.JCARLSON@uci.edu>
	<bbaeab100701011944n7b73a33aq68b3ac0f50b7b8f3@mail.gmail.com>
Message-ID: <20070102103234.GC19263@phd.pp.ru>

On Mon, Jan 01, 2007 at 07:44:08PM -0800, Brett Cannon wrote:
> >* telnetlib
> >>     + Telnet is not used very much anymore.
> >>         - Telnet is unsafe.
> >>         - Most people use SSH instead.
> >
> >I don't know how common telnet lib use is currently, but I have found it
> >useful in writing plaintext line-based clients (like SMTP, POP, HTTP,
> >IMAP, etc.), before moving on to writing async* derived clients.
> 
> Fair enough.  If more people step up it can stay.

   There are devices (ADSL modems, e.g.) that can be connected to by
telnet. It is simpler to parse their telnet output (uptime and traffic)
than their HTTP/HTML output from a cron job.

Oleg.
-- 
     Oleg Broytmann            http://phd.pp.ru/            phd at phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.

From collinw at gmail.com  Tue Jan  2 19:08:56 2007
From: collinw at gmail.com (Collin Winter)
Date: Tue, 2 Jan 2007 12:08:56 -0600
Subject: [Python-3000] What's the point of annotations?
Message-ID: <43aa6ff70701021008y29a086a9i52294fd3b193f97a@mail.gmail.com>

I was explaining function annotations to a friend this past weekend
and found that, even though I had written the PEP and spent months
debating the little details of how annotations were to work, I was
hard-pressed to answer the question of "why are we doing this?"

The biggest problem I faced then and now is justifying the use-cases
for annotations. Here's what I could come up with off the top of my
head: information for typecheckers; doc strings for parameters; extra
information for IDEs; extra information for static analysis tools like
pylint. These can all be addressed together:

Are the users clamoring for these things? Do these address real
problems that users are having? Not to my knowledge.


1) Information for typecheckers -- in
http://mail.python.org/pipermail/python-ideas/2007-January/000035.html,
Guido said that "Collin's existing type annotation library ... could
be made more elegant by attaching the types directly to the
arguments". As far as I can tell, the only gains in elegance are that
you don't have to repeat the names of a function's parameters in the
typechecking decorator. None of my users have ever complained about
this tiny bit of repetition, and I've never felt it an undue burden in
my own usage.

It could even be considered an advantage, since including the
"annotations" in the typechecking decorator means all I have to do to
remove typechecking from a function is delete a single line, rather
than pick through a function's declaration, removing the relevant
bits.


2) Doc strings for parameters --

def foo(a: "the object to be frobnicated",
             b=7: "this controls the level of frobnication",
             c="Rojo": "the name of a color, in Spanish, that should
be applied to the \
                   frobnicated thing") -> "Returns an integer between
9 and 13":
    """ Frobnicate an object in a spanish way """
    ...

What does this accomplish that can't be achieved with any of the
standard documentation syntaxes in existence today?


3) Type information for IDEs -- I can see it being genuinely useful to
be able to get parameter/return type information in a tooltip message.
But IDLE can do this already, without annotations:
http://www.python.org/idle/doc/idle2.html#Tips


4) Type information for static analysis tools -- Quoting Nick Coghlan
(http://mail.python.org/pipermail/python-3000/2006-August/003043.html):
"annotations wouldn't be useful for tools like pychecker ... to be
really useful for a tool like pychecker they'd have to be ubiquitous,
and that's really not Python any more". Agreed.


I could say You Aren't Going to Need It, but that gets the tense
wrong; we're getting along without annotations quite nicely here in
the present. In short: I'd like to request that PEP 3107 be rejected
as an overly-specific, unnecessary addition to the language.

Basking-in-the-irony-ly,
Collin Winter

From barry at python.org  Tue Jan  2 19:15:17 2007
From: barry at python.org (Barry Warsaw)
Date: Tue, 2 Jan 2007 13:15:17 -0500
Subject: [Python-3000] What's the point of annotations?
In-Reply-To: <43aa6ff70701021008y29a086a9i52294fd3b193f97a@mail.gmail.com>
References: <43aa6ff70701021008y29a086a9i52294fd3b193f97a@mail.gmail.com>
Message-ID: <6779DA21-C3B5-4587-BC91-2AF0ADFE0069@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Jan 2, 2007, at 1:08 PM, Collin Winter wrote:

> The biggest problem I faced then and now is justifying the use-cases
> for annotations. Here's what I could come up with off the top of my
> head: information for typecheckers; doc strings for parameters; extra
> information for IDEs; extra information for static analysis tools like
> pylint. These can all be addressed together:

I've been using epydoc to document my public interfaces.  It seems  
like most of those use cases are (or easily could be) supported by  
epydoc docstrings.  Certainly for documenting parameters, epydoc is  
much more readable than the example you gave.  If you can get over  
parsing a docstring, it might already be enough.

- -Barry


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRZqhNnEjvBPtnXfVAQIISQQAgBXaHH7l+FqcZa9N0uaEuW8T8rN0A0c4
CoIYJJWu4s5Wb0h9j0J7e9opLHj/Pg9MZwNzJtEHIA2DnnNSiL34TxWkhWUxPuqC
C/9fRinBZ2HeFhwNo8p/Vol6kl31GzUgkMSvh7ovjml5dCfY9IgL7ynYFUGVD0UV
W6JaS/gXe08=
=AMVp
-----END PGP SIGNATURE-----

From pje at telecommunity.com  Tue Jan  2 19:22:53 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue, 02 Jan 2007 13:22:53 -0500
Subject: [Python-3000] What's the point of annotations?
In-Reply-To: <43aa6ff70701021008y29a086a9i52294fd3b193f97a@mail.gmail.co
 m>
Message-ID: <5.1.1.6.0.20070102132113.02a5a968@sparrow.telecommunity.com>

At 12:08 PM 1/2/2007 -0600, Collin Winter wrote:
>I could say You Aren't Going to Need It, but that gets the tense
>wrong; we're getting along without annotations quite nicely here in
>the present. In short: I'd like to request that PEP 3107 be rejected
>as an overly-specific, unnecessary addition to the language.

You left off overloading and argument adaptation as use cases.  These are 
considerably more readable when specified in-line, just like decorators are 
more readable placed above the function than after it.

As with decorators, these uses happen today, but are more awkward without 
the syntax.


From sluggoster at gmail.com  Tue Jan  2 19:58:59 2007
From: sluggoster at gmail.com (Mike Orr)
Date: Tue, 2 Jan 2007 10:58:59 -0800
Subject: [Python-3000] PEP 3108: Standard Library Reorganization
In-Reply-To: <5F1ED8EA-061B-499E-B052-2924E190A903@python.org>
References: <bbaeab100701011614l27a87f45pb7e782c1d78a0725@mail.gmail.com>
	<encabb$e5s$1@sea.gmane.org>
	<200701021447.01263.anthony@interlink.com.au>
	<5F1ED8EA-061B-499E-B052-2924E190A903@python.org>
Message-ID: <6e9196d20701021058w4870e3c2t216a1c12570322fd@mail.gmail.com>

On 1/1/07, Barry Warsaw <barry at python.org> wrote:
> I also have this feeling that by ditching so much that's widely used,
> we're setting Python 3.0 up for a lot of criticism that the batteries
> were removed.

I was uncomfortable with telnet leaving for that reason.

> For example, as icky as all those server modules are,
> they're darn handy and a lot of code has been written for them.
> Maybe I'm totally off base here, but it seems like it's one thing to
> restructure the hierarchy, and pepify the names, but it's another to
> just remove code unless there are compelling enough alternatives that
> folks are willing to rewrite everything to use them.

There is the 'lib-old' directory for modules that are unsupported but
we're not ready to finish off.  Alternatively, they can be put in a
Cheeseshop package.

-- 
Mike Orr <sluggoster at gmail.com>

From bjourne at gmail.com  Tue Jan  2 20:13:14 2007
From: bjourne at gmail.com (=?ISO-8859-1?Q?BJ=F6rn_Lindqvist?=)
Date: Tue, 2 Jan 2007 20:13:14 +0100
Subject: [Python-3000] What's the point of annotations?
In-Reply-To: <5.1.1.6.0.20070102132113.02a5a968@sparrow.telecommunity.com>
References: <5.1.1.6.0.20070102132113.02a5a968@sparrow.telecommunity.com>
Message-ID: <740c3aec0701021113x38f4778cg5fbee6f6087efb8e@mail.gmail.com>

On 1/2/07, Phillip J. Eby <pje at telecommunity.com> wrote:
> At 12:08 PM 1/2/2007 -0600, Collin Winter wrote:
> >I could say You Aren't Going to Need It, but that gets the tense
> >wrong; we're getting along without annotations quite nicely here in
> >the present. In short: I'd like to request that PEP 3107 be rejected
> >as an overly-specific, unnecessary addition to the language.
>
> You left off overloading and argument adaptation as use cases.  These are
> considerably more readable when specified in-line, just like decorators are
> more readable placed above the function than after it.

I don't understand. How does function annotations in itself make
overloading and argument adaptation more readable? And with
"overloading," do you mean function overloading? I thought that wasn't
possible in Python and even if it was, it was a bad idea. The FAQ says
something like that. I'd like to see a before-and-after example.

> As with decorators, these uses happen today, but are more awkward without
> the syntax.

Where?

-- 
mvh Bj?rn

From sluggoster at gmail.com  Tue Jan  2 20:16:25 2007
From: sluggoster at gmail.com (Mike Orr)
Date: Tue, 2 Jan 2007 11:16:25 -0800
Subject: [Python-3000] PEP 3108: Standard Library Reorganization
In-Reply-To: <20070101225107.8315.JCARLSON@uci.edu>
References: <20070101174925.82FB.JCARLSON@uci.edu>
	<bbaeab100701011944n7b73a33aq68b3ac0f50b7b8f3@mail.gmail.com>
	<20070101225107.8315.JCARLSON@uci.edu>
Message-ID: <6e9196d20701021116v272f1c06tb515d29c16c6d03f@mail.gmail.com>

On 1/1/07, Josiah Carlson <jcarlson at uci.edu> wrote:
> > > * stat
> > > >     + ``os.stat`` now returns a tuple with attributes.
> > >
> > > The stat module also offers useful functions like stat.IS_DIR() for
> > > determining if an object is a directory.
> >
> >
> > Perhaps the functions should move to os or be part of the object returned by
> > os.stat.
>
> Methods sound better (to me) than functions in the os module.  If for
> the reason that os.stat(fn).ST_ISDIR() would be easier to remember than
> os.ST_ISDIR(os.stat(fn).st_mode) .

os.path.isdir() already exists, and it does the same thing in a more
friendly way.  The issue is the four functions that have no
counterpart outside the stat module: S_ISCHR, S_ISBLK, S_ISFIFO,
S_ISSOCK.  These might as well be parallel to the others.

I think Jim is writing a PEP that would move all the os.path.is*()
functions into os because they access the filesystem (except
os.path.isabs()).  That's a separate issue.

-- 
Mike Orr <sluggoster at gmail.com>

From pje at telecommunity.com  Tue Jan  2 20:19:05 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue, 02 Jan 2007 14:19:05 -0500
Subject: [Python-3000] What's the point of annotations?
In-Reply-To: <740c3aec0701021113x38f4778cg5fbee6f6087efb8e@mail.gmail.co
 m>
References: <5.1.1.6.0.20070102132113.02a5a968@sparrow.telecommunity.com>
	<5.1.1.6.0.20070102132113.02a5a968@sparrow.telecommunity.com>
Message-ID: <5.1.1.6.0.20070102141711.0279fbd0@sparrow.telecommunity.com>

At 08:13 PM 1/2/2007 +0100, BJ?rn Lindqvist wrote:
>On 1/2/07, Phillip J. Eby <pje at telecommunity.com> wrote:
>>At 12:08 PM 1/2/2007 -0600, Collin Winter wrote:
>> >I could say You Aren't Going to Need It, but that gets the tense
>> >wrong; we're getting along without annotations quite nicely here in
>> >the present. In short: I'd like to request that PEP 3107 be rejected
>> >as an overly-specific, unnecessary addition to the language.
>>
>>You left off overloading and argument adaptation as use cases.  These are
>>considerably more readable when specified in-line, just like decorators are
>>more readable placed above the function than after it.
>
>I don't understand. How does function annotations in itself make
>overloading and argument adaptation more readable? And with
>"overloading," do you mean function overloading? I thought that wasn't
>possible in Python and even if it was, it was a bad idea. The FAQ says
>something like that.

Please see the lengthy Python-3000 threads, and Guido's blog.

>  I'd like to see a before-and-after example.

@overload(int, str)
def foo(bar, baz):
     ...

@overload
def foo(bar: int, baz: str)
     ...


From jimjjewett at gmail.com  Tue Jan  2 21:20:39 2007
From: jimjjewett at gmail.com (Jim Jewett)
Date: Tue, 2 Jan 2007 15:20:39 -0500
Subject: [Python-3000] self-contained exceptions
In-Reply-To: <ca471dc20701011630l633130e0n356eb9b176f1d610@mail.gmail.com>
References: <1d85506f0701011438q6d8b65bfi4ef1e8f9fa509650@mail.gmail.com>
	<ca471dc20701011630l633130e0n356eb9b176f1d610@mail.gmail.com>
Message-ID: <fb6fbf560701021220h4cb1b656u21c83d1b8b916fe5@mail.gmail.com>

On 1/1/07, Guido van Rossum <guido at python.org> wrote:

> I believe the main argument *against* having the tb included in the
> exception is that it would greatly increase the cyclic nature of
> tracebacks and stack frames, which in turn would cause lots of code to
> break due to late GC'ing of unclosed files, etc. For example, this
> code:
>
> try:
>   1/0
> except Exception, err:
>   pass
>
> will introduce a cycle from err -> traceback -> stack frame -> err,
> keeping all locals in the same scope alive until the next GC happens.

This should probably be made explicit in PEP 344.

I also keep thinking that there ought to be a solution involving weak
references, though I don't trust quite trust myself on what that
should be.  (The only obviously correct solution is to make the new
err->traceback link a weakref and not clean the sys.* interface.)

-jJ

From guido at python.org  Tue Jan  2 21:28:35 2007
From: guido at python.org (Guido van Rossum)
Date: Tue, 2 Jan 2007 12:28:35 -0800
Subject: [Python-3000] self-contained exceptions
In-Reply-To: <fb6fbf560701021220h4cb1b656u21c83d1b8b916fe5@mail.gmail.com>
References: <1d85506f0701011438q6d8b65bfi4ef1e8f9fa509650@mail.gmail.com>
	<ca471dc20701011630l633130e0n356eb9b176f1d610@mail.gmail.com>
	<fb6fbf560701021220h4cb1b656u21c83d1b8b916fe5@mail.gmail.com>
Message-ID: <ca471dc20701021228k592aced9g2244040f7ebc5b00@mail.gmail.com>

On 1/2/07, Jim Jewett <jimjjewett at gmail.com> wrote:
> On 1/1/07, Guido van Rossum <guido at python.org> wrote:
>
> > I believe the main argument *against* having the tb included in the
> > exception is that it would greatly increase the cyclic nature of
> > tracebacks and stack frames, which in turn would cause lots of code to
> > break due to late GC'ing of unclosed files, etc. For example, this
> > code:
> >
> > try:
> >   1/0
> > except Exception, err:
> >   pass
> >
> > will introduce a cycle from err -> traceback -> stack frame -> err,
> > keeping all locals in the same scope alive until the next GC happens.
>
> This should probably be made explicit in PEP 344.

Feel free to send a patch to me and the primary author (Ping). If Ping
agrees I can check it in.

> I also keep thinking that there ought to be a solution involving weak
> references, though I don't trust quite trust myself on what that
> should be.  (The only obviously correct solution is to make the new
> err->traceback link a weakref and not clean the sys.* interface.)

The problem with a weakref is that it won't keep the tb alive when you
save the exception for later (as is done for instance by the unittest
module). So it is of limited value and could even be confusing. OTOH
it *does* remove the need to call sys.exc_info() in the most common
case, so I don't want to rule it out completely.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From jimjjewett at gmail.com  Tue Jan  2 23:45:02 2007
From: jimjjewett at gmail.com (Jim Jewett)
Date: Tue, 2 Jan 2007 17:45:02 -0500
Subject: [Python-3000] PEP 3108: Standard Library Reorganization
In-Reply-To: <bbaeab100701011614l27a87f45pb7e782c1d78a0725@mail.gmail.com>
References: <bbaeab100701011614l27a87f45pb7e782c1d78a0725@mail.gmail.com>
Message-ID: <fb6fbf560701021445o7332dcc0hc9d1767b4da338d5@mail.gmail.com>

On 1/1/07, Brett Cannon <brett at python.org> wrote:
> =================
>
> Previously deprecated
> ---------------------
>
> Modules in this section have been deprecated at some point in the
> Python 2.x release series but are currently still distributed with
> Python

    (though not on all platforms).

> Minimal usage
> -------------
>
> Some modules that are platform-independent have minimal usage.  This
> can be from how easy it is to implement the functionality from scratch
> or because the audience for the code is small.

> * imputil

Assuming your import rewrite happens, then yes, all the legacy import
modules should be removed. :D

> * telnetlib
>     + Telnet is not used very much anymore.
>         - Telnet is unsafe.
>         - Most people use SSH instead.

I still use telnet.  I don't typically script it, but if I need to
script a network connection other than http, that is where I start.

I also note that SSH isn't available by default on windows (or some
unix builds), nor is there a stdlib wrapper for it.

> * asynchat/asyncore
>     + Third-party libraries provide better solutions.
>         - twisted [#twisted]_

Too much kool-aid.

"Everyone" sort of agrees that there are problems with the existing
libraries, but not on precisely what those problems are, let alone
what the solution should look like.  Some of the "this is obvious"
patches were rejected as breaking something for someone else.

> * getopt
>     + optparse provides better functionality.

Yes and no.  getopt does provide a (non-python-specific) standard
parsing, and is still widely used.

I would say keep it unless the converter could automate the
translation to *lightweigt* optparse usage.  (Shouldn't add more than
~15% to the script's "here is where I deal with options" code.)

> Modules to Rename
> =================

> PEP 8 violations
>  ----------------

> PEP 8 specifies that modules "should have short, lowercase names,
> without underscores" [#pep-0008]_.  There is no mention, though, if
> this rule extends to modules contained within a package.  The
> assumption is that underscores are acceptable in module names when
> they are contained within a package but that any uppercase letters is
> not.

PEP 8 should be clarified before this is implemented.

> * autoGIL
>     autogil
> * Carbon
>     carbon
> * ColorPicker
>     colorpicker
> * EasyDialogs
>     easydialogs
> * FrameWork
>     framework
> * MacOS
>     macos
> * MiniAEFrame
>     miniaeframe
> * Nav
>     nav
> * PixMapWrapper
>     pixmapwrapper
> * W
>     w

It would be best to rename these to indicate that they are
mac-specific.  (Well, that or include pure-python or Tk
implementations on other platforms.  There are already 3rd-party ports
for some.)  Some of them seem like the obvious answer for certain
problems, but then either the code isn't cross-platform (if you're
using a Mac) or the module doesn't exist (which makes you wonder if
you installed correctly).

Something as simple as mac.colorpicker, easydialogs, mac.nav, ...
would have been helpful.

I suppose the same question would apply to _winreg, and to any
not-mentioned platform-specific modules, but in practice, I seem to
have the most frustration with Mac-only modules, even though there
seem to be more Linux-only developers.

-jJ

From jimjjewett at gmail.com  Tue Jan  2 23:51:30 2007
From: jimjjewett at gmail.com (Jim Jewett)
Date: Tue, 2 Jan 2007 17:51:30 -0500
Subject: [Python-3000] Fwd:  PEP 3108: Standard Library Reorganization
In-Reply-To: <fb6fbf560701021355h22105298md534d186e8bb8c73@mail.gmail.com>
References: <bbaeab100701011614l27a87f45pb7e782c1d78a0725@mail.gmail.com>
	<encabb$e5s$1@sea.gmane.org>
	<200701021447.01263.anthony@interlink.com.au>
	<5F1ED8EA-061B-499E-B052-2924E190A903@python.org>
	<fb6fbf560701021355h22105298md534d186e8bb8c73@mail.gmail.com>
Message-ID: <fb6fbf560701021451q63546a2j4e7fe0f2e0227d6f@mail.gmail.com>

On 1/2/07, Barry Warsaw <barry at python.org> wrote:

> As for the mass extinction being planned, I'm not so sure.  I have a
> vague unease about it.  For one thing I think that the str/unicode ->
> unicode/bytearray change will be disruptive enough that we may not
> fully understand what interfaces we really want until that happens.

That probably won't affect which legacy formats (gopher, aiff, etc) get used.

> I also have this feeling that by ditching so much that's widely used,
> we're setting Python 3.0 up for a lot of criticism that the batteries
> were removed.  For example, as icky as all those server modules are,
> they're darn handy and a lot of code has been written for them.

I think the server modules in particular were listed only for
consolidation, not removal.

> Maybe I'm totally off base here, but it seems like it's one thing to
> restructure the hierarchy, and pepify the names, but it's another to
> just remove code unless there are compelling enough alternatives that
> folks are willing to rewrite everything to use them.

There were several modules I didn't recognize.  (Some don't exist
under windows, but some do.)  I still remember printing off several
inches of stdlib documentation to figure out what was possible.

The only way to fix that is to reduce the number of (top-level) modules.

I'm not saying that the functionality should go away, but I am saying
that

    When I find/see a reference to an obviously relevant module,
    it should be the *right* answer.

At the moment, there are three sibling candidates for http servers
(plus the server cookielib, and possible direct usage of httplib and
urllib*...)

-jJ

From brett at python.org  Wed Jan  3 00:16:39 2007
From: brett at python.org (Brett Cannon)
Date: Tue, 2 Jan 2007 15:16:39 -0800
Subject: [Python-3000] PEP 3108: Standard Library Reorganization
In-Reply-To: <20070102103234.GC19263@phd.pp.ru>
References: <bbaeab100701011614l27a87f45pb7e782c1d78a0725@mail.gmail.com>
	<20070101174925.82FB.JCARLSON@uci.edu>
	<bbaeab100701011944n7b73a33aq68b3ac0f50b7b8f3@mail.gmail.com>
	<20070102103234.GC19263@phd.pp.ru>
Message-ID: <bbaeab100701021516n1b1fa897ic620fd5f496f3267@mail.gmail.com>

On 1/2/07, Oleg Broytmann <phd at phd.pp.ru> wrote:
>
> On Mon, Jan 01, 2007 at 07:44:08PM -0800, Brett Cannon wrote:
> > >* telnetlib
> > >>     + Telnet is not used very much anymore.
> > >>         - Telnet is unsafe.
> > >>         - Most people use SSH instead.
> > >
> > >I don't know how common telnet lib use is currently, but I have found
> it
> > >useful in writing plaintext line-based clients (like SMTP, POP, HTTP,
> > >IMAP, etc.), before moving on to writing async* derived clients.
> >
> > Fair enough.  If more people step up it can stay.
>
>    There are devices (ADSL modems, e.g.) that can be connected to by
> telnet. It is simpler to parse their telnet output (uptime and traffic)
> than their HTTP/HTML output from a cron job.


OK, telnetlib can stay.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20070102/a4684f4f/attachment.html 

From brett at python.org  Wed Jan  3 00:21:33 2007
From: brett at python.org (Brett Cannon)
Date: Tue, 2 Jan 2007 15:21:33 -0800
Subject: [Python-3000] PEP 3108: Standard Library Reorganization
In-Reply-To: <6e9196d20701012213v4e65da9xd77d1bc0ea2656cf@mail.gmail.com>
References: <bbaeab100701011614l27a87f45pb7e782c1d78a0725@mail.gmail.com>
	<6e9196d20701012213v4e65da9xd77d1bc0ea2656cf@mail.gmail.com>
Message-ID: <bbaeab100701021521u65eb10d3q54df6d251a485202@mail.gmail.com>

On 1/1/07, Mike Orr <sluggoster at gmail.com> wrote:
>
> On 1/1/07, Brett Cannon <brett at python.org> wrote:
> > * thread
> >     + People should use 'threading' instead.
>
> 'threading' has no equivalent to 'thread.get_ident()'.  If this
> function is ported, I'd support renaming 'thread'.  You need it to
> print "Thread #123 doing bla bla bla" messages, to distinguish threads
> in a thread pool, to keep track of thread-local data, etc.  The
> threading.Thread name isn't the same thing because it's an arbitrary
> string, not a guaranteed-unique integer, and the routine creating the
> threads may be third-party and not setting unique identifiers.


This one you are going to have to argue it out with the people who want to
hide  the interface.  As for thread.get_ident(), it could be added to
threading.

> * stat
> >     + ``os.stat`` now returns a tuple with attributes.
>
> Certain things in 'stat' do not exist elsewhere:  S_ISCHR, S_ISBLK,
> S_ISFIFO, S_ISSOCK.  These could be made into more friendly
> os.path.is*() functions that take the pathname rather than the
> st_mode.  I've never used them, but it would be a pity to make people
> calculate the bitmasks manually.


It would be better to add them as methods to the tuple object returned by
os.stat.  But the module should still go.

> * fileinput
>
> +1 to leave it.



People seem to really like this module, so it can stay.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20070102/d35e4bf5/attachment.htm 

From brett at python.org  Wed Jan  3 00:24:22 2007
From: brett at python.org (Brett Cannon)
Date: Tue, 2 Jan 2007 15:24:22 -0800
Subject: [Python-3000] PEP 3108: Standard Library Reorganization
In-Reply-To: <8B3F3B90-9C0C-4A95-B8B2-3D3DE5DEEF0B@python.org>
References: <bbaeab100701011614l27a87f45pb7e782c1d78a0725@mail.gmail.com>
	<3d2ce8cb0701011903h694d7a5g58e008f79f48e00b@mail.gmail.com>
	<8B3F3B90-9C0C-4A95-B8B2-3D3DE5DEEF0B@python.org>
Message-ID: <bbaeab100701021524i788def3bo1ede740ea4f3c3f2@mail.gmail.com>

On 1/1/07, Barry Warsaw <barry at python.org> wrote:
>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On Jan 1, 2007, at 10:03 PM, Mike Klaas wrote:
>
> > On 1/1/07, Brett Cannon <brett at python.org> wrote:
> >> * new
> >>     + Just a rebinding of names from the 'types' module.
> >>     + Can also call ``type`` built-in to get most types easily.
> >
> > Perhaps 'types' should be deprecated instead?
>
> Don't forget that the types module contains some types that aren't
> readily available elsewhere.  It shouldn't be deprecated so quickly.


It hasn't been.  It was just an idea.  And since no one is stepping up to
support the idea it is pretty much dead in my mind.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20070102/6a6b99d4/attachment.html 

From brett at python.org  Wed Jan  3 00:39:22 2007
From: brett at python.org (Brett Cannon)
Date: Tue, 2 Jan 2007 15:39:22 -0800
Subject: [Python-3000] PEP 3108: Standard Library Reorganization
In-Reply-To: <5F1ED8EA-061B-499E-B052-2924E190A903@python.org>
References: <bbaeab100701011614l27a87f45pb7e782c1d78a0725@mail.gmail.com>
	<encabb$e5s$1@sea.gmane.org>
	<200701021447.01263.anthony@interlink.com.au>
	<5F1ED8EA-061B-499E-B052-2924E190A903@python.org>
Message-ID: <bbaeab100701021539p128dd4cbuff802d3d34e78a39@mail.gmail.com>

On 1/1/07, Barry Warsaw <barry at python.org> wrote:
>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On Jan 1, 2007, at 10:46 PM, Anthony Baxter wrote:
>
> > I could not agree more. The list of modules Brett gave covers enough
> > that pretty much every single piece of code I have will need to be
> > edited. Worse yet, I _can't_ edit it and change it to the new names
> > unless I want to abandon pre-2.6 compatibility. This is completely
> > unacceptable. Living with a huge spew of warnings is not an
> > acceptable outcome - for those of us running production systems in
> > Python, a warning should be an indication of something to be fixed,
> > not pointless log-filler.
>
> I agree with everything Anthony just said.  One of the things I tried
> to do with the email package in Python 2.5 was transition to PEP 8
> names, but hack in enough backward compatibility that the old module
> names would Just Work.  I think I was largely successful at that --
> the only disconnect is that code that failed in Python 2.4 would fail
> slightly differently in Python 2.5, but nothing that worked in 2.4
> would fail in 2.5.


The PEP has already been changed to use PendingDeprecationWarning, so that
will be as noisy as it gets.  From this point forward it will just be a
quieter warning.

To me, a standard library reorganization would proceed like this:
> figure out the new hierarchy and package names now and implement them
> for Python 2.6.  Make sure every old package name continues to work,
> but encourage people (through documentation and social coercion, not
> through warnings) to start using the new hierarchy.  Issue no
> warnings in Python 2.6 unless the magical -3 flag were given, but
> never make the old hierarchy issue errors in Python 2.x.  I
> personally wouldn't care if the old hierarchy just disappears in
> Python 3.0 if the Magical Code Tranmogrifier did most of the
> conversion work well enough.


Well, if someone wants to propose a -3 flag for warnings that's fine, but I
think that is a separate PEP.

As for just having the old naming never have a warning, I can live with that
if people are okay with it just disappearing in Py3K and completely relying
on the refactoring tool.  Is that what people want?


As for the mass extinction being planned, I'm not so sure.  I have a
> vague unease about it.  For one thing I think that the str/unicode ->
> unicode/bytearray change will be disruptive enough that we may not
> fully understand what interfaces we really want until that happens.
> I also have this feeling that by ditching so much that's widely used,
> we're setting Python 3.0 up for a lot of criticism that the batteries
> were removed.  For example, as icky as all those server modules are,
> they're darn handy and a lot of code has been written for them.
> Maybe I'm totally off base here, but it seems like it's one thing to
> restructure the hierarchy, and pepify the names, but it's another to
> just remove code unless there are compelling enough alternatives that
> folks are willing to rewrite everything to use them.


The server modules are not slated for removal.  All I proposed in the PEP
was grouping them in a package.  One person suggested (or at least I thought
they had) of there removal.  I said it can be discussed.  I am much more
lenient in removing modules for the removal list then adding to it.

But I don't think it is that far overboard.  Most of the stuff being
suggested is either old or marginal (in my opinion).  And people are already
speaking up on the ones that really should stick around (telnetlib and
fileinput).  This is all a give-and-take and we can find a balance.  Plus
Guido has not cleared anything so he might totally disagree with what I have
said (but I bet he is waiting for people to help me get this list to a good
set first).  Even with what I suggested there is still a ton of stuff in the
stdlib.  I really don't think anyone is going to think we pulled a battery
out of the stdlib.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20070102/7cee4ba1/attachment.htm 

From brett at python.org  Wed Jan  3 00:47:38 2007
From: brett at python.org (Brett Cannon)
Date: Tue, 2 Jan 2007 15:47:38 -0800
Subject: [Python-3000] PEP 3108: Standard Library Reorganization
In-Reply-To: <20070101225107.8315.JCARLSON@uci.edu>
References: <20070101174925.82FB.JCARLSON@uci.edu>
	<bbaeab100701011944n7b73a33aq68b3ac0f50b7b8f3@mail.gmail.com>
	<20070101225107.8315.JCARLSON@uci.edu>
Message-ID: <bbaeab100701021547l2eddd412wc11330c4cfcb46dd@mail.gmail.com>

On 1/1/07, Josiah Carlson <jcarlson at uci.edu> wrote:
>
>
> "Brett Cannon" <brett at python.org> wrote:
> > On 1/1/07, Josiah Carlson <jcarlson at uci.edu> wrote:
> > > "Brett Cannon" <brett at python.org> wrote:
> > > * telnetlib
> > > >     + Telnet is not used very much anymore.
> > > >         - Telnet is unsafe.
> > > >         - Most people use SSH instead.
> > >
> > > I don't know how common telnet lib use is currently, but I have found
> it
> > > useful in writing plaintext line-based clients (like SMTP, POP, HTTP,
> > > IMAP, etc.), before moving on to writing async* derived clients.
> >
> > Fair enough.  If more people step up it can stay.
> >
> > > * base64/quopri/uu
> > > >     + Support exists in the codecs module.
> > > >     + If removed (along with binhex), also remove binascii.
> > > >         - C implementation of base64, binhex, and uu modules.
> > >
> > > The binascii module has become a catch all module for various
> plaintext
> > > <-> binary conversions.
> >
> >
> > Right, but is it necessarily the best thing that it has become a
> catch-all?
>
> I would guess that Raymond Hettinger would have something to say about
> this (being that he was going to add the long/int <-> binary conversion
> to binascii). I don't know if he has been paying attention and/or
> reading py3k.


Raymond might, but I don't think he is following Py3K.  But MAL kicked the
PEP over to python-dev so he might notice now.

Are you at least okay with base64, quopri, and uu going?  You are just
arguing for the saving of binascii, right?

Does anyone else care to try to save binascii?

> > * asynchat/asyncore
> > > >     + Third-party libraries provide better solutions.
> > > >         - twisted [#twisted]_
> > > >     + Deprecation previously supported [#py-dev-summary-2004-11-01]_
> > >
> > > There is quite a bit of user code that depends on asynchat/asyncore
> > > modules.  While there are many people who end up using twisted,
> forcing
> > > all asynchronous socket users to "drink the twisted kool-aid" would
> set
> > > a bad precident for any commonly used stdlib Python library with a
> more
> > > fully-featured 3rd party module.
> >
> > Well, I just remember the thread that led to the suggestion the two
> modules
> > go.  It can be saved from the chopping block but I remember part of the
> > issue was no one was wanting to maintain the modules anymore and they
> were
> > starting to collect dust.
>
> Removing asynchat/asyncore would also necessitate removing smtpd, which
> is a subclass of asynchat.async_chat .
>
> Dennis Allison offers the following in that thread,
>
>     It might be worthwhile to list what the existing problems are with
>     the asyncore and asynchat libraries.  There is also the problem that
>     many applications, Zope among them, depend upon them.
>
> http://mail.python.org/pipermail/python-dev/2004-November/049832.html
>
> A later post points off to the Zope mailing list for reasons why
> asyncore/asynchat are a problem, but Martin points out that he doesn't
> see any "problems" specified by the poster as applicable.
>
> http://mail.python.org/pipermail/python-dev/2004-November/049834.html
>
> Guido states,
>
>     IMO (after having worked with it extensively in ZEO/zrpc) asyncore
>     is a really small piece of functionality that you'll need no matter
>     how you slice it, if you don't want to do all your I/O synchronously
>     (which would mean one thread per connection).
>
> http://mail.zope.org/pipermail/zope3-dev/2002-October/003232.html
>
> If Martin and Guido think that asyncore is fine, and one would need to
> completely switch paradigms to use the only alternate, then it really
> doesn't make sense to remove it.
>
>
> If asyncore/asynchat/smtpd needs a maintainer, then I volunteer.  Assign
> all bugs, patches, etc., to me and I'll handle them.


That's good enough for me.  They are saved.

> > * stat
> > > >     + ``os.stat`` now returns a tuple with attributes.
> > >
> > > The stat module also offers useful functions like stat.IS_DIR() for
> > > determining if an object is a directory.
> >
> >
> > Perhaps the functions should move to os or be part of the object
> returned by
> > os.stat.
>
> Methods sound better (to me) than functions in the os module.  If for
> the reason that os.stat(fn).ST_ISDIR() would be easier to remember than
> os.ST_ISDIR(os.stat(fn).st_mode) .


I have added the requirement for the removal of the stat module.  And
obviously the names will need some work.

> > * SocketServer
> > > >     socketserver
> > >
> > > SocketServer and its derivatives (*HTTPServer) are tricky (I've been
> > > seeing them used quite a bit)...
> >
> >
> > Well, I am not proposing to remove them although I would not cry if the
> > HTTPServer children disappeared, but that's just me.  We will have to
> see if
> > anyone else is up for letting them go.
>
> I'm not voting for their removal, just stating that they are trickly to
> handle, both in a naming, and in an understanding perspective.  Honestly,
> which is easier to read and/or understand to you: basehttpserver or
> BaseHTTPServer?  I prefer CamelCase.


Perhaps, but consistency is more important (at least to me).  Best bet would
be to put them in a package.

> > * Servers
> > > >     + BaseHTTPServer
> > > >     + CGIHTTPServer
> > > >     + DocXMLRPCServer
> > > >     + SimpleHTTPServer
> > > >     + SimpleXMLRPCServer
> > > >     + SocketServer
> > >
> > > Would it be a reasonable semantic to say that 'no top level modules or
> > > packages can violate PEP 8 module/package naming guidelines', but
> > > sub-modules or packages may use CamelCase by historical precident or
> > > when doing so would make understanding the purpose of the
> module/package
> > > easier?
> >
> > Could.  The renaming as of right now only enforces it on the package
> name
> > stringently.  But the style guide is not my call.
>
> xml.etree.ElementTree could be used as an example of what could/should
> or shouldn't be done.


It is its own special open issue for that reason.  =)  I personally would
prefer they go into a package, we ditch the CamelCase and move to
underscores for them.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20070102/d9474830/attachment.html 

From collinw at gmail.com  Wed Jan  3 00:48:20 2007
From: collinw at gmail.com (Collin Winter)
Date: Tue, 2 Jan 2007 17:48:20 -0600
Subject: [Python-3000] What's the point of annotations?
In-Reply-To: <5.1.1.6.0.20070102141711.0279fbd0@sparrow.telecommunity.com>
References: <5.1.1.6.0.20070102132113.02a5a968@sparrow.telecommunity.com>
	<5.1.1.6.0.20070102141711.0279fbd0@sparrow.telecommunity.com>
Message-ID: <43aa6ff70701021548s937327dt87a6722b25e39dca@mail.gmail.com>

On 1/2/07, Phillip J. Eby <pje at telecommunity.com> wrote:
> At 08:13 PM 1/2/2007 +0100, BJ?rn Lindqvist wrote:
> >On 1/2/07, Phillip J. Eby <pje at telecommunity.com> wrote:
> >>You left off overloading and argument adaptation as use cases.  These are
> >>considerably more readable when specified in-line, just like decorators are
> >>more readable placed above the function than after it.
>
> >  I'd like to see a before-and-after example.
>
> @overload(int, str)
> def foo(bar, baz):
>      ...
>
> @overload
> def foo(bar: int, baz: str)
>      ...

Is whatever savings you see there worth changing the language for? Is
function overloading really that important and that common?

I'm still not convinced that the language syntax should change to make
life a tiny bit easier for third-party libraries. Not every n-line
function should go in the stdlib or be turned in to syntax.

Thanks,
Collin Winter

From brett at python.org  Wed Jan  3 00:53:00 2007
From: brett at python.org (Brett Cannon)
Date: Tue, 2 Jan 2007 15:53:00 -0800
Subject: [Python-3000] PEP 3108: Standard Library Reorganization
In-Reply-To: <fb6fbf560701021445o7332dcc0hc9d1767b4da338d5@mail.gmail.com>
References: <bbaeab100701011614l27a87f45pb7e782c1d78a0725@mail.gmail.com>
	<fb6fbf560701021445o7332dcc0hc9d1767b4da338d5@mail.gmail.com>
Message-ID: <bbaeab100701021553n6b110f8ds6763c27cbc17fa13@mail.gmail.com>

On 1/2/07, Jim Jewett <jimjjewett at gmail.com> wrote:
>
> On 1/1/07, Brett Cannon <brett at python.org> wrote:
> > =================
> >
> > Previously deprecated
> > ---------------------
> >
> > Modules in this section have been deprecated at some point in the
> > Python 2.x release series but are currently still distributed with
> > Python
>
>     (though not on all platforms).
>
> > Minimal usage
> > -------------
> >
> > Some modules that are platform-independent have minimal usage.  This
> > can be from how easy it is to implement the functionality from scratch
> > or because the audience for the code is small.
>
> > * imputil
>
> Assuming your import rewrite happens, then yes, all the legacy import
> modules should be removed. :D


=)  The rewrite has basically happened.  I still need to add support for
case-insensitive filesystems, but otherwise it's done (code needs at least
one read-through, mostly to possibly rename or hide some of the API).  Still
need to do docs and choose a name for the modle.

> * telnetlib
> >     + Telnet is not used very much anymore.
> >         - Telnet is unsafe.
> >         - Most people use SSH instead.
>
> I still use telnet.  I don't typically script it, but if I need to
> script a network connection other than http, that is where I start.
>
> I also note that SSH isn't available by default on windows (or some
> unix builds), nor is there a stdlib wrapper for it.


It's already been saved.


> > * asynchat/asyncore
> >     + Third-party libraries provide better solutions.
> >         - twisted [#twisted]_
>
> Too much kool-aid.
>
> "Everyone" sort of agrees that there are problems with the existing
> libraries, but not on precisely what those problems are, let alone
> what the solution should look like.  Some of the "this is obvious"
> patches were rejected as breaking something for someone else.


Saved.

> * getopt
> >     + optparse provides better functionality.
>
> Yes and no.  getopt does provide a (non-python-specific) standard
> parsing, and is still widely used.
>
> I would say keep it unless the converter could automate the
> translation to *lightweigt* optparse usage.  (Shouldn't add more than
> ~15% to the script's "here is where I deal with options" code.)
>
> > Modules to Rename
> > =================
>
> > PEP 8 violations
> >  ----------------
>
> > PEP 8 specifies that modules "should have short, lowercase names,
> > without underscores" [#pep-0008]_.  There is no mention, though, if
> > this rule extends to modules contained within a package.  The
> > assumption is that underscores are acceptable in module names when
> > they are contained within a package but that any uppercase letters is
> > not.
>
> PEP 8 should be clarified before this is implemented.


That's a Barry/Guido thing.  I am happy with my interpretation.

> * autoGIL
> >     autogil
> > * Carbon
> >     carbon
> > * ColorPicker
> >     colorpicker
> > * EasyDialogs
> >     easydialogs
> > * FrameWork
> >     framework
> > * MacOS
> >     macos
> > * MiniAEFrame
> >     miniaeframe
> > * Nav
> >     nav
> > * PixMapWrapper
> >     pixmapwrapper
> > * W
> >     w
>
> It would be best to rename these to indicate that they are
> mac-specific.  (Well, that or include pure-python or Tk
> implementations on other platforms.  There are already 3rd-party ports
> for some.)  Some of them seem like the obvious answer for certain
> problems, but then either the code isn't cross-platform (if you're
> using a Mac) or the module doesn't exist (which makes you wonder if
> you installed correctly).
>
> Something as simple as mac.colorpicker, easydialogs, mac.nav, ...
> would have been helpful.
>
> I suppose the same question would apply to _winreg, and to any
> not-mentioned platform-specific modules, but in practice, I seem to
> have the most frustration with Mac-only modules, even though there
> seem to be more Linux-only developers.



That's why one of the open issues is to group platform-specific stuff (that
is definitely platform-specific) should be in a package for that platform.
Just need support from people to push for it with Guido.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20070102/abe797e9/attachment.htm 

From barry at python.org  Wed Jan  3 01:02:53 2007
From: barry at python.org (Barry Warsaw)
Date: Tue, 2 Jan 2007 19:02:53 -0500
Subject: [Python-3000] PEP 3108: Standard Library Reorganization
In-Reply-To: <bbaeab100701021547l2eddd412wc11330c4cfcb46dd@mail.gmail.com>
References: <20070101174925.82FB.JCARLSON@uci.edu>
	<bbaeab100701011944n7b73a33aq68b3ac0f50b7b8f3@mail.gmail.com>
	<20070101225107.8315.JCARLSON@uci.edu>
	<bbaeab100701021547l2eddd412wc11330c4cfcb46dd@mail.gmail.com>
Message-ID: <6832200B-6A66-4E3D-88C1-E7ADF326CD2E@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Jan 2, 2007, at 6:47 PM, Brett Cannon wrote:

> Are you at least okay with base64, quopri, and uu going?  You are just
> arguing for the saving of binascii, right?
>
> Does anyone else care to try to save binascii?

base64 and quopri implement specific RFCs so I think they should  
stay.  uu implements a defacto standard, but I don't like its  
interface (it uses file-like objects instead of strings).  For Py3K,  
I would be in favor of consolidating things, making interfaces  
consistent, and probably re-homing the modules under a top-level  
package.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRZryrXEjvBPtnXfVAQKesQP+IaGMjFcfXKE+wJ3bID4J72V0QLwFtRrP
5smxwHHvNoDFxRSP2YnsN2ISv41p5O8KRnU73tyv6ueFoDG4vNlHp+sDFQefmWzO
9FGA6IVIR8DUzlW5b8VeHAELEIrol4E6k5y6JeS0CjdbdFz9XTZ0UZCG0sM//Dvx
V1dYG1ToNKQ=
=L6j2
-----END PGP SIGNATURE-----

From jimjjewett at gmail.com  Wed Jan  3 01:04:16 2007
From: jimjjewett at gmail.com (Jim Jewett)
Date: Tue, 2 Jan 2007 19:04:16 -0500
Subject: [Python-3000] self-contained exceptions
In-Reply-To: <ca471dc20701021228k592aced9g2244040f7ebc5b00@mail.gmail.com>
References: <1d85506f0701011438q6d8b65bfi4ef1e8f9fa509650@mail.gmail.com>
	<ca471dc20701011630l633130e0n356eb9b176f1d610@mail.gmail.com>
	<fb6fbf560701021220h4cb1b656u21c83d1b8b916fe5@mail.gmail.com>
	<ca471dc20701021228k592aced9g2244040f7ebc5b00@mail.gmail.com>
Message-ID: <fb6fbf560701021604g6bd4655dgea4d3afaf23a7375@mail.gmail.com>

On 1/2/07, Guido van Rossum <guido at python.org> wrote:
> On 1/2/07, Jim Jewett <jimjjewett at gmail.com> wrote:
> > On 1/1/07, Guido van Rossum <guido at python.org> wrote:
> > > I believe the main argument *against*

> > This should probably be made explicit in PEP 344.

> Feel free to send a patch to me and the primary author (Ping). If Ping
> agrees I can check it in.

http://www.python.org/sf/1626538

From pje at telecommunity.com  Wed Jan  3 01:20:38 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue, 02 Jan 2007 19:20:38 -0500
Subject: [Python-3000] What's the point of annotations?
In-Reply-To: <43aa6ff70701021548s937327dt87a6722b25e39dca@mail.gmail.com
 >
References: <5.1.1.6.0.20070102141711.0279fbd0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070102132113.02a5a968@sparrow.telecommunity.com>
	<5.1.1.6.0.20070102141711.0279fbd0@sparrow.telecommunity.com>
Message-ID: <5.1.1.6.0.20070102185639.02920e70@sparrow.telecommunity.com>

At 05:48 PM 1/2/2007 -0600, Collin Winter wrote:
>Is whatever savings you see there worth changing the language for? Is
>function overloading really that important and that common?

Ah, but now you're making a different argument.  Your original statement 
was that "we're getting along without annotations quite nicely"; my 
response was merely to refute that point.  There do indeed exist use cases 
that are better off with annotation than without.

In addition, you're reducing my statement to only a single use case: not 
only did I list two, but there are plenty of others.  I felt that two use 
cases were sufficient to refute the proposition that there were *no* use 
cases.  But there are certainly others.

For example, interfacing to at least Java and Objective C have also been 
mentioned as use cases, and I can think of a variety of other uses such as 
database query mapping, predicate logic functions, and web or other types 
of RPC parameter marshalling.

Now, if you want to pick at each and every one of these use cases and 
assert that the one use case, *by itself*, doesn't justify the feature, 
well, you may be right.  But as with decorators, that doesn't necessarily 
mean the feature itself is wrong!  Decorators are an excellent example of a 
feature that meets many use cases, few if any of which would be sufficient 
in themselves to justify a language change.


From rasky at develer.com  Wed Jan  3 01:21:37 2007
From: rasky at develer.com (Giovanni Bajo)
Date: Wed, 03 Jan 2007 01:21:37 +0100
Subject: [Python-3000] PEP 3108: Standard Library Reorganization
In-Reply-To: <fb6fbf560701021445o7332dcc0hc9d1767b4da338d5@mail.gmail.com>
References: <bbaeab100701011614l27a87f45pb7e782c1d78a0725@mail.gmail.com>
	<fb6fbf560701021445o7332dcc0hc9d1767b4da338d5@mail.gmail.com>
Message-ID: <enesub$64s$1@sea.gmane.org>

Jim Jewett wrote:

>> * getopt
>>     + optparse provides better functionality.
> 
> Yes and no.  getopt does provide a (non-python-specific) standard
> parsing, and is still widely used.
> I would say keep it unless the converter could automate the
> translation to *lightweigt* optparse usage.  (Shouldn't add more than
> ~15% to the script's "here is where I deal with options" code.)

I'm not sure what you mean here. In my experience, converting to optparse 
results in *less* code, especially if the script has a relatively complex 
command line interface.

Let alone the fact that it makes possible to adhere to the "say it once" 
principle, while getopt forces to several repetitions (each option is repeated 
in the getopt argument strings, in the if/else chain, and in the usage screen 
at least).

Are you worried more for existing code or for new code? Do you believe getopt 
still makes sense even for new code, and if so, would you please elaborate on 
this?
-- 
Giovanni Bajo


From janssen at parc.com  Wed Jan  3 01:51:20 2007
From: janssen at parc.com (Bill Janssen)
Date: Tue, 2 Jan 2007 16:51:20 PST
Subject: [Python-3000] PEP 3108: Standard Library Reorganization
In-Reply-To: <bbaeab100701011614l27a87f45pb7e782c1d78a0725@mail.gmail.com> 
References: <bbaeab100701011614l27a87f45pb7e782c1d78a0725@mail.gmail.com>
Message-ID: <07Jan2.165122pst."58648"@synergy1.parc.xerox.com>

Brett,

Thanks for writing this up.

Here are my thoughts on this (after 3 weeks away, I'm still catching up):

The link to PEP 4 is broken.

"nis" -- NIS is still widely used by many people; probably premature to remove.

"md5" and "sha" -- should note pointer to "hashlib".

"base64"/"quopri"/"uu" -- still quite useful, so I'd suggest removing
them but retaining "binascii", with a new implementation that uses the
"codecs" support directly.

Thank God "asyncore" was 'saved'.

I hate the renaming ideas, but I agree it's probably necessary.

Let me suggest a consolidation idea I don't see there:  a grouping
similar to that which has been done with email, but called "web".
This package would include "urllib", "urllib2", "urlparse", "httplib",
"cgi", "BaseHTTPServer", "BaseCGIServer", "HTMLParser", "Cookie",
"cookielib", and "SimpleHTTPServer" (at least).  The Python ECMAScript
interpreter could go there, too, when it's written :-).

Bill

From brett at python.org  Wed Jan  3 01:52:29 2007
From: brett at python.org (Brett Cannon)
Date: Tue, 2 Jan 2007 16:52:29 -0800
Subject: [Python-3000] PEP 3108: Standard Library Reorganization
In-Reply-To: <6832200B-6A66-4E3D-88C1-E7ADF326CD2E@python.org>
References: <20070101174925.82FB.JCARLSON@uci.edu>
	<bbaeab100701011944n7b73a33aq68b3ac0f50b7b8f3@mail.gmail.com>
	<20070101225107.8315.JCARLSON@uci.edu>
	<bbaeab100701021547l2eddd412wc11330c4cfcb46dd@mail.gmail.com>
	<6832200B-6A66-4E3D-88C1-E7ADF326CD2E@python.org>
Message-ID: <bbaeab100701021652i616fa41fwac312e5bae90b122@mail.gmail.com>

On 1/2/07, Barry Warsaw <barry at python.org> wrote:
>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On Jan 2, 2007, at 6:47 PM, Brett Cannon wrote:
>
> > Are you at least okay with base64, quopri, and uu going?  You are just
> > arguing for the saving of binascii, right?
> >
> > Does anyone else care to try to save binascii?
>
> base64 and quopri implement specific RFCs so I think they should
> stay.  uu implements a defacto standard, but I don't like its
> interface (it uses file-like objects instead of strings).  For Py3K,
> I would be in favor of consolidating things, making interfaces
> consistent, and probably re-homing the modules under a top-level
> package.


But isn't that what the codecs module is for?  If cleaning up the API is
going to happen shouldn't be to add whatever encoders/decoders that codecs
is missing to replace these modules?

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20070102/dd018b7c/attachment.html 

From janssen at parc.com  Wed Jan  3 02:00:49 2007
From: janssen at parc.com (Bill Janssen)
Date: Tue, 2 Jan 2007 17:00:49 PST
Subject: [Python-3000] PEP 3108: Standard Library Reorganization
In-Reply-To: <5F1ED8EA-061B-499E-B052-2924E190A903@python.org> 
References: <bbaeab100701011614l27a87f45pb7e782c1d78a0725@mail.gmail.com>
	<encabb$e5s$1@sea.gmane.org>
	<200701021447.01263.anthony@interlink.com.au>
	<5F1ED8EA-061B-499E-B052-2924E190A903@python.org>
Message-ID: <07Jan2.170057pst."58648"@synergy1.parc.xerox.com>

Barry Warsaw writes:
> To me, a standard library reorganization would proceed like this:  
> figure out the new hierarchy and package names now and implement them  
> for Python 2.6.  Make sure every old package name continues to work,  
> but encourage people (through documentation and social coercion, not  
> through warnings) to start using the new hierarchy.  Issue no  
> warnings in Python 2.6 unless the magical -3 flag were given, but  
> never make the old hierarchy issue errors in Python 2.x.

I second Barry's recommendation.  I too would like to see a phase-in
approach, with both set of module names available side-by-side for one
or two releases (certainly for Python 3.0), and a switch to set (on by
default in 3.x, off by default in 2.x) which causes deprecation
warnings for use of the "old" names.

Bill



From brett at python.org  Wed Jan  3 02:02:22 2007
From: brett at python.org (Brett Cannon)
Date: Tue, 2 Jan 2007 17:02:22 -0800
Subject: [Python-3000] PEP 3108: Standard Library Reorganization
In-Reply-To: <4987521444494646097@unknownmsgid>
References: <bbaeab100701011614l27a87f45pb7e782c1d78a0725@mail.gmail.com>
	<4987521444494646097@unknownmsgid>
Message-ID: <bbaeab100701021702q3a33909vd029c0640a8c9d99@mail.gmail.com>

On 1/2/07, Bill Janssen <janssen at parc.com> wrote:
>
> Brett,
>
> Thanks for writing this up.


Welcome!  Sometimes I worry about my penchant for Python masochism.

Here are my thoughts on this (after 3 weeks away, I'm still catching up):
>
> The link to PEP 4 is broken.


Thanks, fixed.

"nis" -- NIS is still widely used by many people; probably premature to
> remove.


Yuck, really?  Anyone else agree with this?

"md5" and "sha" -- should note pointer to "hashlib".


Eh, I am not going to worry about it since it is listed because of an
existing deprecation so someone has already explained it for me.  =)

"base64"/"quopri"/"uu" -- still quite useful, so I'd suggest removing
> them but retaining "binascii", with a new implementation that uses the
> "codecs" support directly.


Seems people want to hold on to either binascii or the trio of modules, but
I need to understand why we shouldn't just rework them so that they are not
public and have people use the codecs module.

Thank God "asyncore" was 'saved'.


=)

I hate the renaming ideas, but I agree it's probably necessary.


Yep.  From now on at least people can come up with better names.  If people
have better names

Let me suggest a consolidation idea I don't see there:  a grouping
> similar to that which has been done with email, but called "web".
> This package would include "urllib", "urllib2", "urlparse", "httplib",
> "cgi", "BaseHTTPServer", "BaseCGIServer", "HTMLParser", "Cookie",
> "cookielib", and "SimpleHTTPServer" (at least).



Done.

  The Python ECMAScript
> interpreter could go there, too, when it's written :-).



Ugh, not if I can help it.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20070102/94853fb3/attachment-0001.htm 

From janssen at parc.com  Wed Jan  3 02:05:53 2007
From: janssen at parc.com (Bill Janssen)
Date: Tue, 2 Jan 2007 17:05:53 PST
Subject: [Python-3000] PEP 3108: Standard Library Reorganization
In-Reply-To: <bbaeab100701021547l2eddd412wc11330c4cfcb46dd@mail.gmail.com> 
References: <20070101174925.82FB.JCARLSON@uci.edu>
	<bbaeab100701011944n7b73a33aq68b3ac0f50b7b8f3@mail.gmail.com>
	<20070101225107.8315.JCARLSON@uci.edu>
	<bbaeab100701021547l2eddd412wc11330c4cfcb46dd@mail.gmail.com>
Message-ID: <07Jan2.170602pst."58648"@synergy1.parc.xerox.com>

> Perhaps, but consistency is more important (at least to me).  Best bet would
> be to put them in a package.
> 
> > > * Servers
> > > > >     + BaseHTTPServer
> > > > >     + CGIHTTPServer
> > > > >     + DocXMLRPCServer
> > > > >     + SimpleHTTPServer
> > > > >     + SimpleXMLRPCServer
> > > > >     + SocketServer

Hmmm, "servers" is a somewhat pointless common facet, though.

I'd put SocketServer into the "socket" module; the two XML-RPC servers
into an "xmlrpc" package, and the others into a "web/servers" package.

Bill

From krstic at solarsail.hcs.harvard.edu  Wed Jan  3 02:14:59 2007
From: krstic at solarsail.hcs.harvard.edu (=?UTF-8?B?SXZhbiBLcnN0acSH?=)
Date: Wed, 03 Jan 2007 02:14:59 +0100
Subject: [Python-3000] PEP 3108: Standard Library Reorganization
In-Reply-To: <bbaeab100701021702q3a33909vd029c0640a8c9d99@mail.gmail.com>
References: <bbaeab100701011614l27a87f45pb7e782c1d78a0725@mail.gmail.com>	<4987521444494646097@unknownmsgid>
	<bbaeab100701021702q3a33909vd029c0640a8c9d99@mail.gmail.com>
Message-ID: <459B0393.6010501@solarsail.hcs.harvard.edu>

Brett Cannon wrote:
>     "nis" -- NIS is still widely used by many people; probably premature
>     to remove.
> 
> Yuck, really?  Anyone else agree with this?

Yes, absolutely; Bill beat me to the punch. I know of new deployments
going with NIS instead of LDAP, even.

-- 
Ivan Krsti? <krstic at solarsail.hcs.harvard.edu> | GPG: 0x147C722D

From jcarlson at uci.edu  Wed Jan  3 02:46:12 2007
From: jcarlson at uci.edu (Josiah Carlson)
Date: Tue, 02 Jan 2007 17:46:12 -0800
Subject: [Python-3000] PEP 3108: Standard Library Reorganization
In-Reply-To: <bbaeab100701021547l2eddd412wc11330c4cfcb46dd@mail.gmail.com>
References: <20070101225107.8315.JCARLSON@uci.edu>
	<bbaeab100701021547l2eddd412wc11330c4cfcb46dd@mail.gmail.com>
Message-ID: <20070102165914.8CC4.JCARLSON@uci.edu>


"Brett Cannon" <brett at python.org> wrote:
> On 1/1/07, Josiah Carlson <jcarlson at uci.edu> wrote:
> > I would guess that Raymond Hettinger would have something to say about
> > this (being that he was going to add the long/int <-> binary conversion
> > to binascii). I don't know if he has been paying attention and/or
> > reading py3k.
> 
> Raymond might, but I don't think he is following Py3K.  But MAL kicked the
> PEP over to python-dev so he might notice now.
> 
> Are you at least okay with base64, quopri, and uu going?  You are just
> arguing for the saving of binascii, right?

More or less.  One concern I have about switching to a codecs-only
approach is that one will not be able to use bytes.encode('base64'), as
per previous discussions on the topic (that I may be remembering
incorrectly), only bytes.decode(...) will be available (think in terms
of unicode text encoding and decoding). One would need to perform
bytes.decode('latin1').encode('base64'). Using the base64 module
bypasses such round trip foolishness.  I believe the same holds for
quopri and uu.


> > If asyncore/asynchat/smtpd needs a maintainer, then I volunteer.  Assign
> > all bugs, patches, etc., to me and I'll handle them.
> 
> That's good enough for me.  They are saved.

Someone needs to add me to the project on SF.  Or we can wait until the
new tracker is up.  I don't see anyone rushing to patch the async
modules.


> > I'm not voting for their removal, just stating that they are trickly to
> > handle, both in a naming, and in an understanding perspective.  Honestly,
> > which is easier to read and/or understand to you: basehttpserver or
> > BaseHTTPServer?  I prefer CamelCase.
> 
> Perhaps, but consistency is more important (at least to me).  Best bet would
> be to put them in a package.

I agree.


> > xml.etree.ElementTree could be used as an example of what could/should
> > or shouldn't be done.
> 
> It is its own special open issue for that reason.  =)  I personally would
> prefer they go into a package, we ditch the CamelCase and move to
> underscores for them.

Either is fine with me.  Sticking with CamelCase makes the renaming
easier.


 - Josiah


From barry at python.org  Wed Jan  3 03:51:17 2007
From: barry at python.org (Barry Warsaw)
Date: Tue, 2 Jan 2007 21:51:17 -0500
Subject: [Python-3000] PEP 3108: Standard Library Reorganization
In-Reply-To: <bbaeab100701021652i616fa41fwac312e5bae90b122@mail.gmail.com>
References: <20070101174925.82FB.JCARLSON@uci.edu>
	<bbaeab100701011944n7b73a33aq68b3ac0f50b7b8f3@mail.gmail.com>
	<20070101225107.8315.JCARLSON@uci.edu>
	<bbaeab100701021547l2eddd412wc11330c4cfcb46dd@mail.gmail.com>
	<6832200B-6A66-4E3D-88C1-E7ADF326CD2E@python.org>
	<bbaeab100701021652i616fa41fwac312e5bae90b122@mail.gmail.com>
Message-ID: <DE13AEBD-715D-49DD-93D9-0CC786A05C85@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Jan 2, 2007, at 7:52 PM, Brett Cannon wrote:

>> base64 and quopri implement specific RFCs so I think they should
>> stay.  uu implements a defacto standard, but I don't like its
>> interface (it uses file-like objects instead of strings).  For Py3K,
>> I would be in favor of consolidating things, making interfaces
>> consistent, and probably re-homing the modules under a top-level
>> package.
>
> But isn't that what the codecs module is for?  If cleaning up the  
> API is
> going to happen shouldn't be to add whatever encoders/decoders that  
> codecs
> is missing to replace these modules?

IMO, no.  The "problem" with the codecs is that it doesn't seem like  
a natural interface for doing these kinds of conversions.  One  
problem is documentation: if I want to base64 encode some bytes with  
an alternate alphabet and the only hammer I've got is the codecs  
module, how do I find out about this hammer and where do I go for  
(specifically the base64 semantics) documentation?  Or let's say I  
want to base32 encode some byte but I want to fold the case and map 0  
and 1 to an alternative alphabet.  How do I pass that into the  
codec?  Finally, the codecs tend to take strings as arguments so if I  
misspell "base63" I'm going to get a lookup error which is much more  
confusing than the name or attribute error ("hmm, maybe my distro  
didn't package up all the codecs").

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRZsaN3EjvBPtnXfVAQKhBwP9FwraAd5XueoD7pTTAiZsLpSp8HjPAXI6
NGFrI4IE7gTVvBK1+TlAh/MySJl3oUFro1/m5q93r8yzrCm1KODavj0QDdYm+loV
/gsVOTTD4D46YBQf06MeUaTk1Xe7eUTGPp4UMPf/iMMjnSnXYx4CYN3PJ8cZPSKp
d6X3T0N7dPs=
=NB2+
-----END PGP SIGNATURE-----

From anthony at interlink.com.au  Wed Jan  3 04:53:16 2007
From: anthony at interlink.com.au (Anthony Baxter)
Date: Wed, 3 Jan 2007 14:53:16 +1100
Subject: [Python-3000] PEP 3108: Standard Library Reorganization
In-Reply-To: <bbaeab100701021547l2eddd412wc11330c4cfcb46dd@mail.gmail.com>
References: <20070101174925.82FB.JCARLSON@uci.edu>
	<20070101225107.8315.JCARLSON@uci.edu>
	<bbaeab100701021547l2eddd412wc11330c4cfcb46dd@mail.gmail.com>
Message-ID: <200701031453.17348.anthony@interlink.com.au>

On Wednesday 03 January 2007 10:47, Brett Cannon wrote:
> > > > * base64/quopri/uu
> > > >
> > > > >     + Support exists in the codecs module.
> > > > >     + If removed (along with binhex), also remove
> > > > > binascii. - C implementation of base64, binhex, and uu
> > > > > modules.
> > > >
> Are you at least okay with base64, quopri, and uu going?  You are
> just arguing for the saving of binascii, right?
>
> Does anyone else care to try to save binascii?

I am deeply uncomfortable with base64 going. This is used by a _lot_ 
of code, and the alternative (using the codecs module) is not at 
all well publicised. For instance, the docs for the base64 module 
don't even mention that you can use the codecs module as an 
alternative. I just looked up how to do it, and it provides a 
different, and to my eyes, less pleasant API. For instance these 
two:

base64.encodestring('hello world')
or 
base64.b64encode('hello world')

become
codecs.getencoder('base64')('hello')[0]

Additionally, base32 and base16 are not supported by codecs, 
according to the docs, and neither is the ability to specify 
alternate character mappings (I don't know how heavily used the 
last is, though).

I think any removal should take into account how heavily the module 
is used, as well as how aggressively we've previously pushed people 
to an alternative. For base64, the relevant answers are "quite 
heavily used"[1] and "not at all"[2]. 

For those reasons, I'm a strong -1 on removing base64. I don't have 
a problem with it being reimplemented on top of the codecs module, 
so long as that doesn't lead to any serious performance slowdowns.
A couple of quick tests with timeit seems to show that the two are 
approximately equal in performance, but more testing is needed.

[1] google for 'filetype:py base64'
[2] I don't recall it ever being discussed before now

Anthony
-- 
Anthony Baxter     <anthony at interlink.com.au>
It's never too late to have a happy childhood.

From barry at python.org  Wed Jan  3 04:54:10 2007
From: barry at python.org (Barry Warsaw)
Date: Tue, 2 Jan 2007 22:54:10 -0500
Subject: [Python-3000] PEP 3108: Standard Library Reorganization
In-Reply-To: <07Jan2.165122pst."58648"@synergy1.parc.xerox.com>
References: <bbaeab100701011614l27a87f45pb7e782c1d78a0725@mail.gmail.com>
	<07Jan2.165122pst."58648"@synergy1.parc.xerox.com>
Message-ID: <2A49056F-5851-4A4E-8391-576A60E68E64@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Jan 2, 2007, at 7:51 PM, Bill Janssen wrote:

> Let me suggest a consolidation idea I don't see there:  a grouping
> similar to that which has been done with email, but called "web".
> This package would include "urllib", "urllib2", "urlparse", "httplib",
> "cgi", "BaseHTTPServer", "BaseCGIServer", "HTMLParser", "Cookie",
> "cookielib", and "SimpleHTTPServer" (at least).  The Python ECMAScript
> interpreter could go there, too, when it's written :-).

+1
- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRZso4nEjvBPtnXfVAQL90gP/SDFFMA0Cbz+cnSAa+FoA89LKg9nLngC9
yEblSI9CEbpokKBPYiLlvM3D/TjPc6LZC3QlYMsiDY8zhRxFtVff0jOA5d6Fh1G9
TJTWSoFpxOTLpNb1UGDPd+0pyvlVGSG+oLmA0x5shjUJRril7PXG1BLrDkRwK24+
q12Gnti4e3Q=
=pg9t
-----END PGP SIGNATURE-----

From mike.klaas at gmail.com  Wed Jan  3 05:06:07 2007
From: mike.klaas at gmail.com (Mike Klaas)
Date: Tue, 2 Jan 2007 20:06:07 -0800
Subject: [Python-3000] PEP 3108: Standard Library Reorganization
In-Reply-To: <200701031453.17348.anthony@interlink.com.au>
References: <20070101174925.82FB.JCARLSON@uci.edu>
	<20070101225107.8315.JCARLSON@uci.edu>
	<bbaeab100701021547l2eddd412wc11330c4cfcb46dd@mail.gmail.com>
	<200701031453.17348.anthony@interlink.com.au>
Message-ID: <3d2ce8cb0701022006k7444f7a8t2a715a7a3b17e9e9@mail.gmail.com>

On 1/2/07, Anthony Baxter <anthony at interlink.com.au> wrote:

> Additionally, base32 and base16 are not supported by codecs,
> according to the docs, and neither is the ability to specify
> alternate character mappings (I don't know how heavily used the
> last is, though).

We use the "urlsafe" version heavily, but I've never needed to use any
other mapping.

-Mike

From barry at python.org  Wed Jan  3 05:37:49 2007
From: barry at python.org (Barry Warsaw)
Date: Tue, 2 Jan 2007 23:37:49 -0500
Subject: [Python-3000] PEP 3108: Standard Library Reorganization
In-Reply-To: <200701031453.17348.anthony@interlink.com.au>
References: <20070101174925.82FB.JCARLSON@uci.edu>
	<20070101225107.8315.JCARLSON@uci.edu>
	<bbaeab100701021547l2eddd412wc11330c4cfcb46dd@mail.gmail.com>
	<200701031453.17348.anthony@interlink.com.au>
Message-ID: <6319D387-3A48-41A4-BD51-979DDE8C585E@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Jan 2, 2007, at 10:53 PM, Anthony Baxter wrote:

> Additionally, base32 and base16 are not supported by codecs,
> according to the docs, and neither is the ability to specify
> alternate character mappings (I don't know how heavily used the
> last is, though).

Which reminds me of another problem I have with the codecs module.   
codecs are a potential security issue because they are a backdoor way  
to get modules imported.  For example, if I get an email with a  
specified charset, the natural thing is to want to use .decode() to  
turn that into a unicode.  The problem is that Python can be  
essentially tricked into importing any module that way.  Try this:

Python 2.4.3 (#1, Jun 12 2006, 19:42:21)
[GCC 4.0.1 (Apple Computer, Inc. build 5341)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
 >>> import sys
 >>> sys.modules['smtplib']
Traceback (most recent call last):
   File "<stdin>", line 1, in ?
KeyError: 'smtplib'
 >>> 'foo'.decode('smtplib')
Traceback (most recent call last):
   File "<stdin>", line 1, in ?
LookupError: unknown encoding: smtplib
 >>> sys.modules['smtplib']
<module 'smtplib' from '/opt/local/Library/Frameworks/ 
Python.framework/Versions/2.4/lib/python2.4/smtplib.pyc'>
 >>>

Okay, so this doesn't open any more holes than are already opened,  
but I worry that the style of encouraging codec use will make such  
potential holes more widespread.  The problem is that it's very  
difficult to code defensively around this because you can't really  
whitelist or blacklist the set of valid codecs, and you can't  
feasibly audit every importable module to see if it has nasty import  
side effects.

It would help if codec lookup wasn't import based, or somehow any  
import side-effects could be isolated.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRZszHXEjvBPtnXfVAQLzPwQAg8C3nTDIQPwomlj35zfxPmlvvSP13KCs
sR2Jl+d+JhSwUwle0zWRlGRz3eAj8Rctt/VAfiAVDF91heGv6xWIbLgnq/BzJdSr
5nHrXcvQAuyu8xYQHYbtrKiCchO25mPRqXOrYcLSM7misJ8635qrWNU/23fiaNmw
aJos4kk6o6c=
=rWh+
-----END PGP SIGNATURE-----

From qrczak at knm.org.pl  Wed Jan  3 06:17:26 2007
From: qrczak at knm.org.pl (Marcin 'Qrczak' Kowalczyk)
Date: Wed, 03 Jan 2007 06:17:26 +0100
Subject: [Python-3000] What's the point of annotations?
In-Reply-To: <43aa6ff70701021008y29a086a9i52294fd3b193f97a@mail.gmail.com>
	(Collin Winter's message of "Tue, 2 Jan 2007 12:08:56 -0600")
References: <43aa6ff70701021008y29a086a9i52294fd3b193f97a@mail.gmail.com>
Message-ID: <87vejo4rkp.fsf@qrnik.zagroda>

"Collin Winter" <collinw at gmail.com> writes:

> I was explaining function annotations to a friend this past weekend
> and found that, even though I had written the PEP and spent months
> debating the little details of how annotations were to work, I was
> hard-pressed to answer the question of "why are we doing this?"

It's a limited way to extend the syntax, without actually admitting
to do that :-)

-- 
   __("<         Marcin Kowalczyk
   \__/       qrczak at knm.org.pl
    ^^     http://qrnik.knm.org.pl/~qrczak/

From talin at acm.org  Wed Jan  3 08:29:27 2007
From: talin at acm.org (Talin)
Date: Tue, 02 Jan 2007 23:29:27 -0800
Subject: [Python-3000] PEP 3108: Standard Library Reorganization
In-Reply-To: <bbaeab100701011614l27a87f45pb7e782c1d78a0725@mail.gmail.com>
References: <bbaeab100701011614l27a87f45pb7e782c1d78a0725@mail.gmail.com>
Message-ID: <459B5B57.2000002@acm.org>

Just a quick comment on the discussion in general: It seems that there 
are a number of modules which fall in to the set of "widely used, but 
obscure and/or unmaintained". Perhaps what is needed is to annotate the 
list of modules to be dropped with a set of prerequisite conditions - 
i.e. what has to happen before a particular module can be removed.

It also seems to me that a few of these items don't need to be dropped 
so much as refactored - so that Py3K will have the same functionality, 
but with a cleaner and more orthogonal interface.

-- Talin


From brett at python.org  Wed Jan  3 08:39:03 2007
From: brett at python.org (Brett Cannon)
Date: Tue, 2 Jan 2007 23:39:03 -0800
Subject: [Python-3000] PEP 3108: Standard Library Reorganization
In-Reply-To: <459B0393.6010501@solarsail.hcs.harvard.edu>
References: <bbaeab100701011614l27a87f45pb7e782c1d78a0725@mail.gmail.com>
	<4987521444494646097@unknownmsgid>
	<bbaeab100701021702q3a33909vd029c0640a8c9d99@mail.gmail.com>
	<459B0393.6010501@solarsail.hcs.harvard.edu>
Message-ID: <bbaeab100701022339p65c4300dp7f1eb5573b0d34d2@mail.gmail.com>

On 1/2/07, Ivan Krsti? <krstic at solarsail.hcs.harvard.edu> wrote:
>
> Brett Cannon wrote:
> >     "nis" -- NIS is still widely used by many people; probably premature
> >     to remove.
> >
> > Yuck, really?  Anyone else agree with this?
>
> Yes, absolutely; Bill beat me to the punch. I know of new deployments
> going with NIS instead of LDAP, even.



OK, it's saved.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20070102/6578d130/attachment.htm 

From brett at python.org  Wed Jan  3 08:47:07 2007
From: brett at python.org (Brett Cannon)
Date: Tue, 2 Jan 2007 23:47:07 -0800
Subject: [Python-3000] PEP 3108: Standard Library Reorganization
In-Reply-To: <20070102165914.8CC4.JCARLSON@uci.edu>
References: <20070101225107.8315.JCARLSON@uci.edu>
	<bbaeab100701021547l2eddd412wc11330c4cfcb46dd@mail.gmail.com>
	<20070102165914.8CC4.JCARLSON@uci.edu>
Message-ID: <bbaeab100701022347k23f3750ej117de625c9daaa57@mail.gmail.com>

On 1/2/07, Josiah Carlson <jcarlson at uci.edu> wrote:
>
>
> "Brett Cannon" <brett at python.org> wrote:
> > On 1/1/07, Josiah Carlson <jcarlson at uci.edu> wrote:
> > > I would guess that Raymond Hettinger would have something to say about
> > > this (being that he was going to add the long/int <-> binary
> conversion
> > > to binascii). I don't know if he has been paying attention and/or
> > > reading py3k.
> >
> > Raymond might, but I don't think he is following Py3K.  But MAL kicked
> the
> > PEP over to python-dev so he might notice now.
> >
> > Are you at least okay with base64, quopri, and uu going?  You are just
> > arguing for the saving of binascii, right?
>
> More or less.  One concern I have about switching to a codecs-only
> approach is that one will not be able to use bytes.encode('base64'), as
> per previous discussions on the topic (that I may be remembering
> incorrectly), only bytes.decode(...) will be available (think in terms
> of unicode text encoding and decoding). One would need to perform
> bytes.decode('latin1').encode('base64'). Using the base64 module
> bypasses such round trip foolishness.  I believe the same holds for
> quopri and uu.


I don't remember the whole bytes discussion in terms of encoding and
decoding.

> > If asyncore/asynchat/smtpd needs a maintainer, then I volunteer.  Assign
> > > all bugs, patches, etc., to me and I'll handle them.
> >
> > That's good enough for me.  They are saved.
>
> Someone needs to add me to the project on SF.  Or we can wait until the
> new tracker is up.  I don't see anyone rushing to patch the async
> modules.


I can't add you myself; someone else will need to do it (they will need your
SF ID).

> > I'm not voting for their removal, just stating that they are trickly to
> > > handle, both in a naming, and in an understanding
> perspective.  Honestly,
> > > which is easier to read and/or understand to you: basehttpserver or
> > > BaseHTTPServer?  I prefer CamelCase.
> >
> > Perhaps, but consistency is more important (at least to me).  Best bet
> would
> > be to put them in a package.
>
> I agree.


I will probably do a separate email to get support or not for package
suggestions once the list of modules to  remove is settled upon.

> > xml.etree.ElementTree could be used as an example of what could/should
> > > or shouldn't be done.
> >
> > It is its own special open issue for that reason.  =)  I personally
> would
> > prefer they go into a package, we ditch the CamelCase and move to
> > underscores for them.
>
> Either is fine with me.  Sticking with CamelCase makes the renaming
> easier.


Not really because if they move to a package they will most likely have to
go from ``import BaseHTTPServer`` to ``from web import BaseHTTPServer``, at
which point we can do ``from web import base_http_server as BaseHTTPServer``
just as easily in the refactoring tool.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20070102/09eb9b38/attachment.html 

From brett at python.org  Wed Jan  3 08:57:42 2007
From: brett at python.org (Brett Cannon)
Date: Tue, 2 Jan 2007 23:57:42 -0800
Subject: [Python-3000] PEP 3108: Standard Library Reorganization
In-Reply-To: <200701031453.17348.anthony@interlink.com.au>
References: <20070101174925.82FB.JCARLSON@uci.edu>
	<20070101225107.8315.JCARLSON@uci.edu>
	<bbaeab100701021547l2eddd412wc11330c4cfcb46dd@mail.gmail.com>
	<200701031453.17348.anthony@interlink.com.au>
Message-ID: <bbaeab100701022357q12de3b43u825e07639a9d594c@mail.gmail.com>

On 1/2/07, Anthony Baxter <anthony at interlink.com.au> wrote:
>
> On Wednesday 03 January 2007 10:47, Brett Cannon wrote:
> > > > > * base64/quopri/uu
> > > > >
> > > > > >     + Support exists in the codecs module.
> > > > > >     + If removed (along with binhex), also remove
> > > > > > binascii. - C implementation of base64, binhex, and uu
> > > > > > modules.
> > > > >
> > Are you at least okay with base64, quopri, and uu going?  You are
> > just arguing for the saving of binascii, right?
> >
> > Does anyone else care to try to save binascii?
>
> I am deeply uncomfortable with base64 going. This is used by a _lot_
> of code, and the alternative (using the codecs module) is not at
> all well publicised. For instance, the docs for the base64 module
> don't even mention that you can use the codecs module as an
> alternative. I just looked up how to do it, and it provides a
> different, and to my eyes, less pleasant API. For instance these
> two:
>
> base64.encodestring('hello world')
> or
> base64.b64encode('hello world')
>
> become
> codecs.getencoder('base64')('hello')[0]


It is starting to sound like the codecs module needs a little work.

Additionally, base32 and base16 are not supported by codecs,
> according to the docs, and neither is the ability to specify
> alternate character mappings (I don't know how heavily used the
> last is, though).


Yeah, I noticed the base32 and base16 codecs were missing.

I think any removal should take into account how heavily the module
> is used, as well as how aggressively we've previously pushed people
> to an alternative.


Fine with me, but I need help figuring out which modules are used heavily
(which, luckily, people are doing).

base64 seems to have quite the following, so I will take it off the list.  I
assume then that means quopri and uu should be saved as well?  And I am
assuming binascii comes along with the package (although alternative idea is
below).

For base64, the relevant answers are "quite
> heavily used"[1] and "not at all"[2].


It seems that the codecs module might need a cleanup in terms of API or
something.  Or at least have people be more aware of it.

For those reasons, I'm a strong -1 on removing base64. I don't have
> a problem with it being reimplemented on top of the codecs module,
> so long as that doesn't lead to any serious performance slowdowns.
> A couple of quick tests with timeit seems to show that the two are
> approximately equal in performance, but more testing is needed.


That's an idea if someone is up for doing that.  Could also hide binascii as
_binascii and have base64, quopri, and uu use it then to follow what we are
doing with cStringIO/StringIO et. al.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20070102/64b0456b/attachment.htm 

From brett at python.org  Wed Jan  3 09:00:08 2007
From: brett at python.org (Brett Cannon)
Date: Wed, 3 Jan 2007 00:00:08 -0800
Subject: [Python-3000] PEP 3108: Standard Library Reorganization
In-Reply-To: <DE13AEBD-715D-49DD-93D9-0CC786A05C85@python.org>
References: <20070101174925.82FB.JCARLSON@uci.edu>
	<bbaeab100701011944n7b73a33aq68b3ac0f50b7b8f3@mail.gmail.com>
	<20070101225107.8315.JCARLSON@uci.edu>
	<bbaeab100701021547l2eddd412wc11330c4cfcb46dd@mail.gmail.com>
	<6832200B-6A66-4E3D-88C1-E7ADF326CD2E@python.org>
	<bbaeab100701021652i616fa41fwac312e5bae90b122@mail.gmail.com>
	<DE13AEBD-715D-49DD-93D9-0CC786A05C85@python.org>
Message-ID: <bbaeab100701030000g637de811rc262ae531f971249@mail.gmail.com>

On 1/2/07, Barry Warsaw <barry at python.org> wrote:
>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On Jan 2, 2007, at 7:52 PM, Brett Cannon wrote:
>
> >> base64 and quopri implement specific RFCs so I think they should
> >> stay.  uu implements a defacto standard, but I don't like its
> >> interface (it uses file-like objects instead of strings).  For Py3K,
> >> I would be in favor of consolidating things, making interfaces
> >> consistent, and probably re-homing the modules under a top-level
> >> package.
> >
> > But isn't that what the codecs module is for?  If cleaning up the
> > API is
> > going to happen shouldn't be to add whatever encoders/decoders that
> > codecs
> > is missing to replace these modules?
>
> IMO, no.  The "problem" with the codecs is that it doesn't seem like
> a natural interface for doing these kinds of conversions.  One
> problem is documentation: if I want to base64 encode some bytes with
> an alternate alphabet and the only hammer I've got is the codecs
> module, how do I find out about this hammer and where do I go for
> (specifically the base64 semantics) documentation?  Or let's say I
> want to base32 encode some byte but I want to fold the case and map 0
> and 1 to an alternative alphabet.  How do I pass that into the
> codec?  Finally, the codecs tend to take strings as arguments so if I
> misspell "base63" I'm going to get a lookup error which is much more
> confusing than the name or attribute error ("hmm, maybe my distro
> didn't package up all the codecs").



It sounds like codecs needs a little lovin'.  The thing has been around
since Python 2.0 which is before my time on python-dev so I don't know
anything about its design.  But I would assume it is going to get used a lot
more come Py3K for Unicode conversion.  Maybe some of your and Anthony's
issues can be addressed so that at least some point in the future we can
transition people over to a common setup for encodings and off of base64 and
such (but it won't be Python 3.0).

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20070103/3cf605b7/attachment.html 

From brett at python.org  Wed Jan  3 09:02:39 2007
From: brett at python.org (Brett Cannon)
Date: Wed, 3 Jan 2007 00:02:39 -0800
Subject: [Python-3000] PEP 3108: Standard Library Reorganization
In-Reply-To: <6319D387-3A48-41A4-BD51-979DDE8C585E@python.org>
References: <20070101174925.82FB.JCARLSON@uci.edu>
	<20070101225107.8315.JCARLSON@uci.edu>
	<bbaeab100701021547l2eddd412wc11330c4cfcb46dd@mail.gmail.com>
	<200701031453.17348.anthony@interlink.com.au>
	<6319D387-3A48-41A4-BD51-979DDE8C585E@python.org>
Message-ID: <bbaeab100701030002t25b59a77wdc1ba61954c6ed3d@mail.gmail.com>

On 1/2/07, Barry Warsaw <barry at python.org> wrote:
>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On Jan 2, 2007, at 10:53 PM, Anthony Baxter wrote:
>
> > Additionally, base32 and base16 are not supported by codecs,
> > according to the docs, and neither is the ability to specify
> > alternate character mappings (I don't know how heavily used the
> > last is, though).
>
> Which reminds me of another problem I have with the codecs module.
> codecs are a potential security issue because they are a backdoor way
> to get modules imported.  For example, if I get an email with a
> specified charset, the natural thing is to want to use .decode() to
> turn that into a unicode.  The problem is that Python can be
> essentially tricked into importing any module that way.  Try this:
>
> Python 2.4.3 (#1, Jun 12 2006, 19:42:21)
> [GCC 4.0.1 (Apple Computer, Inc. build 5341)] on darwin
> Type "help", "copyright", "credits" or "license" for more information.
> >>> import sys
> >>> sys.modules['smtplib']
> Traceback (most recent call last):
>    File "<stdin>", line 1, in ?
> KeyError: 'smtplib'
> >>> 'foo'.decode('smtplib')
> Traceback (most recent call last):
>    File "<stdin>", line 1, in ?
> LookupError: unknown encoding: smtplib
> >>> sys.modules['smtplib']
> <module 'smtplib' from '/opt/local/Library/Frameworks/
> Python.framework/Versions/2.4/lib/python2.4/smtplib.pyc'>
> >>>




Okay, so this doesn't open any more holes than are already opened,
> but I worry that the style of encouraging codec use will make such
> potential holes more widespread.  The problem is that it's very
> difficult to code defensively around this because you can't really
> whitelist or blacklist the set of valid codecs, and you can't
> feasibly audit every importable module to see if it has nasty import
> side effects.
>
> It would help if codec lookup wasn't import based, or somehow any
> import side-effects could be isolated.


I bet using an absolute import on the part of the codecs module to specify
the encodings package would deal with a lot of your worries.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20070103/bc27ba03/attachment.htm 

From brett at python.org  Wed Jan  3 09:04:30 2007
From: brett at python.org (Brett Cannon)
Date: Wed, 3 Jan 2007 00:04:30 -0800
Subject: [Python-3000] PEP 3108: Standard Library Reorganization
In-Reply-To: <2A49056F-5851-4A4E-8391-576A60E68E64@python.org>
References: <bbaeab100701011614l27a87f45pb7e782c1d78a0725@mail.gmail.com>
	<2A49056F-5851-4A4E-8391-576A60E68E64@python.org>
Message-ID: <bbaeab100701030004h60c97683kd5912c0e7093de43@mail.gmail.com>

On 1/2/07, Barry Warsaw <barry at python.org> wrote:
>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On Jan 2, 2007, at 7:51 PM, Bill Janssen wrote:
>
> > Let me suggest a consolidation idea I don't see there:  a grouping
> > similar to that which has been done with email, but called "web".
> > This package would include "urllib", "urllib2", "urlparse", "httplib",
> > "cgi", "BaseHTTPServer", "BaseCGIServer", "HTMLParser", "Cookie",
> > "cookielib", and "SimpleHTTPServer" (at least).  The Python ECMAScript
> > interpreter could go there, too, when it's written :-).
>
> +1


I just want to let everyone know that I will start a separate thread on
possible new packages (or solidation into a package in the case of
'collections') after this whole discussion of what modules should not be
removed has settled down and people are happy with the list.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20070103/54094308/attachment.html 

From p.f.moore at gmail.com  Wed Jan  3 10:56:29 2007
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 3 Jan 2007 09:56:29 +0000
Subject: [Python-3000] PEP 3108: Standard Library Reorganization
In-Reply-To: <6832200B-6A66-4E3D-88C1-E7ADF326CD2E@python.org>
References: <20070101174925.82FB.JCARLSON@uci.edu>
	<bbaeab100701011944n7b73a33aq68b3ac0f50b7b8f3@mail.gmail.com>
	<20070101225107.8315.JCARLSON@uci.edu>
	<bbaeab100701021547l2eddd412wc11330c4cfcb46dd@mail.gmail.com>
	<6832200B-6A66-4E3D-88C1-E7ADF326CD2E@python.org>
Message-ID: <79990c6b0701030156l36c36f53ub221a4842983fb05@mail.gmail.com>

On 1/3/07, Barry Warsaw <barry at python.org> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On Jan 2, 2007, at 6:47 PM, Brett Cannon wrote:
>
> > Are you at least okay with base64, quopri, and uu going?  You are just
> > arguing for the saving of binascii, right?
> >
> > Does anyone else care to try to save binascii?
>
> base64 and quopri implement specific RFCs so I think they should
> stay.  uu implements a defacto standard, but I don't like its
> interface (it uses file-like objects instead of strings).  For Py3K,
> I would be in favor of consolidating things, making interfaces
> consistent, and probably re-homing the modules under a top-level
> package.

Agreed. Although I very rarely (if ever) use these modules, my
immediate reaction when I saw these was "OK, so if I needed to do
this, how would I?" The answer may be to use encodings, but I'm not
sure. At the very least, a note along the lines of

    uu.encode/decode - replace with str.{en/de}code("uu")

would clarify that while the module is going, the functionality isn't.

Paul

From barry at python.org  Wed Jan  3 14:05:51 2007
From: barry at python.org (Barry Warsaw)
Date: Wed, 3 Jan 2007 08:05:51 -0500
Subject: [Python-3000] PEP 3108: Standard Library Reorganization
In-Reply-To: <bbaeab100701022357q12de3b43u825e07639a9d594c@mail.gmail.com>
References: <20070101174925.82FB.JCARLSON@uci.edu>
	<20070101225107.8315.JCARLSON@uci.edu>
	<bbaeab100701021547l2eddd412wc11330c4cfcb46dd@mail.gmail.com>
	<200701031453.17348.anthony@interlink.com.au>
	<bbaeab100701022357q12de3b43u825e07639a9d594c@mail.gmail.com>
Message-ID: <173F5577-3E90-497B-AF74-9D405FA2D87E@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Jan 3, 2007, at 2:57 AM, Brett Cannon wrote:

> That's an idea if someone is up for doing that.  Could also hide  
> binascii as
> _binascii and have base64, quopri, and uu use it then to follow  
> what we are
> doing with cStringIO/StringIO et. al.

I think that's a possibility.  There may be some stuff in binascii  
that we need to expose in other places.  The other thing is that all  
of this stuff might be better off in a super-package, although I'm  
not sure what that would be called.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRZuqNHEjvBPtnXfVAQKtCwP/efz5YMRAUl8txXQma8+zFRGe5ltvN6h/
UrJicgszZ42PDmgOL8v9L5B7JF/76BSD8q6B4oIFs74D/LtECbkC35QQ1koPZb4U
Z7ltnfu2WQ/6Du9LN6yE7OJHP0wtOM07F4pAlAzPfsi29ZAArjFdistGL3b5Cxgp
0tnvwGbFxHQ=
=ROop
-----END PGP SIGNATURE-----

From barry at python.org  Wed Jan  3 14:07:34 2007
From: barry at python.org (Barry Warsaw)
Date: Wed, 3 Jan 2007 08:07:34 -0500
Subject: [Python-3000] PEP 3108: Standard Library Reorganization
In-Reply-To: <bbaeab100701030002t25b59a77wdc1ba61954c6ed3d@mail.gmail.com>
References: <20070101174925.82FB.JCARLSON@uci.edu>
	<20070101225107.8315.JCARLSON@uci.edu>
	<bbaeab100701021547l2eddd412wc11330c4cfcb46dd@mail.gmail.com>
	<200701031453.17348.anthony@interlink.com.au>
	<6319D387-3A48-41A4-BD51-979DDE8C585E@python.org>
	<bbaeab100701030002t25b59a77wdc1ba61954c6ed3d@mail.gmail.com>
Message-ID: <09DEF368-EF32-4FB0-B0D4-60199E6C3419@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Jan 3, 2007, at 3:02 AM, Brett Cannon wrote:

> I bet using an absolute import on the part of the codecs module to  
> specify
> the encodings package would deal with a lot of your worries.

Indeed!
- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRZuqlnEjvBPtnXfVAQLzBAP/QUapLBXlG29fFjuPTSheklZvh8UW82YH
UtcC8OKHRx4/yVnuiMqqZ7Z7hmY8MbQNqjF0OiBAnXSGgjar8yKZRTx6xtV04kwg
HHyQp2CNFVIsGmVrWm5Zd7Ee1aazvNu8vtjTCSoBL5VWKMycwamJmg27zaPhL0VA
U80FC6+NxuQ=
=q2AD
-----END PGP SIGNATURE-----

From murman at gmail.com  Wed Jan  3 15:23:29 2007
From: murman at gmail.com (Michael Urman)
Date: Wed, 3 Jan 2007 08:23:29 -0600
Subject: [Python-3000] PEP 3108: Standard Library Reorganization
In-Reply-To: <200701031453.17348.anthony@interlink.com.au>
References: <20070101174925.82FB.JCARLSON@uci.edu>
	<20070101225107.8315.JCARLSON@uci.edu>
	<bbaeab100701021547l2eddd412wc11330c4cfcb46dd@mail.gmail.com>
	<200701031453.17348.anthony@interlink.com.au>
Message-ID: <dcbbbb410701030623i555f1b27t5f030d4415fb56e@mail.gmail.com>

On 1/2/07, Anthony Baxter <anthony at interlink.com.au> wrote:
> base64.encodestring('hello world')
> or
> base64.b64encode('hello world')
>
> become
> codecs.getencoder('base64')('hello')[0]

Isn't this a lot more idiomatically translated as
    'hello world'.encode('base64')
or is there still too much worry that this will require
encoding/decoding ascii bytestrings?

Your additional points about base32 and base16 still apply to
.encode(), as they each result in an unknown encoding LookupError.

-- 
Michael Urman  http://www.tortall.net/mu/blog

From barry at python.org  Wed Jan  3 15:42:54 2007
From: barry at python.org (Barry Warsaw)
Date: Wed, 3 Jan 2007 09:42:54 -0500
Subject: [Python-3000] PEP 3108: Standard Library Reorganization
In-Reply-To: <dcbbbb410701030623i555f1b27t5f030d4415fb56e@mail.gmail.com>
References: <20070101174925.82FB.JCARLSON@uci.edu>
	<20070101225107.8315.JCARLSON@uci.edu>
	<bbaeab100701021547l2eddd412wc11330c4cfcb46dd@mail.gmail.com>
	<200701031453.17348.anthony@interlink.com.au>
	<dcbbbb410701030623i555f1b27t5f030d4415fb56e@mail.gmail.com>
Message-ID: <EDAB6255-B0E2-4A5C-9009-22ECB2FB4B8F@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Jan 3, 2007, at 9:23 AM, Michael Urman wrote:

> On 1/2/07, Anthony Baxter <anthony at interlink.com.au> wrote:
>> base64.encodestring('hello world')
>> or
>> base64.b64encode('hello world')
>>
>> become
>> codecs.getencoder('base64')('hello')[0]
>
> Isn't this a lot more idiomatically translated as
>     'hello world'.encode('base64')
> or is there still too much worry that this will require
> encoding/decoding ascii bytestrings?
>
> Your additional points about base32 and base16 still apply to
> .encode(), as they each result in an unknown encoding LookupError.

And don't forget that options can be passed to the encoders.

But fundamentally, I really dislike having to specify the encoder as  
a string rather than as a POFC (plain ol' function call).

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRZvA7nEjvBPtnXfVAQLYegP/ad1cGpFPqWwvTypuVwhzDKkQ0Zvi8SL1
VwuPDASO+m5gtofYp7GXWHrjMv7AUjq6lrZIp/LFSiwlS56yXZHhcJM3OJFVCq6p
FAoz0nuW7KN4C3AAC2SjEcTb/pG2czZndtlivA7IKSL5L8OZQg0sFgPMKDI8IioL
h6lcgBmYVX0=
=DnSG
-----END PGP SIGNATURE-----

From collinw at gmail.com  Wed Jan  3 18:06:06 2007
From: collinw at gmail.com (Collin Winter)
Date: Wed, 3 Jan 2007 11:06:06 -0600
Subject: [Python-3000] What's the point of annotations?
In-Reply-To: <5.1.1.6.0.20070102185639.02920e70@sparrow.telecommunity.com>
References: <5.1.1.6.0.20070102132113.02a5a968@sparrow.telecommunity.com>
	<5.1.1.6.0.20070102141711.0279fbd0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070102185639.02920e70@sparrow.telecommunity.com>
Message-ID: <43aa6ff70701030906x42419f58q4f2f94d8f584d818@mail.gmail.com>

On 1/2/07, Phillip J. Eby <pje at telecommunity.com> wrote:
> At 05:48 PM 1/2/2007 -0600, Collin Winter wrote:
> >Is whatever savings you see there worth changing the language for? Is
> >function overloading really that important and that common?
>
> Ah, but now you're making a different argument.  Your original statement
> was that "we're getting along without annotations quite nicely"; my
> response was merely to refute that point.  There do indeed exist use cases
> that are better off with annotation than without.

The argument I was trying (and perhaps failing) to make is that it
seems like the language is being changed to make life easier for some
pretty obscure and relatively rare use-cases, even if those cases are
taken together.

(Rearranging a bit)
> Now, if you want to pick at each and every one of these use cases and
> assert that the one use case, *by itself*, doesn't justify the feature,
> well, you may be right.  But as with decorators, that doesn't necessarily
> mean the feature itself is wrong!  Decorators are an excellent example of a
> feature that meets many use cases, few if any of which would be sufficient
> in themselves to justify a language change.

> For example, interfacing to at least Java and Objective C have also been
> mentioned as use cases, and I can think of a variety of other uses such as
> database query mapping, predicate logic functions, and web or other types
> of RPC parameter marshalling.

I don't see annotations and decorators as falling in to the same
category in that respect. Decorators make life easier for a much wider
audience than do annotations. Everyone uses decorators, but how many
people write/use Java/ObjC bridges? Predicate logic functions? RPC
parameter marshaling?

Decorators took something that everyone was already doing and made it
easier. Taken together, the corpus of annotations use-cases still
represents a fairly rarefied subset of Python usage.

Maybe I'm wrong, though. A serious question to anyone doing a lot of,
e.g., RPC work: will annotations make your life easier? In what way?

Thanks,
Collin Winter

PS: I think it's interesting how nearly all the use-cases mentioned
involve adding type information to functions.

From bill at janssen.org  Wed Jan  3 18:11:57 2007
From: bill at janssen.org (Bill Janssen)
Date: Wed, 3 Jan 2007 09:11:57 PST
Subject: [Python-3000] PEP 3108: Standard Library Reorganization
In-Reply-To: <dcbbbb410701030623i555f1b27t5f030d4415fb56e@mail.gmail.com> 
References: <20070101174925.82FB.JCARLSON@uci.edu>
	<20070101225107.8315.JCARLSON@uci.edu>
	<bbaeab100701021547l2eddd412wc11330c4cfcb46dd@mail.gmail.com>
	<200701031453.17348.anthony@interlink.com.au>
	<dcbbbb410701030623i555f1b27t5f030d4415fb56e@mail.gmail.com>
Message-ID: <07Jan3.091158pst."58648"@synergy1.parc.xerox.com>

> Isn't this a lot more idiomatically translated as
>     'hello world'.encode('base64')
> or is there still too much worry that this will require
> encoding/decoding ascii bytestrings?

Will this work properly in Python 3K?  I tend to use base64, for
example, to turn byte sequences into safe strings, so I need a way of
doing "encode" on byte sequences, not just strings.

In general, base64.encode() should operate on byte sequences and
return strings, and decode() should operate on strings and return byte
sequences.

Bill

From rhamph at gmail.com  Wed Jan  3 18:41:52 2007
From: rhamph at gmail.com (Adam Olsen)
Date: Wed, 3 Jan 2007 10:41:52 -0700
Subject: [Python-3000] self-contained exceptions
In-Reply-To: <ca471dc20701021228k592aced9g2244040f7ebc5b00@mail.gmail.com>
References: <1d85506f0701011438q6d8b65bfi4ef1e8f9fa509650@mail.gmail.com>
	<ca471dc20701011630l633130e0n356eb9b176f1d610@mail.gmail.com>
	<fb6fbf560701021220h4cb1b656u21c83d1b8b916fe5@mail.gmail.com>
	<ca471dc20701021228k592aced9g2244040f7ebc5b00@mail.gmail.com>
Message-ID: <aac2c7cb0701030941o7e7d72dai76861554011fd3a8@mail.gmail.com>

On 1/2/07, Guido van Rossum <guido at python.org> wrote:
> > On 1/1/07, Guido van Rossum <guido at python.org> wrote:
> > > will introduce a cycle from err -> traceback -> stack frame -> err,

> The problem with a weakref is that it won't keep the tb alive when you
> save the exception for later (as is done for instance by the unittest
> module). So it is of limited value and could even be confusing. OTOH
> it *does* remove the need to call sys.exc_info() in the most common
> case, so I don't want to rule it out completely.

That can be solved by moving the weakref to the stack frame -> exc
part, and only turning it from a strong reference into a weak
reference when the function exits.  When debugging via the raised
exception, the chain of __context__ references would keep it alive.

The disadvantage is it requires the local variable created by "except
Exception, exc" be special, perhaps not allowing any further
assignment to it or other limitations.

-- 
Adam Olsen, aka Rhamphoryncus

From guido at python.org  Wed Jan  3 18:43:25 2007
From: guido at python.org (Guido van Rossum)
Date: Wed, 3 Jan 2007 09:43:25 -0800
Subject: [Python-3000] What's the point of annotations?
In-Reply-To: <43aa6ff70701030906x42419f58q4f2f94d8f584d818@mail.gmail.com>
References: <5.1.1.6.0.20070102132113.02a5a968@sparrow.telecommunity.com>
	<5.1.1.6.0.20070102141711.0279fbd0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070102185639.02920e70@sparrow.telecommunity.com>
	<43aa6ff70701030906x42419f58q4f2f94d8f584d818@mail.gmail.com>
Message-ID: <ca471dc20701030943j5cef82e6o6cc1419bbba22cef@mail.gmail.com>

On 1/3/07, Collin Winter <collinw at gmail.com> wrote:
> PS: I think it's interesting how nearly all the use-cases mentioned
> involve adding type information to functions.

Not surprising, since that's the context where this was first brought
up. The earliest proposals *did* suggest standard type-checking and/or
adaptation semantics. Those were shot down, but the conclusion was
that some sort of syntax to tie attributes directly to arguments would
make it much easier to implement and experiment with alternative
schemes. Not all of those schemes need to use decorators, BTW -- it is
easily conceivable to use a metaclass that implements some kind of of
type checking or adaptation for all methods that have any kind of
argument annotation.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From pje at telecommunity.com  Wed Jan  3 19:05:31 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 03 Jan 2007 13:05:31 -0500
Subject: [Python-3000] What's the point of annotations?
In-Reply-To: <43aa6ff70701030906x42419f58q4f2f94d8f584d818@mail.gmail.co
 m>
References: <5.1.1.6.0.20070102185639.02920e70@sparrow.telecommunity.com>
	<5.1.1.6.0.20070102132113.02a5a968@sparrow.telecommunity.com>
	<5.1.1.6.0.20070102141711.0279fbd0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070102185639.02920e70@sparrow.telecommunity.com>
Message-ID: <5.1.1.6.0.20070103125604.02807198@sparrow.telecommunity.com>

At 11:06 AM 1/3/2007 -0600, Collin Winter wrote:
>I don't see annotations and decorators as falling in to the same
>category in that respect. Decorators make life easier for a much wider
>audience than do annotations. Everyone uses decorators, but how many
>people write/use Java/ObjC bridges?

Um, everybody using Jython or programming OS X with Python?  Also, now that 
you remind me of it, IronPython could use 'em for C# overloads.

>Predicate logic functions?

Take a look at the number of cookbook recipes out there.

>RPC
>parameter marshaling?

If annotations were available in 1996, Zope would likely have been using 
them for this (even though the word "Zope" didn't exist yet).  Annotating 
argument types on the function side is far more elegant than annotating 
them in the HTML form field names.


>Maybe I'm wrong, though. A serious question to anyone doing a lot of,
>e.g., RPC work: will annotations make your life easier? In what way?

Ask the folks working on *any* sort of foreign language bridging, who were 
also a major voice in some of the decorators discussion.

>PS: I think it's interesting how nearly all the use-cases mentioned
>involve adding type information to functions.

Um, yeah, that was kind of the idea.  :)  Note, however, that Java 
annotations (the place we stole the "@" sign from) can be applied to 
arguments as well as to methods and classes, and they *already* have the 
type information without needing annotations!  So, you might look to see 
what *else* they use it for, besides documentation and IDE hints.  I 
haven't done any Java work in a long time, myself.

(The problem with your approach to this, though, is that basically you 
still just keep complaining that each use case could be done some other 
way.  Of course they can.  That's not the point.  Might as well argue that 
we don't need both mappings *and* objects with attributes, because 
JavaScript gets by with combining the two.)


From bjourne at gmail.com  Wed Jan  3 19:10:36 2007
From: bjourne at gmail.com (=?ISO-8859-1?Q?BJ=F6rn_Lindqvist?=)
Date: Wed, 3 Jan 2007 19:10:36 +0100
Subject: [Python-3000] PEP 3108: Standard Library Reorganization
In-Reply-To: <bbaeab100701011614l27a87f45pb7e782c1d78a0725@mail.gmail.com>
References: <bbaeab100701011614l27a87f45pb7e782c1d78a0725@mail.gmail.com>
Message-ID: <740c3aec0701031010u7eb14f23i6d7ab710e2409e6c@mail.gmail.com>

Have the deprecations proposed by PEP 324
(http://www.python.org/dev/peps/pep-0324/), the subprocess module,
been forgotten or am I blind?

"The functions and modules that this new module is trying to
replace (os.system, os.spawn*, os.popen*, popen2.*, commands.*)
are expected to be available in future Python versions for a long
time, to preserve backwards compatibility."

With all the different ways to start a subprocess in Python, dropping
commands and popen2 should be pretty easy, right?

-- 
mvh Bj?rn

From jcarlson at uci.edu  Wed Jan  3 19:35:35 2007
From: jcarlson at uci.edu (Josiah Carlson)
Date: Wed, 03 Jan 2007 10:35:35 -0800
Subject: [Python-3000] PEP 3108: Standard Library Reorganization
In-Reply-To: <bbaeab100701022347k23f3750ej117de625c9daaa57@mail.gmail.com>
References: <20070102165914.8CC4.JCARLSON@uci.edu>
	<bbaeab100701022347k23f3750ej117de625c9daaa57@mail.gmail.com>
Message-ID: <20070103102317.8CCA.JCARLSON@uci.edu>


"Brett Cannon" <brett at python.org> wrote:
> I don't remember the whole bytes discussion in terms of encoding and
> decoding.

Check "the bytes type" summary and threads here:
http://mail.python.org/pipermail/python-dev/2006-April/063588.html

Also check the "encode and decode interface in Python 3.0" summary and
threads here:
http://mail.python.org/pipermail/python-dev/2006-April/063605.html


The three options were more or less:
1) remove bytes.encode and text.decode
2) change the exceptions messages to be more informative
3) do nothing


No one particularly liked #3, #2 could be done today and be usable in
both 2.6 and 3.0 .  Marc-Andre Lemburg was (at the time) in favor of #2,
where Guido preferred #1 (at least in the messages I've re-read).  I
personally prefer #2, as it allows for the most flexibility.


 - Josiah


From jcarlson at uci.edu  Wed Jan  3 19:44:27 2007
From: jcarlson at uci.edu (Josiah Carlson)
Date: Wed, 03 Jan 2007 10:44:27 -0800
Subject: [Python-3000] PEP 3108: Standard Library Reorganization
In-Reply-To: <EDAB6255-B0E2-4A5C-9009-22ECB2FB4B8F@python.org>
References: <dcbbbb410701030623i555f1b27t5f030d4415fb56e@mail.gmail.com>
	<EDAB6255-B0E2-4A5C-9009-22ECB2FB4B8F@python.org>
Message-ID: <20070103103849.8CCD.JCARLSON@uci.edu>


Barry Warsaw <barry at python.org> wrote:
> On Jan 3, 2007, at 9:23 AM, Michael Urman wrote:
> > On 1/2/07, Anthony Baxter <anthony at interlink.com.au> wrote:
> >> base64.encodestring('hello world')
> >> or
> >> base64.b64encode('hello world')
> >>
> >> become
> >> codecs.getencoder('base64')('hello')[0]
> >
> > Isn't this a lot more idiomatically translated as
> >     'hello world'.encode('base64')
> > or is there still too much worry that this will require
> > encoding/decoding ascii bytestrings?
> >
> > Your additional points about base32 and base16 still apply to
> > .encode(), as they each result in an unknown encoding LookupError.
> 
> And don't forget that options can be passed to the encoders.
> 
> But fundamentally, I really dislike having to specify the encoder as  
> a string rather than as a POFC (plain ol' function call).

What about:

    'hello world'.encode.base64()

Then incorrectly named encodings can raise an attribute error.  Users
can discover what they can encode/decode their objects to via dir(obj.encode),
etc.

It may take some name mangling on the part of the iso-* codecs (iso_*?),
and it may even make sense for modules in the encodings package to
specify if they can take optional arguments (for the alternate alphabet
for base64, etc.).


 - Josiah

From collinw at gmail.com  Wed Jan  3 20:00:31 2007
From: collinw at gmail.com (Collin Winter)
Date: Wed, 3 Jan 2007 13:00:31 -0600
Subject: [Python-3000] What's the point of annotations?
In-Reply-To: <5.1.1.6.0.20070103125604.02807198@sparrow.telecommunity.com>
References: <5.1.1.6.0.20070102132113.02a5a968@sparrow.telecommunity.com>
	<5.1.1.6.0.20070102141711.0279fbd0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070102185639.02920e70@sparrow.telecommunity.com>
	<5.1.1.6.0.20070103125604.02807198@sparrow.telecommunity.com>
Message-ID: <43aa6ff70701031100w75b386d5k1a585475ad111fbe@mail.gmail.com>

On 1/3/07, Phillip J. Eby <pje at telecommunity.com> wrote:
> At 11:06 AM 1/3/2007 -0600, Collin Winter wrote:
> >PS: I think it's interesting how nearly all the use-cases mentioned
> >involve adding type information to functions.
>
> Um, yeah, that was kind of the idea.  :)

I mention that as an amusing contrast to the "you could use
annotations for *anything*!" tone of the latter discussions and the
PEP.

> Note, however, that Java
> annotations (the place we stole the "@" sign from) can be applied to
> arguments as well as to methods and classes, and they *already* have the
> type information without needing annotations!  So, you might look to see
> what *else* they use it for, besides documentation and IDE hints.  I
> haven't done any Java work in a long time, myself.

I've never done any serious Java work, but the examples Google has
turned up seem to focus on working around limitations in Java's
reflection and type systems. What were you thinking of?

> (The problem with your approach to this, though, is that basically you
> still just keep complaining that each use case could be done some other
> way.  Of course they can.  That's not the point.  Might as well argue that
> we don't need both mappings *and* objects with attributes, because
> JavaScript gets by with combining the two.)

No, my point is that not every n-line function needs to be made into syntax.

Collin Winter

From barry at python.org  Wed Jan  3 20:05:35 2007
From: barry at python.org (Barry Warsaw)
Date: Wed, 3 Jan 2007 14:05:35 -0500
Subject: [Python-3000] PEP 3108: Standard Library Reorganization
In-Reply-To: <20070103103849.8CCD.JCARLSON@uci.edu>
References: <dcbbbb410701030623i555f1b27t5f030d4415fb56e@mail.gmail.com>
	<EDAB6255-B0E2-4A5C-9009-22ECB2FB4B8F@python.org>
	<20070103103849.8CCD.JCARLSON@uci.edu>
Message-ID: <C5AB5933-00CF-4A3C-8780-00C93FAA0741@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Jan 3, 2007, at 1:44 PM, Josiah Carlson wrote:

> What about:
>
>     'hello world'.encode.base64()
>
> Then incorrectly named encodings can raise an attribute error.  Users
> can discover what they can encode/decode their objects to via dir 
> (obj.encode),
> etc.

That's better.  I'm not sure it's a total win over what we have now,  
unless you're thinking that the mapping between .encode.<whatever>  
and .encode('whatever') is automatic.

- -Barry


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRZv+f3EjvBPtnXfVAQLNGAQAt8fggwFX6g7RUGaZ7o8UYQ+WjDkKKxx5
bp6CtZ058jTn5wXoHl6NXzaRkxbuaNgTYrrbZ/s1XQlad9uhMJsYWY7wFJm5DaHk
kO1fM5afVCpIOJXHcl1T9Jh9Y8HF4T+cdtybAGo/iO42LLrCWBga1CZqeHFV8VpR
Ru+s+sB50DU=
=8RZM
-----END PGP SIGNATURE-----

From brett at python.org  Wed Jan  3 20:42:15 2007
From: brett at python.org (Brett Cannon)
Date: Wed, 3 Jan 2007 11:42:15 -0800
Subject: [Python-3000] PEP 3108: Standard Library Reorganization
In-Reply-To: <740c3aec0701031010u7eb14f23i6d7ab710e2409e6c@mail.gmail.com>
References: <bbaeab100701011614l27a87f45pb7e782c1d78a0725@mail.gmail.com>
	<740c3aec0701031010u7eb14f23i6d7ab710e2409e6c@mail.gmail.com>
Message-ID: <bbaeab100701031142k5c98efd0l9f15f1748d210d57@mail.gmail.com>

On 1/3/07, BJ?rn Lindqvist <bjourne at gmail.com> wrote:
>
> Have the deprecations proposed by PEP 324
> (http://www.python.org/dev/peps/pep-0324/), the subprocess module,
> been forgotten or am I blind?


I forgot.

"The functions and modules that this new module is trying to
> replace (os.system, os.spawn*, os.popen*, popen2.*, commands.*)
> are expected to be available in future Python versions for a long
> time, to preserve backwards compatibility."
>
> With all the different ways to start a subprocess in Python, dropping
> commands and popen2 should be pretty easy, right?


Yep.  I will add the proper modules to be removed.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20070103/1eea3f2e/attachment.htm 

From pje at telecommunity.com  Wed Jan  3 21:12:48 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 03 Jan 2007 15:12:48 -0500
Subject: [Python-3000] What's the point of annotations?
In-Reply-To: <43aa6ff70701031100w75b386d5k1a585475ad111fbe@mail.gmail.co
 m>
References: <5.1.1.6.0.20070103125604.02807198@sparrow.telecommunity.com>
	<5.1.1.6.0.20070102132113.02a5a968@sparrow.telecommunity.com>
	<5.1.1.6.0.20070102141711.0279fbd0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070102185639.02920e70@sparrow.telecommunity.com>
	<5.1.1.6.0.20070103125604.02807198@sparrow.telecommunity.com>
Message-ID: <5.1.1.6.0.20070103141056.04408560@sparrow.telecommunity.com>

At 01:00 PM 1/3/2007 -0600, Collin Winter wrote:
>On 1/3/07, Phillip J. Eby <pje at telecommunity.com> wrote:
>>At 11:06 AM 1/3/2007 -0600, Collin Winter wrote:
>> >PS: I think it's interesting how nearly all the use-cases mentioned
>> >involve adding type information to functions.
>>
>>Um, yeah, that was kind of the idea.  :)
>
>I mention that as an amusing contrast to the "you could use
>annotations for *anything*!" tone of the latter discussions and the
>PEP.

I would guess that's because you're thinking "type" means "Python type 
object", which is a significantly narrower concept than is being talked 
about here.


>I've never done any serious Java work, but the examples Google has
>turned up seem to focus on working around limitations in Java's
>reflection and type systems. What were you thinking of?

Java annotations came in "after my time", so to speak.  I don't have 
anything specific in mind, I just know they exist and I gather they are used.


>>(The problem with your approach to this, though, is that basically you
>>still just keep complaining that each use case could be done some other
>>way.  Of course they can.  That's not the point.  Might as well argue that
>>we don't need both mappings *and* objects with attributes, because
>>JavaScript gets by with combining the two.)
>
>No, my point is that not every n-line function needs to be made into syntax.

Um, okay, but that has nothing to do with annotations.  :)

We added decorator syntax to move their declaration closer to the target 
and eliminate some duplication.  This is exactly the same; it's to improve 
clarity for the *reader* of the code, at least as much as it is a 
convenience for the writer of it.


From mbk.lists at gmail.com  Wed Jan  3 21:25:34 2007
From: mbk.lists at gmail.com (Mike Krell)
Date: Wed, 3 Jan 2007 13:25:34 -0700
Subject: [Python-3000] What's the point of annotations?
In-Reply-To: <5.1.1.6.0.20070103125604.02807198@sparrow.telecommunity.com>
References: <5.1.1.6.0.20070102132113.02a5a968@sparrow.telecommunity.com>
	<5.1.1.6.0.20070102141711.0279fbd0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070102185639.02920e70@sparrow.telecommunity.com>
	<5.1.1.6.0.20070103125604.02807198@sparrow.telecommunity.com>
Message-ID: <da7032ce0701031225ld342683t17cee8efb2efb9ef@mail.gmail.com>

> Also, now that you remind me of it, IronPython could use [annotations] for C# overloads.

FWIW, the IronPython folks are looking for a CPython-compatible syntax
for specifying .net attributes.  Function decorators (plus class
decorators) don't cover all the use cases because so many different
entities can be the target of a .net attribute.

Whether the proposed annotations help "enough" is a question for those
above my pay grade, i.e., I haven't been paying enough attention :-)

Cf. several related threads on the ironpython mailing list -- e.g.,

http://lists.ironpython.com/pipermail/users-ironpython.com/2006-September/003497.html

   Mike

From ronaldoussoren at mac.com  Wed Jan  3 21:26:50 2007
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Wed, 3 Jan 2007 21:26:50 +0100
Subject: [Python-3000] PEP 3108: Standard Library Reorganization
In-Reply-To: <bbaeab100701011614l27a87f45pb7e782c1d78a0725@mail.gmail.com>
References: <bbaeab100701011614l27a87f45pb7e782c1d78a0725@mail.gmail.com>
Message-ID: <12C8E611-8CBA-43A3-A70B-98A04FEB7B16@mac.com>


On 2 Jan, 2007, at 1:14, Brett Cannon wrote:


>
> * buildtools
>    2.3

This one is still used by buildapplet (a mac specific tool/module).  
However, see below for more on this.

> * cfmfile
>    2.4

mac specific, I don't know if this works on OSX (Jack probably knows).

> * macfs
>    2.3

Mac specific, I have no idea whether this is still in use.

> * Mac
>    + applesingle
>        - Undocumented.
>            * AppleSingle is a binary file format for A/UX.
>                + A/UX no longer distributed.

No problems here.

> * UNIX
>    + nis
>        - Wrapper for NIS.
>            * NIS has been replaced by LDAP, DNS, and Kerberos.

That's rather optimistic, NIS is still in active use. NIS is often  
easier to setup than LDAP/Kerberos, especially with older proprietary  
unix versions.
> * telnetlib
>    + Telnet is not used very much anymore.
>        - Telnet is unsafe.
>        - Most people use SSH instead.

I still use telnet to connect to embedded systems, or even ancient  
unix systems.  I must admit that I don't often use the telnetlib  
module but mostly just use plain socket connections and ignore the  
telnet protocol.

> * asynchat/asyncore
>    + Third-party libraries provide better solutions.
>        - twisted [#twisted]_
>    + Deprecation previously supported [#py-dev-summary-2004-11-01]_

Asyncore is still in active use, also not everyone agrees that  
Twisted is better.

> Modules to Rename
> =================
>
>
> PEP 8 specifies that modules "should have short, lowercase names,
> without underscores" [#pep-0008]_.

Why does the restriction on underscores exist? Removing that  
restriction would make lowercase-only names easier to use  
(basehttpserver vs. base_http_server).


> * autoGIL
>    autogil

This one is mac specific. It's probably possible to completely drop  
this one with some changes to  the Carbon bindings.

> * Carbon
>    carbon

Also mac specific. I'm pretty sure the capitalization is on purpose,  
this is a wrapper for Apple's Carbon libraries.

> * EasyDialogs
>    easydialogs
> * FrameWork
>    framework

Also mac specific. Framework is badly documented and hasn't even been  
updated to support Carbon Events, which itself is ancient technology  
by now.

> * MacOS
>    macos

> * MiniAEFrame
>    miniaeframe

Mac specific.

> * Nav
>    nav

Mac specific.

> * PixMapWrapper
>    pixmapwrapper

Mac specific

> * W
>    w

Mac specific and ancient (see Framework). It's also no longer present  
in the trunk.


>
>
> Merging C and Python implementations of the same interface
> ----------------------------------------------------------
>
> Several interfaces have both a Python and C implementation.  While it
> is great to have a C implementation for speed with a Python
> implementation as fallback, there is no need to expose the two
> implementations independently in the stdlib.  For Python 3.0 all
> interfaces with two implementations will be merged into a single
> public interface.

+lots on that. A Python and C implementation that are almost but not  
quite the same can be confusing at times.


> Open Issues
> ===========
> * mac
>    + Various Mac-specific modules.
>    + Same can be done for other platform-specific code.

The mac libraries need some serious love. Most of the extensions and  
corresponding python libraries are generated from Apple's headers  
files, but generating them requires the old OS9 SDK. It should be  
possible to convert the toolchain to use the current OSX headers, but  
that requires someone that either knows how bgen works or is willing  
to spend time to learn bgen.

Several parts of the mac library should be deprecated in 2.6 and  
removed in 3.0. Applescript/OSA support is an example of that: the  
applescript support in the stdlib is awkward to use and has some bugs  
(especially on intel macs). There's also a much better way to  
interface with Applescript/OSA: appscript (http:// 
appscript.sourceforge.net/).

Support for CoreFoundation should also be dropped, Carbon.CF is  
incomplete and barely tested. I'm already working on extending PyObjC  
to include full support for CoreFoundation (and other frameworks  
based on CF) [http://pyobjc.sf.net].

>
> Modules reliant on obsolete/rarely used file formats?
> -----------------------------------------------------
>
> Several modules in the stdlib work on a specific file format.  It is
> possible some of these formats are no longer used and thus the stdlib
> modules for them can go.  Below is a list of some modules which rely
> on a file format that may be obsolete.
>
> * aifc
>    AIFF and AIFF-C audio files.  Appears to be only user of the cl
>    module (which is undocumented).

AIFF is not an obsolete file format (http://en.wikipedia.org/wiki/AIFF).

> * binhex
>    binhex4 encoding.

binhex is still used in the mac world, although most software seems  
to move to more cross-platform-friendly formats.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3562 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-3000/attachments/20070103/d628ead7/attachment-0001.bin 

From jcarlson at uci.edu  Wed Jan  3 21:48:47 2007
From: jcarlson at uci.edu (Josiah Carlson)
Date: Wed, 03 Jan 2007 12:48:47 -0800
Subject: [Python-3000] PEP 3108: Standard Library Reorganization
In-Reply-To: <C5AB5933-00CF-4A3C-8780-00C93FAA0741@python.org>
References: <20070103103849.8CCD.JCARLSON@uci.edu>
	<C5AB5933-00CF-4A3C-8780-00C93FAA0741@python.org>
Message-ID: <20070103124007.8CD2.JCARLSON@uci.edu>


Barry Warsaw <barry at python.org> wrote:
> On Jan 3, 2007, at 1:44 PM, Josiah Carlson wrote:
> 
> > What about:
> >
> >     'hello world'.encode.base64()
> >
> > Then incorrectly named encodings can raise an attribute error.  Users
> > can discover what they can encode/decode their objects to via dir 
> > (obj.encode),
> > etc.
> 
> That's better.  I'm not sure it's a total win over what we have now,  
> unless you're thinking that the mapping between .encode.<whatever>  
> and .encode('whatever') is automatic.

I'm thinking that .encode('whatever') would disappear (maybe be
deprecated for 2.x if it was seen as a solution to long-standing issues),
to be replaced by .encode.whatever().


 - Josiah


From barry at python.org  Wed Jan  3 21:59:01 2007
From: barry at python.org (Barry Warsaw)
Date: Wed, 3 Jan 2007 15:59:01 -0500
Subject: [Python-3000] PEP 3108: Standard Library Reorganization
In-Reply-To: <20070103124007.8CD2.JCARLSON@uci.edu>
References: <20070103103849.8CCD.JCARLSON@uci.edu>
	<C5AB5933-00CF-4A3C-8780-00C93FAA0741@python.org>
	<20070103124007.8CD2.JCARLSON@uci.edu>
Message-ID: <54493BCF-194C-4C16-8273-0529DB7AE70E@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Jan 3, 2007, at 3:48 PM, Josiah Carlson wrote:

>> That's better.  I'm not sure it's a total win over what we have now,
>> unless you're thinking that the mapping between .encode.<whatever>
>> and .encode('whatever') is automatic.
>
> I'm thinking that .encode('whatever') would disappear (maybe be
> deprecated for 2.x if it was seen as a solution to long-standing  
> issues),
> to be replaced by .encode.whatever().

Hmm, that does sound interesting.  Sometimes you have a string that  
names the encoding, such as when you get it from an email header.   
Other times you know what function you want to call, such as is  
usually the case with base64 and friends.  OTOH, the former can  
always be handled with getattr().

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRZwZFXEjvBPtnXfVAQJOtAQAnQxa0iO9vMBfydR/Hx8rDIzJR8vVUvOY
lstDTS8EqW8I46C9Az01s+uVlx5bNgnEaKaPm4Q6VSZZZZi3Eaxz9VWRbOXec9iQ
lLoBAOyCo6mwz/GBv2sDp7dUM5hzhU/stETcWnSYXNqZpcV7uStSSSfsrxUhzl6D
CXs6qGddVvg=
=DfGN
-----END PGP SIGNATURE-----

From brett at python.org  Wed Jan  3 22:07:22 2007
From: brett at python.org (Brett Cannon)
Date: Wed, 3 Jan 2007 13:07:22 -0800
Subject: [Python-3000] PEP 3108: Standard Library Reorganization
In-Reply-To: <12C8E611-8CBA-43A3-A70B-98A04FEB7B16@mac.com>
References: <bbaeab100701011614l27a87f45pb7e782c1d78a0725@mail.gmail.com>
	<12C8E611-8CBA-43A3-A70B-98A04FEB7B16@mac.com>
Message-ID: <bbaeab100701031307v7cd62e6at34c095c0f84b8d2e@mail.gmail.com>

On 1/3/07, Ronald Oussoren <ronaldoussoren at mac.com> wrote:
>
>
> On 2 Jan, 2007, at 1:14, Brett Cannon wrote:
>
>
> >
> > * buildtools
> >    2.3
>
> This one is still used by buildapplet (a mac specific tool/module).
> However, see below for more on this.


Since the deprecation has already occurred it's out of my hands.

> * cfmfile
> >    2.4
>
> mac specific, I don't know if this works on OSX (Jack probably knows).


It's already deprecated so I am not going to worry about it.

> * macfs
> >    2.3
>
> Mac specific, I have no idea whether this is still in use.


Same as above.

> * Mac
> >    + applesingle
> >        - Undocumented.
> >            * AppleSingle is a binary file format for A/UX.
> >                + A/UX no longer distributed.
>
> No problems here.


Yay, no argument!  =)

> * UNIX
> >    + nis
> >        - Wrapper for NIS.
> >            * NIS has been replaced by LDAP, DNS, and Kerberos.
>
> That's rather optimistic, NIS is still in active use. NIS is often
> easier to setup than LDAP/Kerberos, especially with older proprietary
> unix versions.


It's already been saved.

> * telnetlib
> >    + Telnet is not used very much anymore.
> >        - Telnet is unsafe.
> >        - Most people use SSH instead.
>
> I still use telnet to connect to embedded systems, or even ancient
> unix systems.  I must admit that I don't often use the telnetlib
> module but mostly just use plain socket connections and ignore the
> telnet protocol.


The module has been saved.

> * asynchat/asyncore
> >    + Third-party libraries provide better solutions.
> >        - twisted [#twisted]_
> >    + Deprecation previously supported [#py-dev-summary-2004-11-01]_
>
> Asyncore is still in active use, also not everyone agrees that
> Twisted is better.


Been saved.

> Modules to Rename
> > =================
> >
> >
> > PEP 8 specifies that modules "should have short, lowercase names,
> > without underscores" [#pep-0008]_.
>
> Why does the restriction on underscores exist? Removing that
> restriction would make lowercase-only names easier to use
> (basehttpserver vs. base_http_server).


I think Guido said he preferred it that way.

> * autoGIL
> >    autogil
>
> This one is mac specific. It's probably possible to completely drop
> this one with some changes to  the Carbon bindings.


So should this be deprecated to force the change, or leave it be for now?

> * Carbon
> >    carbon
>
> Also mac specific. I'm pretty sure the capitalization is on purpose,
> this is a wrapper for Apple's Carbon libraries.


Possibly, but it still breaks the rules.

> * EasyDialogs
> >    easydialogs
> > * FrameWork
> >    framework
>
> Also mac specific. Framework is badly documented and hasn't even been
> updated to support Carbon Events, which itself is ancient technology
> by now.


So FrameWork can go?

> * MacOS
> >    macos
>
> > * MiniAEFrame
> >    miniaeframe
>
> Mac specific.
>
> > * Nav
> >    nav
>
> Mac specific.
>
> > * PixMapWrapper
> >    pixmapwrapper
>
> Mac specific
>
> > * W
> >    w
>
> Mac specific and ancient (see Framework). It's also no longer present
> in the trunk.


Then why are there still docs for it?

>
> >
> > Merging C and Python implementations of the same interface
> > ----------------------------------------------------------
> >
> > Several interfaces have both a Python and C implementation.  While it
> > is great to have a C implementation for speed with a Python
> > implementation as fallback, there is no need to expose the two
> > implementations independently in the stdlib.  For Python 3.0 all
> > interfaces with two implementations will be merged into a single
> > public interface.
>
> +lots on that. A Python and C implementation that are almost but not
> quite the same can be confusing at times.
>
>
> > Open Issues
> > ===========
> > * mac
> >    + Various Mac-specific modules.
> >    + Same can be done for other platform-specific code.
>
> The mac libraries need some serious love. Most of the extensions and
> corresponding python libraries are generated from Apple's headers
> files, but generating them requires the old OS9 SDK. It should be
> possible to convert the toolchain to use the current OSX headers, but
> that requires someone that either knows how bgen works or is willing
> to spend time to learn bgen.


Does that means we should remove all of the bgen-based modules (is that all
of them)?

Several parts of the mac library should be deprecated in 2.6 and
> removed in 3.0. Applescript/OSA support is an example of that: the
> applescript support in the stdlib is awkward to use and has some bugs
> (especially on intel macs).


Library references says gensuitemodule, aetools, aepack, aetypes, and
MiniAEFrame are for OSA.  Should they all go?

There's also a much better way to
> interface with Applescript/OSA: appscript (http://
> appscript.sourceforge.net/).


Are you up to helping me figure out what Mac modules should go?  If I write
up a list Mac-specific modules can you tell me which ones should go?

Support for CoreFoundation should also be dropped, Carbon.CF is
> incomplete and barely tested. I'm already working on extending PyObjC
> to include full support for CoreFoundation (and other frameworks
> based on CF) [http://pyobjc.sf.net].


That's good to hear.

>
> > Modules reliant on obsolete/rarely used file formats?
> > -----------------------------------------------------
> >
> > Several modules in the stdlib work on a specific file format.  It is
> > possible some of these formats are no longer used and thus the stdlib
> > modules for them can go.  Below is a list of some modules which rely
> > on a file format that may be obsolete.
> >
> > * aifc
> >    AIFF and AIFF-C audio files.  Appears to be only user of the cl
> >    module (which is undocumented).
>
> AIFF is not an obsolete file format (http://en.wikipedia.org/wiki/AIFF).


OK.

> * binhex
> >    binhex4 encoding.
>
> binhex is still used in the mac world, although most software seems
> to move to more cross-platform-friendly formats.



I have not come across a binhex file on OS X in ages (everything seems to be
DMG these days).  Is the use in the MacPython world enough to warrant
keeping this?

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20070103/ef935394/attachment.html 

From ronaldoussoren at mac.com  Wed Jan  3 22:43:42 2007
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Wed, 3 Jan 2007 22:43:42 +0100
Subject: [Python-3000] PEP 3108: Standard Library Reorganization
In-Reply-To: <bbaeab100701031307v7cd62e6at34c095c0f84b8d2e@mail.gmail.com>
References: <bbaeab100701011614l27a87f45pb7e782c1d78a0725@mail.gmail.com>
	<12C8E611-8CBA-43A3-A70B-98A04FEB7B16@mac.com>
	<bbaeab100701031307v7cd62e6at34c095c0f84b8d2e@mail.gmail.com>
Message-ID: <DCEE591C-ED60-494F-9EC3-B269289FC0B2@mac.com>


On 3 Jan, 2007, at 22:07, Brett Cannon wrote:

>
>
> On 1/3/07, Ronald Oussoren <ronaldoussoren at mac.com> wrote:
>
> On 2 Jan, 2007, at 1:14, Brett Cannon wrote:
>
>
> >
> > * buildtools
> >    2.3
>
> This one is still used by buildapplet (a mac specific tool/module).
> However, see below for more on this.
>
> Since the deprecation has already occurred it's out of my hands.

Let me rephrase: buildtools shouldn't be removed in 2.6 as it is  
still in use by buildapplet which isn't deprecated.  I wasn't paying  
much attention to python-dev when 2.3 was released and hadn't noticed  
that buildtools is deprecated or I'd warned about this earlier.


>
> > * cfmfile
> >    2.4
>
> mac specific, I don't know if this works on OSX (Jack probably knows).
>
> It's already deprecated so I am not going to worry about it.
>
> > * macfs
> >    2.3
>
> Mac specific, I have no idea whether this is still in use.
>
> Same as above.
>
> > * Mac
> >    + applesingle
> >        - Undocumented.
> >            * AppleSingle is a binary file format for A/UX.
> >                + A/UX no longer distributed.
>
> No problems here.
>
> Yay, no argument!  =)
>
> > * UNIX
> >    + nis
> >        - Wrapper for NIS.
> >            * NIS has been replaced by LDAP, DNS, and Kerberos.
>
> That's rather optimistic, NIS is still in active use. NIS is often
> easier to setup than LDAP/Kerberos, especially with older proprietary
> unix versions.
>
> It's already been saved.
>
> > * telnetlib
> >    + Telnet is not used very much anymore.
> >        - Telnet is unsafe.
> >        - Most people use SSH instead.
>
> I still use telnet to connect to embedded systems, or even ancient
> unix systems.  I must admit that I don't often use the telnetlib
> module but mostly just use plain socket connections and ignore the
> telnet protocol.
>
> The module has been saved.
>
> > * asynchat/asyncore
> >    + Third-party libraries provide better solutions.
> >        - twisted [#twisted]_
> >    + Deprecation previously supported [#py-dev-summary-2004-11-01]_
>
> Asyncore is still in active use, also not everyone agrees that
> Twisted is better.
>
> Been saved.
>
> > Modules to Rename
> > =================
> >
> >
> > PEP 8 specifies that modules "should have short, lowercase names,
> > without underscores" [#pep-0008]_.
>
> Why does the restriction on underscores exist? Removing that
> restriction would make lowercase-only names easier to use
> (basehttpserver vs. base_http_server).
>
> I think Guido said he preferred it that way.
>
> > * autoGIL
> >    autogil
>
> This one is mac specific. It's probably possible to completely drop
> this one with some changes to  the Carbon bindings.
>
> So should this be deprecated to force the change, or leave it be  
> for now?
>
> > * Carbon
> >    carbon
>
> Also mac specific. I'm pretty sure the capitalization is on purpose,
> this is a wrapper for Apple's Carbon libraries.
>
> Possibly, but it still breaks the rules.

So what, rule's are there to be broken :-). IMHO there should be room  
for deviations from PEP8 when there are good reasons to do so.

>
> > * EasyDialogs
> >    easydialogs
> > * FrameWork
> >    framework
>
> Also mac specific. Framework is badly documented and hasn't even been
> updated to support Carbon Events, which itself is ancient technology
> by now.
>
> So FrameWork can go?

I think so.

>
> > * MacOS
> >    macos
>
> > * MiniAEFrame
> >    miniaeframe
>
> Mac specific.
>
> > * Nav
> >    nav
>
> Mac specific.
>
> > * PixMapWrapper
> >    pixmapwrapper
>
> Mac specific
>
> > * W
> >    w
>
> Mac specific and ancient (see Framework). It's also no longer present
> in the trunk.
>
> Then why are there still docs for it?

I don't know. To be honest I didn't even know there were docs for it.

>
> >
> >
> > Merging C and Python implementations of the same interface
> > ----------------------------------------------------------
> >
> > Several interfaces have both a Python and C implementation.   
> While it
> > is great to have a C implementation for speed with a Python
> > implementation as fallback, there is no need to expose the two
> > implementations independently in the stdlib.  For Python 3.0 all
> > interfaces with two implementations will be merged into a single
> > public interface.
>
> +lots on that. A Python and C implementation that are almost but not
> quite the same can be confusing at times.
>
>
> > Open Issues
> > ===========
> > * mac
> >    + Various Mac-specific modules.
> >    + Same can be done for other platform-specific code.
>
> The mac libraries need some serious love. Most of the extensions and
> corresponding python libraries are generated from Apple's headers
> files, but generating them requires the old OS9 SDK. It should be
> possible to convert the toolchain to use the current OSX headers, but
> that requires someone that either knows how bgen works or is willing
> to spend time to learn bgen.
>
> Does that means we should remove all of the bgen-based modules (is  
> that all of them)?

No that means that someone should look at bgen to make it use the  
current version of the headers. The generated code still works, but  
doesn't expose functionality that was added after MacOS9.

>
> Several parts of the mac library should be deprecated in 2.6 and
> removed in 3.0. Applescript/OSA support is an example of that: the
> applescript support in the stdlib is awkward to use and has some bugs
> (especially on intel macs).
>
> Library references says gensuitemodule, aetools, aepack, aetypes,  
> and MiniAEFrame are for OSA.  Should they all go?

aetools, aepack and aetypes should definitely go. The code in those  
modules has serious byteorder related issues: the code is heavily  
dependend on the fact that Mac's use a bigendian processor and that  
is no longer true.

The others should also go in python 3.0.

>
> There's also a much better way to
> interface with Applescript/OSA: appscript (http://
> appscript.sourceforge.net/).
>
> Are you up to helping me figure out what Mac modules should go?  If  
> I write up a list Mac-specific modules can you tell me which ones  
> should go?

Yes, but not right now. I'm trying to hunt down a rather nasty bug in  
PyObjC. Threading sucks, even if you can rely on the GIL in your C  
code :-(

>
> Support for CoreFoundation should also be dropped, Carbon.CF is
> incomplete and barely tested. I'm already working on extending PyObjC
> to include full support for CoreFoundation (and other frameworks
> based on CF) [http://pyobjc.sf.net].
>
> That's good to hear.


>
> > * binhex
> >    binhex4 encoding.
>
> binhex is still used in the mac world, although most software seems
> to move to more cross-platform-friendly formats.
>
>
> I have not come across a binhex file on OS X in ages (everything  
> seems to be DMG these days).  Is the use in the MacPython world  
> enough to warrant keeping this?

I'm not happy about dropping modules when there are no alternatives.   
Some email clients, such as Eudora can still send attachments in  
binhex format (although not by default).

I know that some companies still use OS9 systems and binhex is used  
more in those environments.

BTW. I'm just -0 on removing binhex. Thanks to setuptools and PyPI  
it's easy enough to move binhex to a separate distribution when it  
turns out that people still use it.

Ronald

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20070103/20091a59/attachment.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3562 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-3000/attachments/20070103/20091a59/attachment-0001.bin 

From brett at python.org  Wed Jan  3 22:58:39 2007
From: brett at python.org (Brett Cannon)
Date: Wed, 3 Jan 2007 13:58:39 -0800
Subject: [Python-3000] PEP 3108: Standard Library Reorganization
In-Reply-To: <DCEE591C-ED60-494F-9EC3-B269289FC0B2@mac.com>
References: <bbaeab100701011614l27a87f45pb7e782c1d78a0725@mail.gmail.com>
	<12C8E611-8CBA-43A3-A70B-98A04FEB7B16@mac.com>
	<bbaeab100701031307v7cd62e6at34c095c0f84b8d2e@mail.gmail.com>
	<DCEE591C-ED60-494F-9EC3-B269289FC0B2@mac.com>
Message-ID: <bbaeab100701031358i5c3cbf79md8b173e624ce02ff@mail.gmail.com>

On 1/3/07, Ronald Oussoren <ronaldoussoren at mac.com> wrote:
>
>
> On 3 Jan, 2007, at 22:07, Brett Cannon wrote:
>
>
>
> On 1/3/07, Ronald Oussoren <ronaldoussoren at mac.com> wrote:
> >
> >
> > On 2 Jan, 2007, at 1:14, Brett Cannon wrote:
> >
> >
> > >
> > > * buildtools
> > >    2.3
> >
> > This one is still used by buildapplet (a mac specific tool/module).
> > However, see below for more on this.
>
>
> Since the deprecation has already occurred it's out of my hands.
>
>
> Let me rephrase: buildtools shouldn't be removed in 2.6 as it is still in
> use by buildapplet which isn't deprecated.  I wasn't paying much attention
> to python-dev when 2.3 was released and hadn't noticed that buildtools is
> deprecated or I'd warned about this earlier.
>

This is PEP is for 3.0, not 2.6.  And it looks like it is only a documented
deprecation, not in PEP 4, so taking it back is no big deal.  Just change
the docs (same with the other Mac-specific deprecations).  If you can file a
bug report to remove the deprecations that you want to keep around.

> * cfmfile
> > >    2.4
> >
> > mac specific, I don't know if this works on OSX (Jack probably knows).
>
>
> It's already deprecated so I am not going to worry about it.
>
> > * macfs
> > >    2.3
> >
> > Mac specific, I have no idea whether this is still in use.
>
>
> Same as above.
>
> > * Mac
> > >    + applesingle
> > >        - Undocumented.
> > >            * AppleSingle is a binary file format for A/UX.
> > >                + A/UX no longer distributed.
> >
> > No problems here.
>
>
> Yay, no argument!  =)
>
> > * UNIX
> > >    + nis
> > >        - Wrapper for NIS.
> > >            * NIS has been replaced by LDAP, DNS, and Kerberos.
> >
> > That's rather optimistic, NIS is still in active use. NIS is often
> > easier to setup than LDAP/Kerberos, especially with older proprietary
> > unix versions.
>
>
> It's already been saved.
>
> > * telnetlib
> > >    + Telnet is not used very much anymore.
> > >        - Telnet is unsafe.
> > >        - Most people use SSH instead.
> >
> > I still use telnet to connect to embedded systems, or even ancient
> > unix systems.  I must admit that I don't often use the telnetlib
> > module but mostly just use plain socket connections and ignore the
> > telnet protocol.
>
>
> The module has been saved.
>
> > * asynchat/asyncore
> > >    + Third-party libraries provide better solutions.
> > >        - twisted [#twisted]_
> > >    + Deprecation previously supported [#py-dev-summary-2004-11-01]_
> >
> > Asyncore is still in active use, also not everyone agrees that
> > Twisted is better.
>
>
> Been saved.
>
> > Modules to Rename
> > > =================
> > >
> > >
> > > PEP 8 specifies that modules "should have short, lowercase names,
> > > without underscores" [#pep-0008]_.
> >
> > Why does the restriction on underscores exist? Removing that
> > restriction would make lowercase-only names easier to use
> > (basehttpserver vs. base_http_server).
>
>
> I think Guido said he preferred it that way.
>
> > * autoGIL
> > >    autogil
> >
> > This one is mac specific. It's probably possible to completely drop
> > this one with some changes to  the Carbon bindings.
>
>
> So should this be deprecated to force the change, or leave it be for now?
>
> > * Carbon
> > >    carbon
> >
> > Also mac specific. I'm pretty sure the capitalization is on purpose,
> > this is a wrapper for Apple's Carbon libraries.
>
>
> Possibly, but it still breaks the rules.
>
>
> So what, rule's are there to be broken :-). IMHO there should be room for
> deviations from PEP8 when there are good reasons to do so.
>

Well, there is also talk of putting platform-specific stuff in their own
package and that would help make it easier to ignore.  =)

> * EasyDialogs
> > >    easydialogs
> > > * FrameWork
> > >    framework
> >
> > Also mac specific. Framework is badly documented and hasn't even been
> > updated to support Carbon Events, which itself is ancient technology
> > by now.
>
>
> So FrameWork can go?
>
>
> I think so.
>

OK, added.

> * MacOS
> > >    macos
> >
> > > * MiniAEFrame
> > >    miniaeframe
> >
> > Mac specific.
> >
> > > * Nav
> > >    nav
> >
> > Mac specific.
> >
> > > * PixMapWrapper
> > >    pixmapwrapper
> >
> > Mac specific
> >
> > > * W
> > >    w
> >
> > Mac specific and ancient (see Framework). It's also no longer present
> > in the trunk.
>
>
> Then why are there still docs for it?
>
>
> I don't know. To be honest I didn't even know there were docs for it.
>

OK.  File a bug report to have the docs pulled if the  thing is not even in
Python anymore.

>
> > >
> > > Merging C and Python implementations of the same interface
> > > ----------------------------------------------------------
> > >
> > > Several interfaces have both a Python and C implementation.  While it
> > > is great to have a C implementation for speed with a Python
> > > implementation as fallback, there is no need to expose the two
> > > implementations independently in the stdlib.  For Python 3.0 all
> > > interfaces with two implementations will be merged into a single
> > > public interface.
> >
> > +lots on that. A Python and C implementation that are almost but not
> > quite the same can be confusing at times.
> >
> >
> > > Open Issues
> > > ===========
> > > * mac
> > >    + Various Mac-specific modules.
> > >    + Same can be done for other platform-specific code.
> >
> > The mac libraries need some serious love. Most of the extensions and
> > corresponding python libraries are generated from Apple's headers
> > files, but generating them requires the old OS9 SDK. It should be
> > possible to convert the toolchain to use the current OSX headers, but
> > that requires someone that either knows how bgen works or is willing
> > to spend time to learn bgen.
>
>
> Does that means we should remove all of the bgen-based modules (is that
> all of them)?
>
>
> No that means that someone should look at bgen to make it use the current
> version of the headers. The generated code still works, but doesn't expose
> functionality that was added after MacOS9.
>

Ah, OK.

Several parts of the mac library should be deprecated in 2.6 and
> > removed in 3.0. Applescript/OSA support is an example of that: the
> > applescript support in the stdlib is awkward to use and has some bugs
> > (especially on intel macs).
>
>
> Library references says gensuitemodule, aetools, aepack, aetypes, and
> MiniAEFrame are for OSA.  Should they all go?
>
>
> aetools, aepack and aetypes should definitely go. The code in those
> modules has serious byteorder related issues: the code is heavily dependend
> on the fact that Mac's use a bigendian processor and that is no longer
> true.
>
> The others should also go in python 3.0.
>


OK I added aetools, aepack, aetypes, and MiniAEFrame to the list of things
to go.

There's also a much better way to
> > interface with Applescript/OSA: appscript (http://
> > appscript.sourceforge.net/).
>
>
> Are you up to helping me figure out what Mac modules should go?  If I
> write up a list Mac-specific modules can you tell me which ones should go?
>
>
> Yes, but not right now. I'm trying to hunt down a rather nasty bug in
> PyObjC. Threading sucks, even if you can rely on the GIL in your C code :-(
>


Good enough.  No rush.  I will pull together the list and send it to you
off-list (with a cc to Bob).

Support for CoreFoundation should also be dropped, Carbon.CF is
> > incomplete and barely tested. I'm already working on extending PyObjC
> > to include full support for CoreFoundation (and other frameworks
> > based on CF) [http://pyobjc.sf.net].
>
>
> That's good to hear.
>
>
>
>
> > * binhex
> > >    binhex4 encoding.
> >
> > binhex is still used in the mac world, although most software seems
> > to move to more cross-platform-friendly formats.
>
>
>
> I have not come across a binhex file on OS X in ages (everything seems to
> be DMG these days).  Is the use in the MacPython world enough to warrant
> keeping this?
>
>
> I'm not happy about dropping modules when there are no alternatives.  Some
> email clients, such as Eudora can still send attachments in binhex format
> (although not by default).
>
> I know that some companies still use OS9 systems and binhex is used more
> in those environments.
>
> BTW. I'm just -0 on removing binhex. Thanks to setuptools and PyPI it's
> easy enough to move binhex to a separate distribution when it turns out that
> people still use it.
>


Well, based on that info binhex can stay.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20070103/f91dc8b9/attachment.html 

From collinw at gmail.com  Wed Jan  3 22:59:55 2007
From: collinw at gmail.com (Collin Winter)
Date: Wed, 3 Jan 2007 15:59:55 -0600
Subject: [Python-3000] What's the point of annotations?
In-Reply-To: <5.1.1.6.0.20070102185639.02920e70@sparrow.telecommunity.com>
References: <5.1.1.6.0.20070102132113.02a5a968@sparrow.telecommunity.com>
	<5.1.1.6.0.20070102141711.0279fbd0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070102185639.02920e70@sparrow.telecommunity.com>
Message-ID: <43aa6ff70701031359p3c045d93u599ab0da61d207c7@mail.gmail.com>

On 1/2/07, Phillip J. Eby <pje at telecommunity.com> wrote:
> For example, interfacing to at least Java and Objective C have also been
> mentioned as use cases, and I can think of a variety of other uses such as
> database query mapping, predicate logic functions, and web or other types
> of RPC parameter marshalling.

Several people have emailed me off-list and mentioned a number of
other use-cases that I hadn't thought of, which is what I was looking
for. I'm now convinced that there's a sufficiently broad spectrum of
uses for annotations. I'll be working up a patch for PEP 3107 that
will include a "possible use cases" section.

Consider my objection dropped.

Thanks,
Collin Winter

From tony at PageDNA.com  Thu Jan  4 00:46:42 2007
From: tony at PageDNA.com (Tony Lownds)
Date: Wed, 3 Jan 2007 15:46:42 -0800
Subject: [Python-3000] What's the point of annotations?
In-Reply-To: <43aa6ff70701031359p3c045d93u599ab0da61d207c7@mail.gmail.com>
References: <5.1.1.6.0.20070102132113.02a5a968@sparrow.telecommunity.com>
	<5.1.1.6.0.20070102141711.0279fbd0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070102185639.02920e70@sparrow.telecommunity.com>
	<43aa6ff70701031359p3c045d93u599ab0da61d207c7@mail.gmail.com>
Message-ID: <FEDC6EBF-19C1-48D8-8001-A0510F5A1432@PageDNA.com>


On Jan 3, 2007, at 1:59 PM, Collin Winter wrote:

> On 1/2/07, Phillip J. Eby <pje at telecommunity.com> wrote:
>> For example, interfacing to at least Java and Objective C have  
>> also been
>> mentioned as use cases, and I can think of a variety of other uses  
>> such as
>> database query mapping, predicate logic functions, and web or  
>> other types
>> of RPC parameter marshalling.
>
> Several people have emailed me off-list and mentioned a number of
> other use-cases that I hadn't thought of, which is what I was looking
> for. I'm now convinced that there's a sufficiently broad spectrum of
> uses for annotations. I'll be working up a patch for PEP 3107 that
> will include a "possible use cases" section.
>
> Consider my objection dropped.
>

Whew! :)

Collin can you remove the use of types in examples at the same time?
I'll have some time tonight to finish those edits if you don't...

Thanks
-Tony




From python at zesty.ca  Thu Jan  4 03:54:00 2007
From: python at zesty.ca (Ka-Ping Yee)
Date: Wed, 3 Jan 2007 20:54:00 -0600 (CST)
Subject: [Python-3000] self-contained exceptions
In-Reply-To: <aac2c7cb0701030941o7e7d72dai76861554011fd3a8@mail.gmail.com>
References: <1d85506f0701011438q6d8b65bfi4ef1e8f9fa509650@mail.gmail.com>
	<ca471dc20701011630l633130e0n356eb9b176f1d610@mail.gmail.com>
	<fb6fbf560701021220h4cb1b656u21c83d1b8b916fe5@mail.gmail.com>
	<ca471dc20701021228k592aced9g2244040f7ebc5b00@mail.gmail.com>
	<aac2c7cb0701030941o7e7d72dai76861554011fd3a8@mail.gmail.com>
Message-ID: <Pine.LNX.4.58.0701032052370.16811@server1.LFW.org>

On Wed, 3 Jan 2007, Adam Olsen wrote:
> That can be solved by moving the weakref to the stack frame -> exc
> part, and only turning it from a strong reference into a weak
> reference when the function exits.  When debugging via the raised
> exception, the chain of __context__ references would keep it alive.

That seems like a reasonable approach to me.  The implementation
might be tricky, but the behaviour sounds correct.


-- ?!ng

From guido at python.org  Thu Jan  4 04:38:24 2007
From: guido at python.org (Guido van Rossum)
Date: Wed, 3 Jan 2007 19:38:24 -0800
Subject: [Python-3000] self-contained exceptions
In-Reply-To: <Pine.LNX.4.58.0701032052370.16811@server1.LFW.org>
References: <1d85506f0701011438q6d8b65bfi4ef1e8f9fa509650@mail.gmail.com>
	<ca471dc20701011630l633130e0n356eb9b176f1d610@mail.gmail.com>
	<fb6fbf560701021220h4cb1b656u21c83d1b8b916fe5@mail.gmail.com>
	<ca471dc20701021228k592aced9g2244040f7ebc5b00@mail.gmail.com>
	<aac2c7cb0701030941o7e7d72dai76861554011fd3a8@mail.gmail.com>
	<Pine.LNX.4.58.0701032052370.16811@server1.LFW.org>
Message-ID: <ca471dc20701031938m29a61a64n65f2201dd4335d66@mail.gmail.com>

On 1/3/07, Ka-Ping Yee <python at zesty.ca> wrote:
> On Wed, 3 Jan 2007, Adam Olsen wrote:
> > That can be solved by moving the weakref to the stack frame -> exc
> > part, and only turning it from a strong reference into a weak
> > reference when the function exits.  When debugging via the raised
> > exception, the chain of __context__ references would keep it alive.
>
> That seems like a reasonable approach to me.  The implementation
> might be tricky, but the behaviour sounds correct.

Which reference are you exactly turning into a weak ref? The hidden
reference in the frame's f_exc_traceback? Or the variable 'err' in

  except Exception, err: ...

? The former goes away anyway so I'm not sure that this helps. And the
latter seems fraught with peril; how do we even know which variables
to treat this way? And why isn't it just as good to simply *delete*
all locals that ever occurred in such a position in an except clause?

Or am I totally misunderstanding?  Do note that there's a long comment
in ceval.c explaining how exceptions are stored in the thread state
and in the frame, right above set_exc_info().

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From pje at telecommunity.com  Thu Jan  4 05:04:01 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 03 Jan 2007 23:04:01 -0500
Subject: [Python-3000] self-contained exceptions
In-Reply-To: <ca471dc20701031938m29a61a64n65f2201dd4335d66@mail.gmail.co
 m>
References: <Pine.LNX.4.58.0701032052370.16811@server1.LFW.org>
	<1d85506f0701011438q6d8b65bfi4ef1e8f9fa509650@mail.gmail.com>
	<ca471dc20701011630l633130e0n356eb9b176f1d610@mail.gmail.com>
	<fb6fbf560701021220h4cb1b656u21c83d1b8b916fe5@mail.gmail.com>
	<ca471dc20701021228k592aced9g2244040f7ebc5b00@mail.gmail.com>
	<aac2c7cb0701030941o7e7d72dai76861554011fd3a8@mail.gmail.com>
	<Pine.LNX.4.58.0701032052370.16811@server1.LFW.org>
Message-ID: <5.1.1.6.0.20070103225854.042a3868@sparrow.telecommunity.com>

At 07:38 PM 1/3/2007 -0800, Guido van Rossum wrote:
>On 1/3/07, Ka-Ping Yee <python at zesty.ca> wrote:
> > On Wed, 3 Jan 2007, Adam Olsen wrote:
> > > That can be solved by moving the weakref to the stack frame -> exc
> > > part, and only turning it from a strong reference into a weak
> > > reference when the function exits.  When debugging via the raised
> > > exception, the chain of __context__ references would keep it alive.
> >
> > That seems like a reasonable approach to me.  The implementation
> > might be tricky, but the behaviour sounds correct.
>
>Which reference are you exactly turning into a weak ref? The hidden
>reference in the frame's f_exc_traceback? Or the variable 'err' in
>
>   except Exception, err: ...
>
>? The former goes away anyway so I'm not sure that this helps. And the
>latter seems fraught with peril; how do we even know which variables
>to treat this way? And why isn't it just as good to simply *delete*
>all locals that ever occurred in such a position in an except clause?

Or perhaps translate blocks of the form:

     except ExcType, e:
         # body

to:

     except ExcType, e:
         try:
             # body
         finally:
             del e

This won't stop you from creating a cycle explicitly, of course, but it 
would ensure that the simple cases would be cycle-free.


From guido at python.org  Thu Jan  4 06:11:10 2007
From: guido at python.org (Guido van Rossum)
Date: Wed, 3 Jan 2007 21:11:10 -0800
Subject: [Python-3000] self-contained exceptions
In-Reply-To: <5.1.1.6.0.20070103225854.042a3868@sparrow.telecommunity.com>
References: <1d85506f0701011438q6d8b65bfi4ef1e8f9fa509650@mail.gmail.com>
	<ca471dc20701011630l633130e0n356eb9b176f1d610@mail.gmail.com>
	<fb6fbf560701021220h4cb1b656u21c83d1b8b916fe5@mail.gmail.com>
	<ca471dc20701021228k592aced9g2244040f7ebc5b00@mail.gmail.com>
	<aac2c7cb0701030941o7e7d72dai76861554011fd3a8@mail.gmail.com>
	<Pine.LNX.4.58.0701032052370.16811@server1.LFW.org>
	<5.1.1.6.0.20070103225854.042a3868@sparrow.telecommunity.com>
Message-ID: <ca471dc20701032111h22a48053nae1c205e21103305@mail.gmail.com>

On 1/3/07, Phillip J. Eby <pje at telecommunity.com> wrote:
> Or perhaps translate blocks of the form:
>
>      except ExcType, e:
>          # body
>
> to:
>
>      except ExcType, e:
>          try:
>              # body
>          finally:
>              del e
>
> This won't stop you from creating a cycle explicitly, of course, but it
> would ensure that the simple cases would be cycle-free.

+1!

I used to dislike this because there are use cases for letting the
exception survive the except clause, but I think I can get used to it,
and it seems the most straightforward solution of all that I've seen.
We could completely get rid of sys.exc_info()! I think we have a
winner here.

Explicitly created cycles are no big deal IMO -- these are no worse
than current code that explicitly stores sys.exc_info()[2].

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From nnorwitz at gmail.com  Thu Jan  4 06:30:15 2007
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Wed, 3 Jan 2007 21:30:15 -0800
Subject: [Python-3000] PEP 3108: Standard Library Reorganization
In-Reply-To: <12C8E611-8CBA-43A3-A70B-98A04FEB7B16@mac.com>
References: <bbaeab100701011614l27a87f45pb7e782c1d78a0725@mail.gmail.com>
	<12C8E611-8CBA-43A3-A70B-98A04FEB7B16@mac.com>
Message-ID: <ee2a432c0701032130j75981817ib46aa68b3eaa014d@mail.gmail.com>

On 1/3/07, Ronald Oussoren <ronaldoussoren at mac.com> wrote:
>
> On 2 Jan, 2007, at 1:14, Brett Cannon wrote:
>
> > * Mac
> >    + applesingle
> >        - Undocumented.
> >            * AppleSingle is a binary file format for A/UX.
> >                + A/UX no longer distributed.
>
> No problems here.

I have to say I almost shed a tear when I saw this.  :-)  A/UX was the
first Unix I used oh so many years ago.

n

From pje at telecommunity.com  Thu Jan  4 06:53:30 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 04 Jan 2007 00:53:30 -0500
Subject: [Python-3000] self-contained exceptions
In-Reply-To: <ca471dc20701032111h22a48053nae1c205e21103305@mail.gmail.co
 m>
References: <5.1.1.6.0.20070103225854.042a3868@sparrow.telecommunity.com>
	<1d85506f0701011438q6d8b65bfi4ef1e8f9fa509650@mail.gmail.com>
	<ca471dc20701011630l633130e0n356eb9b176f1d610@mail.gmail.com>
	<fb6fbf560701021220h4cb1b656u21c83d1b8b916fe5@mail.gmail.com>
	<ca471dc20701021228k592aced9g2244040f7ebc5b00@mail.gmail.com>
	<aac2c7cb0701030941o7e7d72dai76861554011fd3a8@mail.gmail.com>
	<Pine.LNX.4.58.0701032052370.16811@server1.LFW.org>
	<5.1.1.6.0.20070103225854.042a3868@sparrow.telecommunity.com>
Message-ID: <5.1.1.6.0.20070104004429.02a19820@sparrow.telecommunity.com>

At 09:11 PM 1/3/2007 -0800, Guido van Rossum wrote:
>On 1/3/07, Phillip J. Eby <pje at telecommunity.com> wrote:
>>Or perhaps translate blocks of the form:
>>
>>      except ExcType, e:
>>          # body
>>
>>to:
>>
>>      except ExcType, e:
>>          try:
>>              # body
>>          finally:
>>              del e
>>
>>This won't stop you from creating a cycle explicitly, of course, but it
>>would ensure that the simple cases would be cycle-free.
>
>+1!
>
>I used to dislike this because there are use cases for letting the
>exception survive the except clause, but I think I can get used to it,
>and it seems the most straightforward solution of all that I've seen.
>We could completely get rid of sys.exc_info()! I think we have a
>winner here.

Actually, on second thought it occurs to me that the above code isn't a 
100% correct translation, because if "body" contains its own "del e", then 
it will fail.  So a pure translation strategy isn't really practical, 
unfortunately.  We'd have to generate something like:

     if 'e' in locals():
         del e

or add a new bytecode that's equivalent to that.


>Explicitly created cycles are no big deal IMO -- these are no worse
>than current code that explicitly stores sys.exc_info()[2].

True, but now they get that overhead simply by storing the exception 
object.  On the other hand, I suppose code that might store a lot of 
exception objects will just have to be careful to clear tracebacks it 
doesn't need.


From ronaldoussoren at mac.com  Thu Jan  4 07:50:12 2007
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Thu, 4 Jan 2007 07:50:12 +0100
Subject: [Python-3000] PEP 3108: Standard Library Reorganization
In-Reply-To: <ee2a432c0701032130j75981817ib46aa68b3eaa014d@mail.gmail.com>
References: <bbaeab100701011614l27a87f45pb7e782c1d78a0725@mail.gmail.com>
	<12C8E611-8CBA-43A3-A70B-98A04FEB7B16@mac.com>
	<ee2a432c0701032130j75981817ib46aa68b3eaa014d@mail.gmail.com>
Message-ID: <13D4EE34-70D9-4BE5-94DA-220979EB000E@mac.com>


On 4 Jan, 2007, at 6:30, Neal Norwitz wrote:

> On 1/3/07, Ronald Oussoren <ronaldoussoren at mac.com> wrote:
>>
>> On 2 Jan, 2007, at 1:14, Brett Cannon wrote:
>>
>> > * Mac
>> >    + applesingle
>> >        - Undocumented.
>> >            * AppleSingle is a binary file format for A/UX.
>> >                + A/UX no longer distributed.
>>
>> No problems here.
>
> I have to say I almost shed a tear when I saw this.  :-)  A/UX was the
> first Unix I used oh so many years ago.

Cool, another A/UX user :-).

Ronald

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3562 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-3000/attachments/20070104/63ffb461/attachment.bin 

From python at zesty.ca  Thu Jan  4 08:41:49 2007
From: python at zesty.ca (Ka-Ping Yee)
Date: Thu, 4 Jan 2007 01:41:49 -0600 (CST)
Subject: [Python-3000] self-contained exceptions
In-Reply-To: <5.1.1.6.0.20070104004429.02a19820@sparrow.telecommunity.com>
References: <5.1.1.6.0.20070103225854.042a3868@sparrow.telecommunity.com>
	<1d85506f0701011438q6d8b65bfi4ef1e8f9fa509650@mail.gmail.com>
	<ca471dc20701011630l633130e0n356eb9b176f1d610@mail.gmail.com>
	<fb6fbf560701021220h4cb1b656u21c83d1b8b916fe5@mail.gmail.com>
	<ca471dc20701021228k592aced9g2244040f7ebc5b00@mail.gmail.com>
	<aac2c7cb0701030941o7e7d72dai76861554011fd3a8@mail.gmail.com>
	<Pine.LNX.4.58.0701032052370.16811@server1.LFW.org>
	<5.1.1.6.0.20070103225854.042a3868@sparrow.telecommunity.com>
	<5.1.1.6.0.20070104004429.02a19820@sparrow.telecommunity.com>
Message-ID: <Pine.LNX.4.58.0701040139090.16811@server1.LFW.org>

On 1/3/07, Phillip J. Eby <pje at telecommunity.com> wrote:
>Or perhaps translate blocks of the form:
>
>      except ExcType, e:
>          # body
>
>to:
>
>      except ExcType, e:
>          try:
>              # body
>          finally:
>              del e

That's much nicer!  (I feel silly now for not realizing there was
no need for the weakref...)

> Actually, on second thought it occurs to me that the above code isn't a
> 100% correct translation, because if "body" contains its own "del e", then
> it will fail.  So a pure translation strategy isn't really practical,
> unfortunately.  We'd have to generate something like:
>
>      if 'e' in locals():
>          del e

How about this?

     except ExcType, e:
         try:
             # body
         finally:
             e = None


-- ?!ng

From bjourne at gmail.com  Thu Jan  4 09:06:08 2007
From: bjourne at gmail.com (=?ISO-8859-1?Q?BJ=F6rn_Lindqvist?=)
Date: Thu, 4 Jan 2007 09:06:08 +0100
Subject: [Python-3000] PEP 3108: Standard Library Reorganization
In-Reply-To: <bbaeab100701031307v7cd62e6at34c095c0f84b8d2e@mail.gmail.com>
References: <bbaeab100701011614l27a87f45pb7e782c1d78a0725@mail.gmail.com>
	<12C8E611-8CBA-43A3-A70B-98A04FEB7B16@mac.com>
	<bbaeab100701031307v7cd62e6at34c095c0f84b8d2e@mail.gmail.com>
Message-ID: <740c3aec0701040006ydfece1udef4d8e3787e10a7@mail.gmail.com>

On 1/3/07, Brett Cannon <brett at python.org> wrote:
> > > PEP 8 specifies that modules "should have short, lowercase names,
> > > without underscores" [#pep-0008]_.
> >
> > Why does the restriction on underscores exist? Removing that
> > restriction would make lowercase-only names easier to use
> > (basehttpserver vs. base_http_server).
>
> I think Guido said he preferred it that way.

Or maybe he changed his mind?
http://mail.python.org/pipermail/python-3000/2006-December/005234.html

-- 
mvh Bj?rn

From barry at python.org  Thu Jan  4 14:08:44 2007
From: barry at python.org (Barry Warsaw)
Date: Thu, 4 Jan 2007 08:08:44 -0500
Subject: [Python-3000] PEP 3108: Standard Library Reorganization
In-Reply-To: <740c3aec0701040006ydfece1udef4d8e3787e10a7@mail.gmail.com>
References: <bbaeab100701011614l27a87f45pb7e782c1d78a0725@mail.gmail.com>
	<12C8E611-8CBA-43A3-A70B-98A04FEB7B16@mac.com>
	<bbaeab100701031307v7cd62e6at34c095c0f84b8d2e@mail.gmail.com>
	<740c3aec0701040006ydfece1udef4d8e3787e10a7@mail.gmail.com>
Message-ID: <F6CF931B-A0CB-4B02-8766-7F7090B819C4@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Jan 4, 2007, at 3:06 AM, BJ?rn Lindqvist wrote:

> On 1/3/07, Brett Cannon <brett at python.org> wrote:
>>>> PEP 8 specifies that modules "should have short, lowercase names,
>>>> without underscores" [#pep-0008]_.
>>>
>>> Why does the restriction on underscores exist? Removing that
>>> restriction would make lowercase-only names easier to use
>>> (basehttpserver vs. base_http_server).
>>
>> I think Guido said he preferred it that way.
>
> Or maybe he changed his mind?
> http://mail.python.org/pipermail/python-3000/2006-December/005234.html

I propose to change PEP 8 in the following way:

     Package and Module Names

       Modules should have short, all-lowercase names.  Underscores  
can be used
       in the module name if it improves readability.  Python  
packages should
       also have short, all-lowercase names, although the use of  
underscores is
       discouraged.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (Darwin)

iQCVAwUBRZz8XHEjvBPtnXfVAQIUmQQAskVOciDaEabVZMhOg8dzi2gN6+h2DKsc
wVsP2QA6RcHB9vVOQKWBXoJcSWT1pqwBbDHvNKKxH0CQohf+HapOCzU/67ZrJaiG
G60MXF/I9YBkBPfSGi1iaJIbWzSUd8SwCDGxnIEczzLABcvO1x5Y10eX2iSSTTtI
S2VIgbctExQ=
=ptKu
-----END PGP SIGNATURE-----

From pje at telecommunity.com  Thu Jan  4 16:00:10 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 04 Jan 2007 10:00:10 -0500
Subject: [Python-3000] self-contained exceptions
In-Reply-To: <Pine.LNX.4.58.0701040139090.16811@server1.LFW.org>
References: <5.1.1.6.0.20070104004429.02a19820@sparrow.telecommunity.com>
	<5.1.1.6.0.20070103225854.042a3868@sparrow.telecommunity.com>
	<1d85506f0701011438q6d8b65bfi4ef1e8f9fa509650@mail.gmail.com>
	<ca471dc20701011630l633130e0n356eb9b176f1d610@mail.gmail.com>
	<fb6fbf560701021220h4cb1b656u21c83d1b8b916fe5@mail.gmail.com>
	<ca471dc20701021228k592aced9g2244040f7ebc5b00@mail.gmail.com>
	<aac2c7cb0701030941o7e7d72dai76861554011fd3a8@mail.gmail.com>
	<Pine.LNX.4.58.0701032052370.16811@server1.LFW.org>
	<5.1.1.6.0.20070103225854.042a3868@sparrow.telecommunity.com>
	<5.1.1.6.0.20070104004429.02a19820@sparrow.telecommunity.com>
Message-ID: <5.1.1.6.0.20070104095412.029577a8@sparrow.telecommunity.com>

At 01:41 AM 1/4/2007 -0600, Ka-Ping Yee wrote:
>How about this?
>
>      except ExcType, e:
>          try:
>              # body
>          finally:
>              e = None

It's a little bit more difficult to explain in the reference manual.  I was 
figuring we'd say that the exception variable is bound only in the body of 
the except clause; saying it becomes None when you exit sounds weirder to 
me.  How about:

      except ExcType, e:
          try:
              # body
          finally:
              e = None
              del e

Then we get the best of all three worlds: a clean explanation, a clean 
implementation, and a pure source-to-source transformation.

Heck, you could even say the translation is:

      except ExcType:
          e = sys.exception  # or whatever it's called in 3K
          try:
              # body
          finally:
              e = None
              del e

Although I suppose you can't actually implement it that way.  :)


From tomerfiliba at gmail.com  Thu Jan  4 18:18:12 2007
From: tomerfiliba at gmail.com (tomer filiba)
Date: Thu, 4 Jan 2007 19:18:12 +0200
Subject: [Python-3000] self-contained exceptions
In-Reply-To: <ca471dc20701011630l633130e0n356eb9b176f1d610@mail.gmail.com>
References: <1d85506f0701011438q6d8b65bfi4ef1e8f9fa509650@mail.gmail.com>
	<ca471dc20701011630l633130e0n356eb9b176f1d610@mail.gmail.com>
Message-ID: <1d85506f0701040918w5d3ca9edh6137e574a891b5c3@mail.gmail.com>

[Guido]
> > as well as
> > traceback.format_exception (which i greatly dislike).
>
> Why? Because it once insulted you?

because it's tedious, repetitive and unreadable.

try:
    foo
except:
    import traceback
    import sys
    tbtext = "".join(traceback.format_exception(*sys.exc_info()))
    raise RemoteException(tbtext)

it ought to have at least a nicer API, for instance,
tbtext = traceback.format_current()


-tomer

On 1/2/07, Guido van Rossum <guido at python.org> wrote:
> On 1/1/07, tomer filiba <tomerfiliba at gmail.com> wrote:
> > a while back some people suggested having the traceback
> > as an attribute of the exception object, which allows for
> > nested exceptions.
>
> PEP 344, to be precise.
>
> > i would also like to note that having the traceback part of
> > the exception object allows the exception to print itself,
> > i.e., its repr would format the traceback text on it's own.
>
> I'm not sure that's an improvement. I like to b able to have different
> ways of formatting exceptions. And a repr() that's as long as a
> typical traceback isn't a great idea either. Note that currently
> there's nothing to prevent tracebacks from having a repr() that prints
> the exception, and yet this isn't what it does.
>
> > this makes sys.excepthook unnecessary,
>
> Not at all. sys.excepthook can do other things besides formatting the
> exception, and when it does format the exception, it can do so in any
> number of interesting alternative ways (e.g. cgitb.py).
>
> > as well as
> > traceback.format_exception (which i greatly dislike).
>
> Why? Because it once insulted you?
>
> > even if the exception's repr would raise an exception
> > (recursion limit, etc.), that exception would be a builtin
> > one, so we won't end up in a repr-exception-loop.
> >
> > with this approach, sys.excepthook is needless, or at least
> > simplified greatly (just print repr(exc)). for example, exceptions
> > raised by RPyC have a special attribute to them -- remote_traceback,
> > which is the text of the remote traceback. currently i have to
> > replace excepthook to have it displayed with the local exception
> > traceback, which is means just importing RPyC messes with
> > your interpreter's internals, and may cause problems when
> > another library installs its own hook.
> >
> > apart from self-repr'ing, nested exceptions are very useful too:
> >
> > try:
> >     foo()
> > except Exception, ex:
> >     raise MyException("something happened", nested = ex)
>
> It really looks like you've never read PEP 344, or you wouldn't be
> introducing a different way to spell what it already proposes.
>
> > so that when i pdb.pm() my code, i can inspect the inner
> > exception too (which is desirable of course).
> >
> > the major argument against having the traceback part of
> > the exception object was increasing memory consumption,
> > so that exceptions are no longer "light weight" (as used by
> > StopIteration, etc.)
>
> Where did you hear this? It is not *my* objection.
>
> > i don't see why this argument stands. the traceback object
> > exists either way, the only question is in what namespace
> > it exists, and the fact it may "live longer" than necessary,
> > i.e., the traceback exists for as long as the exception object
> > exists.
> >
> > i don't think it's really that bad. first, for StopIteration, the exception
> > object is decref()ed immediately after being raised. moreover,
> > since no one is expected to see that traceback, it can be set
> > directly to None, i.e., a PyErr_SetStopIter() function which sets
> > exc.traceback to None internally.
> >
> > other than that, you *want* to keep the traceback alive. therefore,
> > i suggest adding two new attributes to the exception class:
> > * traceback - the traceback object, or None
> > * nested - nested ("inner") exception, or None
> >
> > Exception.__repr__ will be extended to formatting the
> > traceback (if not None) and formatting all the nested
> > exceptions (+ detect recursion).
> >
> > this may be an expensive operation, but as it only happens
> > when the exceptions is printed to stderr, it's negligible.
>
> I believe the main argument *against* having the tb included in the
> exception is that it would greatly increase the cyclic nature of
> tracebacks and stack frames, which in turn would cause lots of code to
> break due to late GC'ing of unclosed files, etc. For example, this
> code:
>
> try:
>   1/0
> except Exception, err:
>   pass
>
> will introduce a cycle from err -> traceback -> stack frame -> err,
> keeping all locals in the same scope alive until the next GC happens.
>
> --
> --Guido van Rossum (home page: http://www.python.org/~guido/)
>

From guido at python.org  Thu Jan  4 19:21:28 2007
From: guido at python.org (Guido van Rossum)
Date: Thu, 4 Jan 2007 10:21:28 -0800
Subject: [Python-3000] self-contained exceptions
In-Reply-To: <1d85506f0701040918w5d3ca9edh6137e574a891b5c3@mail.gmail.com>
References: <1d85506f0701011438q6d8b65bfi4ef1e8f9fa509650@mail.gmail.com>
	<ca471dc20701011630l633130e0n356eb9b176f1d610@mail.gmail.com>
	<1d85506f0701040918w5d3ca9edh6137e574a891b5c3@mail.gmail.com>
Message-ID: <ca471dc20701041021p148eeeb2rb49027ef7bfe94a1@mail.gmail.com>

[Tomer]
> > > as well as
> > > traceback.format_exception (which i greatly dislike).

[Guido]
> > Why? Because it once insulted you?

[Tomer]
> because it's tedious, repetitive and unreadable.
>
> try:
>     foo
> except:
>     import traceback
>     import sys
>     tbtext = "".join(traceback.format_exception(*sys.exc_info()))
>     raise RemoteException(tbtext)
>
> it ought to have at least a nicer API, for instance,
> tbtext = traceback.format_current()

I don't really think you can blame format_exception() for returning a
list. The use case I am guessing from your example (passing a
traceback as a single string across an RPC boundary) isn't all that
common and you ought to have only one place in your RPC package where
you need to call it.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Thu Jan  4 19:23:16 2007
From: guido at python.org (Guido van Rossum)
Date: Thu, 4 Jan 2007 10:23:16 -0800
Subject: [Python-3000] self-contained exceptions
In-Reply-To: <5.1.1.6.0.20070104095412.029577a8@sparrow.telecommunity.com>
References: <1d85506f0701011438q6d8b65bfi4ef1e8f9fa509650@mail.gmail.com>
	<ca471dc20701011630l633130e0n356eb9b176f1d610@mail.gmail.com>
	<fb6fbf560701021220h4cb1b656u21c83d1b8b916fe5@mail.gmail.com>
	<ca471dc20701021228k592aced9g2244040f7ebc5b00@mail.gmail.com>
	<aac2c7cb0701030941o7e7d72dai76861554011fd3a8@mail.gmail.com>
	<Pine.LNX.4.58.0701032052370.16811@server1.LFW.org>
	<5.1.1.6.0.20070103225854.042a3868@sparrow.telecommunity.com>
	<5.1.1.6.0.20070104004429.02a19820@sparrow.telecommunity.com>
	<Pine.LNX.4.58.0701040139090.16811@server1.LFW.org>
	<5.1.1.6.0.20070104095412.029577a8@sparrow.telecommunity.com>
Message-ID: <ca471dc20701041023q4fabfc76h34ed4a7b491774f4@mail.gmail.com>

On 1/4/07, Phillip J. Eby <pje at telecommunity.com> wrote:
> At 01:41 AM 1/4/2007 -0600, Ka-Ping Yee wrote:
> >How about this?
> >
> >      except ExcType, e:
> >          try:
> >              # body
> >          finally:
> >              e = None
>
> It's a little bit more difficult to explain in the reference manual.  I was
> figuring we'd say that the exception variable is bound only in the body of
> the except clause; saying it becomes None when you exit sounds weirder to
> me.  How about:
>
>       except ExcType, e:
>           try:
>               # body
>           finally:
>               e = None
>               del e
>
> Then we get the best of all three worlds: a clean explanation, a clean
> implementation, and a pure source-to-source transformation.

Great! We need (a) an update to PEP 3100 and (b) a patch to implement
this. (Once we have this we can talk about the rest of PEP 344;
perhaps that PEP should also target Py3k only?)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From ntoronto at cs.byu.edu  Thu Jan  4 20:13:51 2007
From: ntoronto at cs.byu.edu (Neil Toronto)
Date: Thu, 04 Jan 2007 12:13:51 -0700
Subject: [Python-3000] self-contained exceptions
In-Reply-To: <ca471dc20701041023q4fabfc76h34ed4a7b491774f4@mail.gmail.com>
References: <1d85506f0701011438q6d8b65bfi4ef1e8f9fa509650@mail.gmail.com>	<ca471dc20701011630l633130e0n356eb9b176f1d610@mail.gmail.com>	<fb6fbf560701021220h4cb1b656u21c83d1b8b916fe5@mail.gmail.com>	<ca471dc20701021228k592aced9g2244040f7ebc5b00@mail.gmail.com>	<aac2c7cb0701030941o7e7d72dai76861554011fd3a8@mail.gmail.com>	<Pine.LNX.4.58.0701032052370.16811@server1.LFW.org>	<5.1.1.6.0.20070103225854.042a3868@sparrow.telecommunity.com>	<5.1.1.6.0.20070104004429.02a19820@sparrow.telecommunity.com>	<Pine.LNX.4.58.0701040139090.16811@server1.LFW.org>	<5.1.1.6.0.20070104095412.029577a8@sparrow.telecommunity.com>
	<ca471dc20701041023q4fabfc76h34ed4a7b491774f4@mail.gmail.com>
Message-ID: <459D51EF.1050606@cs.byu.edu>

Guido van Rossum wrote:
> On 1/4/07, Phillip J. Eby <pje at telecommunity.com> wrote:
>   
>> At 01:41 AM 1/4/2007 -0600, Ka-Ping Yee wrote:
>>     
>>> How about this?
>>>
>>>      except ExcType, e:
>>>          try:
>>>              # body
>>>          finally:
>>>              e = None
>>>       
>> It's a little bit more difficult to explain in the reference manual.  I was
>> figuring we'd say that the exception variable is bound only in the body of
>> the except clause; saying it becomes None when you exit sounds weirder to
>> me.  How about:
>>
>>       except ExcType, e:
>>           try:
>>               # body
>>           finally:
>>               e = None
>>               del e
>>
>> Then we get the best of all three worlds: a clean explanation, a clean
>> implementation, and a pure source-to-source transformation.
>>     
>
> Great! We need (a) an update to PEP 3100 and (b) a patch to implement
> this. (Once we have this we can talk about the rest of PEP 344;
> perhaps that PEP should also target Py3k only?)

Does it bother anybody that the variable e now has only block scope 
rather than function scope like (almost) every other local? Do the 
benefits outweigh the inconsistency?

One could argue that there's precedent with generator comprehensions 
(also list comprehensions in Py3k), but to me this *looks* more like a 
for loop. As a first-time user I would expect it to behave the same way.

Neil


From tomerfiliba at gmail.com  Thu Jan  4 20:33:18 2007
From: tomerfiliba at gmail.com (tomer filiba)
Date: Thu, 4 Jan 2007 21:33:18 +0200
Subject: [Python-3000] self-contained exceptions
In-Reply-To: <ca471dc20701041021p148eeeb2rb49027ef7bfe94a1@mail.gmail.com>
References: <1d85506f0701011438q6d8b65bfi4ef1e8f9fa509650@mail.gmail.com>
	<ca471dc20701011630l633130e0n356eb9b176f1d610@mail.gmail.com>
	<1d85506f0701040918w5d3ca9edh6137e574a891b5c3@mail.gmail.com>
	<ca471dc20701041021p148eeeb2rb49027ef7bfe94a1@mail.gmail.com>
Message-ID: <1d85506f0701041133g31ed75bw1090230b34be8db6@mail.gmail.com>

[Guido]
> The use case I am guessing from your example (passing a
> traceback as a single string across an RPC boundary) isn't all that
> common and you ought to have only one place in your RPC package where
> you need to call it.

*this* use-case isn't that useful, true. but many times, frameworks need
to store the exception details "for later", for example, unit-testing.
you want to run the entire test, and print a report later.

if exceptions formatted themselves (via repr or something like it),
all of these use cases would have been trivial, and there would be
no need for the traceback module at all or excepthook.

(we would still need excepthook, but only in order to redirect printouts
to other file objects other than stderr.)

imho it's more pythonic to have the exception object be self-contained,
rather than having its state scattered around different modules
(sys, exceptions, and traceback). besides, today exceptions are
made of three separate "components" (type, value and tb), which
can all be unified once string exceptions are removed and exceptions
become new style classes.

if pep 344 is going to solve that, of course i'm +1 and there's no need
for a "competing" version... but the pep has been dormant for so long
that i thought it was neglected.

is this pep going to make it for py3k? any pronouncement on the subject?


-tomer

From brett at python.org  Thu Jan  4 20:36:42 2007
From: brett at python.org (Brett Cannon)
Date: Thu, 4 Jan 2007 11:36:42 -0800
Subject: [Python-3000] self-contained exceptions
In-Reply-To: <459D51EF.1050606@cs.byu.edu>
References: <1d85506f0701011438q6d8b65bfi4ef1e8f9fa509650@mail.gmail.com>
	<ca471dc20701021228k592aced9g2244040f7ebc5b00@mail.gmail.com>
	<aac2c7cb0701030941o7e7d72dai76861554011fd3a8@mail.gmail.com>
	<Pine.LNX.4.58.0701032052370.16811@server1.LFW.org>
	<5.1.1.6.0.20070103225854.042a3868@sparrow.telecommunity.com>
	<5.1.1.6.0.20070104004429.02a19820@sparrow.telecommunity.com>
	<Pine.LNX.4.58.0701040139090.16811@server1.LFW.org>
	<5.1.1.6.0.20070104095412.029577a8@sparrow.telecommunity.com>
	<ca471dc20701041023q4fabfc76h34ed4a7b491774f4@mail.gmail.com>
	<459D51EF.1050606@cs.byu.edu>
Message-ID: <bbaeab100701041136r5725d35bsf782019f9d11054b@mail.gmail.com>

On 1/4/07, Neil Toronto <ntoronto at cs.byu.edu> wrote:
>
> Guido van Rossum wrote:
> > On 1/4/07, Phillip J. Eby <pje at telecommunity.com> wrote:
> >
> >> At 01:41 AM 1/4/2007 -0600, Ka-Ping Yee wrote:
> >>
> >>> How about this?
> >>>
> >>>      except ExcType, e:
> >>>          try:
> >>>              # body
> >>>          finally:
> >>>              e = None
> >>>
> >> It's a little bit more difficult to explain in the reference manual.  I
> was
> >> figuring we'd say that the exception variable is bound only in the body
> of
> >> the except clause; saying it becomes None when you exit sounds weirder
> to
> >> me.  How about:
> >>
> >>       except ExcType, e:
> >>           try:
> >>               # body
> >>           finally:
> >>               e = None
> >>               del e
> >>
> >> Then we get the best of all three worlds: a clean explanation, a clean
> >> implementation, and a pure source-to-source transformation.
> >>
> >
> > Great! We need (a) an update to PEP 3100 and (b) a patch to implement
> > this. (Once we have this we can talk about the rest of PEP 344;
> > perhaps that PEP should also target Py3k only?)
>
> Does it bother anybody that the variable e now has only block scope
> rather than function scope like (almost) every other local? Do the
> benefits outweigh the inconsistency?


Nope, doesn't bother me.

One could argue that there's precedent with generator comprehensions
> (also list comprehensions in Py3k), but to me this *looks* more like a
> for loop. As a first-time user I would expect it to behave the same way.


Well, how often does one reuse the exception variable past the 'execpt'
clause?  I admit I have used the leaked loop variant from a 'for' loop on
occasion, but that is not a big deal since I could get around it very
easily.

I think having the docs explicitly state the variable is local to the
'except' clause only is not that hard of a thing to grap.  We can also say
that if people really need the exception to stick around they can assign it
to another variable.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20070104/385ebd4d/attachment-0001.htm 

From brett at python.org  Thu Jan  4 20:38:18 2007
From: brett at python.org (Brett Cannon)
Date: Thu, 4 Jan 2007 11:38:18 -0800
Subject: [Python-3000] PEP 3108: Standard Library Reorganization
In-Reply-To: <F6CF931B-A0CB-4B02-8766-7F7090B819C4@python.org>
References: <bbaeab100701011614l27a87f45pb7e782c1d78a0725@mail.gmail.com>
	<12C8E611-8CBA-43A3-A70B-98A04FEB7B16@mac.com>
	<bbaeab100701031307v7cd62e6at34c095c0f84b8d2e@mail.gmail.com>
	<740c3aec0701040006ydfece1udef4d8e3787e10a7@mail.gmail.com>
	<F6CF931B-A0CB-4B02-8766-7F7090B819C4@python.org>
Message-ID: <bbaeab100701041138t4a735121pe83cb696242da3ec@mail.gmail.com>

On 1/4/07, Barry Warsaw <barry at python.org> wrote:
>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On Jan 4, 2007, at 3:06 AM, BJ?rn Lindqvist wrote:
>
> > On 1/3/07, Brett Cannon <brett at python.org> wrote:
> >>>> PEP 8 specifies that modules "should have short, lowercase names,
> >>>> without underscores" [#pep-0008]_.
> >>>
> >>> Why does the restriction on underscores exist? Removing that
> >>> restriction would make lowercase-only names easier to use
> >>> (basehttpserver vs. base_http_server).
> >>
> >> I think Guido said he preferred it that way.
> >
> > Or maybe he changed his mind?
> > http://mail.python.org/pipermail/python-3000/2006-December/005234.html
>
> I propose to change PEP 8 in the following way:
>
>      Package and Module Names
>
>        Modules should have short, all-lowercase names.  Underscores
> can be used
>        in the module name if it improves readability.  Python
> packages should
>        also have short, all-lowercase names, although the use of
> underscores is
>        discouraged.


+1 from me.  I would just mention the rule should extends to modules within
packages.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20070104/302de0fc/attachment.html 

From sluggoster at gmail.com  Thu Jan  4 22:22:39 2007
From: sluggoster at gmail.com (Mike Orr)
Date: Thu, 4 Jan 2007 13:22:39 -0800
Subject: [Python-3000] self-contained exceptions
In-Reply-To: <1d85506f0701041133g31ed75bw1090230b34be8db6@mail.gmail.com>
References: <1d85506f0701011438q6d8b65bfi4ef1e8f9fa509650@mail.gmail.com>
	<ca471dc20701011630l633130e0n356eb9b176f1d610@mail.gmail.com>
	<1d85506f0701040918w5d3ca9edh6137e574a891b5c3@mail.gmail.com>
	<ca471dc20701041021p148eeeb2rb49027ef7bfe94a1@mail.gmail.com>
	<1d85506f0701041133g31ed75bw1090230b34be8db6@mail.gmail.com>
Message-ID: <6e9196d20701041322w7757bcf4w3a0630ed139788d6@mail.gmail.com>

On 1/4/07, tomer filiba <tomerfiliba at gmail.com> wrote:
> [Guido]
> > The use case I am guessing from your example (passing a
> > traceback as a single string across an RPC boundary) isn't all that
> > common and you ought to have only one place in your RPC package where
> > you need to call it.
>
> *this* use-case isn't that useful, true. but many times, frameworks need
> to store the exception details "for later", for example, unit-testing.
> you want to run the entire test, and print a report later.

Except that different applications want them formatted differently.
No stack trace, max N levels stack trace, etc.  I frequently use
"Caught exception_name: exception_value" when looping over records
from a file so I can see all the exceptions for all records rather
than aborting at the first one.  Yes, one *can* use

    "%s: %s" % (str(e.__class), str(e)

and sometimes I do, but it looks like boilerplate code that should be
handled in the stdlib somewhere.  The traceback module almost does
what I want:

    traceback.format_exception_only(e.__class__, e)  ->  list of strings

Though I really wish I didn't have to pass the arguments.  If
traceback.format_exc() could have an argument that suppresses the
"Traceback (most recent call last):", that would also work.  (Why not
use None for this as print_tb does?)

Of course, having two or three formatting methods off the exception
object would also be fine.  The point is that one universal format is
insufficient, especially if it includes a long traceback.

-- 
Mike Orr <sluggoster at gmail.com>

From pje at telecommunity.com  Fri Jan  5 02:56:57 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 04 Jan 2007 20:56:57 -0500
Subject: [Python-3000] self-contained exceptions
In-Reply-To: <459D8B2F.8000706@informatik.uni-freiburg.de>
References: <5.1.1.6.0.20070104095412.029577a8@sparrow.telecommunity.com>
	<5.1.1.6.0.20070104004429.02a19820@sparrow.telecommunity.com>
	<5.1.1.6.0.20070103225854.042a3868@sparrow.telecommunity.com>
	<1d85506f0701011438q6d8b65bfi4ef1e8f9fa509650@mail.gmail.com>
	<ca471dc20701011630l633130e0n356eb9b176f1d610@mail.gmail.com>
	<fb6fbf560701021220h4cb1b656u21c83d1b8b916fe5@mail.gmail.com>
	<ca471dc20701021228k592aced9g2244040f7ebc5b00@mail.gmail.com>
	<aac2c7cb0701030941o7e7d72dai76861554011fd3a8@mail.gmail.com>
	<Pine.LNX.4.58.0701032052370.16811@server1.LFW.org>
	<5.1.1.6.0.20070103225854.042a3868@sparrow.telecommunity.com>
	<5.1.1.6.0.20070104004429.02a19820@sparrow.telecommunity.com>
	<Pine.LNX.4.58.0701040139090.16811@server1.LFW.org>
	<5.1.1.6.0.20070104095412.029577a8@sparrow.telecommunity.com>
Message-ID: <5.1.1.6.0.20070104205439.04d74ea8@sparrow.telecommunity.com>

At 12:18 AM 1/5/2007 +0100, Malte Helmert wrote:
>Phillip J. Eby wrote:
> > Heck, you could even say the translation is:
> >
> >       except ExcType:
> >           e = sys.exception  # or whatever it's called in 3K
> >           try:
> >               # body
> >           finally:
> >               e = None
> >               del e
> >
> > Although I suppose you can't actually implement it that way.  :)
>
>What should happen in the last line here?
>
>def f():
>     e = "spam"
>     try:
>         1 / 0
>     except Exception, e:
>         print e
>     print e

It should fail with an UnboundLocalError; anything else (IMO) would 
encourage "overly clever" code that's hard for a reader to decipher.


From anthony at interlink.com.au  Fri Jan  5 07:23:20 2007
From: anthony at interlink.com.au (Anthony Baxter)
Date: Fri, 5 Jan 2007 17:23:20 +1100
Subject: [Python-3000] PEP 3108: Standard Library Reorganization
In-Reply-To: <bbaeab100701011614l27a87f45pb7e782c1d78a0725@mail.gmail.com>
References: <bbaeab100701011614l27a87f45pb7e782c1d78a0725@mail.gmail.com>
Message-ID: <200701051723.20998.anthony@interlink.com.au>


Starting a new thread...

> Several modules in the stdlib work on a specific file format.  It 
> is possible some of these formats are no longer used and thus the 
> stdlib modules for them can go.  Below is a list of some modules 
> which rely on a file format that may be obsolete.
> audioop
>     Raw (8|16|32)-bit wide audio files (as generated by the al and
>     sunaudiodev modules).
> chunk
>     AIFF, AIFF-C, and RMFF audio files.
> sunau
>     Sun AU audio files [#sun-au]_.

audioop and sunau disappearing would make me _very_ _very_ unhappy. 
audioop in particular- I make heavy use of it in a number of 
projects (both opensource and non).

The sunau format is still used in quite a number of places - for 
instance, it's the default format for cisco access gateways, as 
well as a bunch of other hardware. 

I could see the justification for moving all of the audio-specific
modules (these ones, aifc, wave, sndhdr) into a new toplevel 'audio' 
package. Although 'chunk' is also used for some forms of image 
file, IIRC. If we did move it to a new package, I could also see 
adding a new module that worked in a format-independent way (since 
aifc, sunau and wave all have compatible interfaces).


From talin at acm.org  Fri Jan  5 08:51:01 2007
From: talin at acm.org (Talin)
Date: Thu, 04 Jan 2007 23:51:01 -0800
Subject: [Python-3000] PEP 3108: Standard Library Reorganization
In-Reply-To: <200701051723.20998.anthony@interlink.com.au>
References: <bbaeab100701011614l27a87f45pb7e782c1d78a0725@mail.gmail.com>
	<200701051723.20998.anthony@interlink.com.au>
Message-ID: <459E0365.8000201@acm.org>

Anthony Baxter wrote:
> Starting a new thread...
> 
>> Several modules in the stdlib work on a specific file format.  It 
>> is possible some of these formats are no longer used and thus the 
>> stdlib modules for them can go.  Below is a list of some modules 
>> which rely on a file format that may be obsolete.
>> audioop
>>     Raw (8|16|32)-bit wide audio files (as generated by the al and
>>     sunaudiodev modules).
>> chunk
>>     AIFF, AIFF-C, and RMFF audio files.
>> sunau
>>     Sun AU audio files [#sun-au]_.
> 
> audioop and sunau disappearing would make me _very_ _very_ unhappy. 
> audioop in particular- I make heavy use of it in a number of 
> projects (both opensource and non).
> 
> The sunau format is still used in quite a number of places - for 
> instance, it's the default format for cisco access gateways, as 
> well as a bunch of other hardware. 
> 
> I could see the justification for moving all of the audio-specific
> modules (these ones, aifc, wave, sndhdr) into a new toplevel 'audio' 
> package. Although 'chunk' is also used for some forms of image 
> file, IIRC. If we did move it to a new package, I could also see 
> adding a new module that worked in a format-independent way (since 
> aifc, sunau and wave all have compatible interfaces).

Just curious, how much pain would it cause you if these were CheeseShop 
packages instead of built-in?

-- Talin

From nnorwitz at gmail.com  Fri Jan  5 09:01:47 2007
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Fri, 5 Jan 2007 00:01:47 -0800
Subject: [Python-3000] PEP 3107 - Function Annotations
In-Reply-To: <582BC187-758E-4367-B8E3-3755BD5F75D9@pagedna.com>
References: <ca471dc20612271731w53e708e7nea0d21bad77eb316@mail.gmail.com>
	<6B99BFD7-F305-4A69-B7F3-04BAE9A8BC82@PageDNA.com>
	<ee2a432c0612272103w6801f235o26a3859155680d15@mail.gmail.com>
	<ee2a432c0612272253s5ca735ebl9d4f0643f888e4b4@mail.gmail.com>
	<582BC187-758E-4367-B8E3-3755BD5F75D9@pagedna.com>
Message-ID: <ee2a432c0701050001j33822fat28ec543b9590b34a@mail.gmail.com>

On 12/28/06, Tony Lownds <tony at pagedna.com> wrote:
>
> >  * With the modification of MAKE_FUNCTION to be a 32-bit value, this
> > means that EXTENDED_ARG is now used.  This means that the peephole
> > optimizer won't work for the outer function.  I think we should
> > correct this.
>
> Ok. Something like this should fix it in peephople.c. I can post a
> patch.
>
> @@ -535,8 +535,13 @@
>                                  break;
>                          case EXTENDED_ARG:
> -                               goto exitUnchanged;
> +                               if (codestr[i+3] != MAKE_FUNCTION)
> +                                       goto exitUnchanged;
> +                               /* don't visit MAKE_FUNCTION as
> GETARG will be wrong */
> +                               i += 3;
> +                               break;
> +

I'm not sure if it's not that simple or not.  The problem is the jump
fix up code.  There are some assumptions that the code sizes are only
1 or 2.  It's possible that this is a very limited circumstance that
doesn't screw up any of the assumptions.  IIRC there would be problems
if a jump changed from 2 to 4 bytes (or vica versa).  Since the size
can't change for MAKE_FUNCTION, it may be ok in this limited case, but
I'm not sure without reviewing the code in detail.  (It probably is
ok, but I'm not sure.)

Hmm, what would happen if the nested function contained more than 256
args?  Might that mess up the annotation handling?

> >  * I don't understand why there was a new annotations (flag) parameter
> > added in symtable.  My guess was that this was to support lambdas
> > without annotations. If this is true, it would be nicer if we didn't
> > have to special case.  I think there was talk about requiring parens
> > around lambda params.  I don't recall the outcome though.
> >
>
> The flag is there to re-use the logic for visiting parameters, once
> for the
> annotations, once for the parameter names. Those need to be separate
> passes
> because the parameter names are added to the functions symbol table,
> annotations
> to the outer symbol table.

That's good to know.  If there isn't a comment about this in the code,
this info would be good to add.  I don't know if it's clear or not
from the resulting code.  Sometimes it's hard to tell what the code
looks like after applying the patch.  I typically only review the
patch itself, so I miss context.  Though it allows for faster reviews.
:-)

> > Tony, you cleaned up quite a few places in the AST, care to do more
> > cleanup?  I liked your refactorings.
>
> Sure, anything in particular?

Not without looking at the code for a while.  Anything you found
difficult, hard to understand, duplicated, etc.  These would be good
to clean up, if nothing else document more appropriately.  I suspect
quite a few comments are out of date.

> > I also don't love all the names in Grammar.  Guido had mentioned about
> > cleaning up the Grammar.  I don't know if he still wants to or if it
> > can be improved.  It would be nice once all the basic features are in
> > if we could make a cleanup pass for things like Grammar, obsolete
> > APIs, etc.
>
> The only name changes I made in Grammar were made to allow
> the same compiler code to process either lambda's arguments without
> annotations or def's arguments with annotations. fpdef became tfpdef
> and vfpdef, fplist became tfplist and vfplist. tname is a new
> nonterminal
> that holds the annotation itself, and vname is the corresponding version
> without an annotation. "tname" replaced NAME in many places.
>
> Maybe there is a better naming convention that "t" and "v" prefixes? Or
> perhaps "tname/vname" could be improved?

Yeah I think the names could be improved, but I don't have any ideas. :-(

n

From anthony at interlink.com.au  Fri Jan  5 14:49:17 2007
From: anthony at interlink.com.au (Anthony Baxter)
Date: Sat, 6 Jan 2007 00:49:17 +1100
Subject: [Python-3000] PEP 3108: Standard Library Reorganization
In-Reply-To: <459E0365.8000201@acm.org>
References: <bbaeab100701011614l27a87f45pb7e782c1d78a0725@mail.gmail.com>
	<200701051723.20998.anthony@interlink.com.au>
	<459E0365.8000201@acm.org>
Message-ID: <200701060049.19655.anthony@interlink.com.au>

On Friday 05 January 2007 18:51, Talin wrote:
> Just curious, how much pain would it cause you if these were
> CheeseShop packages instead of built-in?

Considerable for the open-source stuff. Windows users, in 
particular, have great difficulty with downloading extensions.
If they also have to build it from source, it's exponentially 
worse.

That's one thing worth noting - if we are proposing to push any 
C-code packages off to the cheeseshop, we need to make sure we make 
available the binary packages for Windows users, and keep doing it 
when there's a new major release, at least if the package has any 
use at all.

Right now, I don't even know that it's possible to build a binary 
extension without spending money - it needs a version of the MS 
compiler that's no longer available. Last I looked, there wasn't a 
free solution that worked (hopefully that's changed now). In any 
case, telling people "now you'll need to download a couple of 
hundred megabytes of compiler" doesn't strike me as a path to 
happiness.

-- 
Anthony Baxter     <anthony at interlink.com.au>
It's never too late to have a happy childhood.

From pje at telecommunity.com  Fri Jan  5 18:18:53 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri, 05 Jan 2007 12:18:53 -0500
Subject: [Python-3000] PEP 3107 - Function Annotations
In-Reply-To: <ee2a432c0612272253s5ca735ebl9d4f0643f888e4b4@mail.gmail.co
 m>
References: <ee2a432c0612272103w6801f235o26a3859155680d15@mail.gmail.com>
	<ca471dc20612271731w53e708e7nea0d21bad77eb316@mail.gmail.com>
	<6B99BFD7-F305-4A69-B7F3-04BAE9A8BC82@PageDNA.com>
	<ee2a432c0612272103w6801f235o26a3859155680d15@mail.gmail.com>
Message-ID: <5.1.1.6.0.20070105121614.04dbe3f8@sparrow.telecommunity.com>

At 10:53 PM 12/27/2006 -0800, Neal Norwitz wrote:
>  * With the modification of MAKE_FUNCTION to be a 32-bit value, this
>means that EXTENDED_ARG is now used.  This means that the peephole
>optimizer won't work for the outer function.  I think we should
>correct this.

Why not just follow the MAKE_FUNCTION with the calculation of the 
annotations, followed by a STORE_ATTR to attach them to the function?  Why 
do we have to change MAKE_FUNCTION for this?
(I.e., can't we just compute the annotation values *after* the function is 
created, but before it's bound to its name?)


From brett at python.org  Fri Jan  5 20:58:08 2007
From: brett at python.org (Brett Cannon)
Date: Fri, 5 Jan 2007 11:58:08 -0800
Subject: [Python-3000] PEP 3108: Standard Library Reorganization
In-Reply-To: <200701051723.20998.anthony@interlink.com.au>
References: <bbaeab100701011614l27a87f45pb7e782c1d78a0725@mail.gmail.com>
	<200701051723.20998.anthony@interlink.com.au>
Message-ID: <bbaeab100701051158u7e6f40e5p22c518852cd497e9@mail.gmail.com>

On 1/4/07, Anthony Baxter <anthony at interlink.com.au> wrote:
>
>
> Starting a new thread...
>
> > Several modules in the stdlib work on a specific file format.  It
> > is possible some of these formats are no longer used and thus the
> > stdlib modules for them can go.  Below is a list of some modules
> > which rely on a file format that may be obsolete.
> > audioop
> >     Raw (8|16|32)-bit wide audio files (as generated by the al and
> >     sunaudiodev modules).
> > chunk
> >     AIFF, AIFF-C, and RMFF audio files.
> > sunau
> >     Sun AU audio files [#sun-au]_.
>
> audioop and sunau disappearing would make me _very_ _very_ unhappy.


Well, I am not about to piss off the release manager.  =)

audioop in particular- I make heavy use of it in a number of
> projects (both opensource and non).
>
> The sunau format is still used in quite a number of places - for
> instance, it's the default format for cisco access gateways, as
> well as a bunch of other hardware.


I just removed that whole section in Open Issues so audioop, sunau, and
chunk are not even being considered for removal.

I could see the justification for moving all of the audio-specific
> modules (these ones, aifc, wave, sndhdr) into a new toplevel 'audio'
> package. Although 'chunk' is also used for some forms of image
> file, IIRC.


Anyone know if this is true? I just want to be sure since I will be
proposing new packages soon.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20070105/0e270825/attachment.html 

From brett at python.org  Fri Jan  5 21:14:27 2007
From: brett at python.org (Brett Cannon)
Date: Fri, 5 Jan 2007 12:14:27 -0800
Subject: [Python-3000] PEP 3108 and modules to be removed (current list)
Message-ID: <bbaeab100701051214h54c0e1f9w73c2e059fafbb04d@mail.gmail.com>

OK, it seems like things have converged on a reasonable list of modules to
be removed.  Here is the current list from the PEP (I cut out all
non-removal stuff for now so this can be a more focused thread).

I think the only module on the list to be removed that is contraversial is
getopt since some people don't want to convert their code over and at least
Laura Creighton has commented on my blog that optparse has some serious
shortcomings (see the comments on getopt on what they are).

If anyone objects to any of the modules, please speak up now so that we can
move on to discussing possible new packages and renaming now that PEP 8 has
been clarified.

---------------------------------------------------------------

Modules to Remove
=================

Guido pronounced that "silly old stuff" is to be deleted from the
stdlib for Py3K [#silly-old-stuff]_.  This is open-ended on purpose.
Each module to be removed needs to have a justification as to why it
should no longer be distributed with Python.  This can range from the
module being deprecated in Python 2.x to being for a platform that is
no longer widely used.

This section of the PEP lists the various modules to be removed. Each
subsection represents a different reason for modules to be
removed.  Each module must have a specific justification on top of
being listed in a specific subsection so as to make sure only modules
that truly deserve to be removed are in fact removed.

When a reason mentions how long it has been since a module has been
"uniquely edited", it is in reference to how long it has been since a
checkin was done specifically for the module and not for a change that
applied universally across the entire stdlib.  If an edit time is not
denoted as "unique" then it is the last time the file was edited,
period.


Previously deprecated
---------------------

Modules in this section have been deprecated at some point in the
Python 2.x release series but are currently still distributed with
Python.  Deprecation information is gathered either from PEP 4 or the
Global Module Index [#pep-0004]_, [#module-index]_.  Each module is
listed with the Python version of initial deprecation.

==============  ==========
Module          Deprecated
==============  ==========
buildtools      2.3
cfmfile         2.4
gopherlib       2.5
macfs           2.3
md5             2.5
mimetools       2.3
MimeWriter      2.3
mimefy          2.3
multifile       2.3
posixfile       1.5
rfc822          2.3
rgbimg          2.5
sha             2.5
==============  ==========


Platform-specific with minimal use
----------------------------------

Python supports many platforms, some of which are not widely held.
And on some of these platforms there are modules that have limited use
to people on those platforms.  Because of their limited usefulness it
would be better to no longer burden the Python development team with
their maintenance.

* IRIX (which is no longer produced [#irix-retirement]_)

  + AL/al

    - Provides sound support on Indy and Indigo workstations.
    - Both workstations are no longer available.
    - Code has not been uniquely edited in three years.

  + cd

    - CD drive control for SGI systems.
    - SGI no longer sells machines with IRIX on them.
    - Code has not been uniquely edited in 14 years.

  + DEVICE/GL/gl/cgen/cgensuport

    - OpenGL access.
    - Has not been edited in at least eight years.
    - Third-party libraries provide better support (PyOpenGL [#pyopengl]_).

  + FL/fl/flp

    - Wrapper for the FORMS library [#irix-forms]_
    - FORMS has not been edited in 12 years.
    - Library is not widely used.
    - First eight hits on Google are for Python docs for fl.

  + fm

    - Wrapper to the IRIS Font Manager library.
    - Only available on SGI machines which no longer come with IRIX.

  + imgfile

    - Wrapper for SGI libimage library for imglib image files
      (``.rgb`` files).
    - Python Imaging Library provdes read-only support [#pil]_.
    - Not uniquely edited in 13 years.

  + jpeg

    - Wrapper for JPEG (de)compressor.
    - Code not uniquely edited in nine years.
    - Third-party libraries provide better support
      (Python Imaging Library [#pil]_).

  + sv

    - Wrapper for Indigo video card.
    - Harware is no longer manufactured.
    - Undocumented.
    - Code not uniquely edited in 13 years.

* Solaris

  + SUNAUDIODEV/sunaudiodev

    - Access to the sound card on Sun machines.
    - Code not uniquely edited in over eight years.

* Mac

  + aetools/aepack/aetypes/MiniAEFrame

    - OSA support is better through third-party modules.

        * Appscript [#appscript]_.

    - Hard-coded endianness which breaks on Intel Macs.
    - Might need to rename if Carbon package dependent.

  + applesingle

    - Undocumented.
    - AppleSingle is a binary file format for A/UX.
    - A/UX no longer distributed.

  + autoGIL

    - Very bad model for using Python with the CFRunLoop.
    - Rename to _autogil if needed by Carbon package.

  + Carbon.CF

    - Half-heared attempt.

  + findertools

    - No longer useful.

  + FrameWork

    - Poorly documented.
    - Not updated to support Carbon Events.

    + icopen

      - Not needed on OS X.
      - Meant to replace 'open' which is usually a bad thing to do.

    + videoreader

      - No longer used.

    + W

      - No longer distributed with Python.


Minimal usage
-------------

Some modules that are platform-independent have minimal usage.  This
can be from how easy it is to implement the functionality from scratch
or because the audience for the code is small.

* audiodev

  + Undocumented.
  + Not edited in five years.
  + If removed sunaudio should go as well (also undocumented; not
    edited in over seven years).

* imputil

  + Undocumented.
  + Never updated to support absolute imports.

* mutex

  + Easy to implement using a semaphore and a queue.
  + Cannot block on a lock attempt.
  + Not uniquely edited since its addition 15 years ago.

* repr

  + Controls output of the repr of objects.
  + String slicing and string interpolation can do similar work.
  + Used by pdb, but do not need to expose API.

* symtable/_symtable

  + Undocumented.

* toaiff

  + Undocumented.
  + Requires ``sox`` library to be installed on the system.

* user

  + Easily handled by allowing the application specify its own
    module name, check for existence, and import if found.

* new

  + Just a rebinding of names from the 'types' module.
  + Can also call ``type`` built-in to get most types easily.

* pure

  + Written before Pure Atria was bought by Rational which was then
    bought by IBM (in other words, very old).


Obsolete
--------

Becoming obsolete signifies that either another module in the stdlib
or a widely distributed third-party library provides a better solution
for what the module is meant for.

* Bastion/rexec

  + Restricted execution / security.
  + Turned off in Python 2.3.
  + Modules deemed unsafe.

* commands/popen2

  + subprocess module replaces them [#pep-0324]_.

* dl

  + ctypes provides better support for same functionality.

* fpformat

  + All functionality is supported by string interpolation.

* getopt

  + optparse provides better functionality.
  + But ...

    - getopt is still widely used.
    - optparse does not handle the issue of when people set
      incompatible options very well.
    - optparse does not allow option arguments to be optional.

* ihooks

  + Documented except for saying that module might be obsolete.
  + For use with rexec which has been turned off since Python 2.3.

* imageop

  + Better support by third-party libraries
    (Python Imaging Library [#pil]_).

* linuxaudiodev

  + Replaced by ossaudiodev.

* stat

  + ``os.stat`` now returns a tuple with attributes.
  + Functions in the module should be made into methods for the object
    returned by os.stat.

* statvfs

  + ``os.statvfs`` now returns a tuple with attributes.

* strop

  + Implements functions used by 'string' module that have now
    become methods on the str type.

* thread

  + People should use 'threading' instead.

    - Rename 'thread' to _thread.
    - Deprecate dummy_thread.

      * Rename to _mockthread.
      * Change of name better reflects modules purpose.

  + Guido has previously supported the deprecation
    [#thread-deprecation]_.

* timing

  + Use timeit or time.
  + Documentation says the module is obsolete [#timing-module]_.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20070105/c896ef20/attachment-0001.htm 

From brett at python.org  Fri Jan  5 21:30:32 2007
From: brett at python.org (Brett Cannon)
Date: Fri, 5 Jan 2007 12:30:32 -0800
Subject: [Python-3000] Comments on PEP 3108
In-Reply-To: <Pine.WNT.4.64.0701051951090.4224@shaolin>
References: <Pine.WNT.4.64.0701051951090.4224@shaolin>
Message-ID: <bbaeab100701051230g5d8a0ac5xe24debbf92728cde@mail.gmail.com>

On 1/5/07, John J Lee <jjl at pobox.com> wrote:
>
> (I'm not subscribed to this list)
>
> > Consolidate dependent modules together into a single module or package?
> > -----------------------------------------------------------------------
> [...]
> > Cookie/cookielib
>
> There is no dependency in either direction between these two, and they do
> quite different things (as ISTR is documented in one or both modules).


Well, considering no one has even commented on that Open Issues section of
the PEP it will probably just be scrapped in favour of some reasonable
packages.

>     + urlparse?
>
> Since RFC 3986 and 3987 have now been published, this should be deprecated
> in favour of module uriparse (once it's been through the review process
> and various known issues with it are resolved).  I think uriparse.py is on
> Python SF patch tracker somewhere (I forget who contributed it, sorry).


Well, if someone wants to do that then urlparse will definitely disappear as
all deprecated modules automatically get dropped in Py3K.

(I'm -1 on the name "uriparse", FWIW, because of the implication that the
> difference between urlparse and uriparse is something to do with the
> difference between URLs and URIs (it is not).  But if urlparse goes away
> fairly quickly, maybe that's not a big deal.)
>
>
> >     + httplib?
>
> Greg Stein pointed out on the web-sig some years back that some of the
> functionality of urllib2 (handling redirections, auth, etc.) could/should
> be added to httplib.  I suspect one could devise a similar system to
> urllib2 handlers (as of Python 2.4) that would nicely allow the redirect
> etc. functionality to be implemented only once, while preserving backwards
> compatibility for both modules.  I may (or may not) work on this myself.


Well, that is beyond the scope of this PEP so I am not going to get into
that.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20070105/d5b9669a/attachment.html 

From janssen at parc.com  Fri Jan  5 21:57:20 2007
From: janssen at parc.com (Bill Janssen)
Date: Fri, 5 Jan 2007 12:57:20 PST
Subject: [Python-3000] PEP 3108 and modules to be removed (current list)
In-Reply-To: <bbaeab100701051214h54c0e1f9w73c2e059fafbb04d@mail.gmail.com> 
References: <bbaeab100701051214h54c0e1f9w73c2e059fafbb04d@mail.gmail.com>
Message-ID: <07Jan5.125728pst."58648"@synergy1.parc.xerox.com>

You know, hashlib, the replacement for sha and md5, was only
introduced in 2.5.  That's bleeding-edge.  I'd be happiest if sha and
md5 stuck around for a few years, to give people time to support older
installations of Python.  After all, Mac OS X still ships with Python
2.3, and lots of Linux installations ship standard with 2.4.

Bill

From nas at arctrix.com  Fri Jan  5 21:58:48 2007
From: nas at arctrix.com (Neil Schemenauer)
Date: Fri, 5 Jan 2007 20:58:48 +0000 (UTC)
Subject: [Python-3000] self-contained exceptions
References: <1d85506f0701011438q6d8b65bfi4ef1e8f9fa509650@mail.gmail.com>
	<ca471dc20701011630l633130e0n356eb9b176f1d610@mail.gmail.com>
	<1d85506f0701040918w5d3ca9edh6137e574a891b5c3@mail.gmail.com>
Message-ID: <enme68$1mc$1@sea.gmane.org>

tomer filiba <tomerfiliba at gmail.com> wrote:
> it ought to have at least a nicer API, for instance,
> tbtext = traceback.format_current()

Do you know about traceback.format_exc()?  I think I added it for
2.4.

  Neil


From brett at python.org  Fri Jan  5 22:31:54 2007
From: brett at python.org (Brett Cannon)
Date: Fri, 5 Jan 2007 13:31:54 -0800
Subject: [Python-3000] PEP 3108 and modules to be removed (current list)
In-Reply-To: <-4637150446973516214@unknownmsgid>
References: <bbaeab100701051214h54c0e1f9w73c2e059fafbb04d@mail.gmail.com>
	<-4637150446973516214@unknownmsgid>
Message-ID: <bbaeab100701051331g1fb49bb5uf93e6d3bcb3a6c75@mail.gmail.com>

On 1/5/07, Bill Janssen <janssen at parc.com> wrote:
>
> You know, hashlib, the replacement for sha and md5, was only
> introduced in 2.5.  That's bleeding-edge.  I'd be happiest if sha and
> md5 stuck around for a few years, to give people time to support older
> installations of Python.  After all, Mac OS X still ships with Python
> 2.3, and lots of Linux installations ship standard with 2.4.


Well, this is only Py3K, not the 2.x series so they are not disappearing
right away.  But there is a clear migration path so I don't think we should
keep those two modules around in Py3K.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20070105/b32dbbb9/attachment.html 

From jimjjewett at gmail.com  Fri Jan  5 23:16:48 2007
From: jimjjewett at gmail.com (Jim Jewett)
Date: Fri, 5 Jan 2007 17:16:48 -0500
Subject: [Python-3000] [Python-checkins] r53276 - peps/trunk/pep-3108.txt
In-Reply-To: <20070105202359.49D491E4002@bag.python.org>
References: <20070105202359.49D491E4002@bag.python.org>
Message-ID: <fb6fbf560701051416u615cbbbcj1c44937834747091@mail.gmail.com>

> +PEP 8 specifies that modules "should have short, all-lowercase names"
> +where "underscores can be used ... if it improves readability"
> +[#pep-0008]_.  The use of underscores is discouraged in package names.

...

> -BaseHTTPServer      basehttpserver
> +BaseHTTPServer      base_http_server
> -CGIHTTPServer       cgihttpserver
> +CGIHTTPServer       cgi_http_server

And so we circle round to (namespace) packages, because is a dot
really any worse than an underscore?

    http.server.base
    http.server.cgi
    http.server.simple

I had thought Guido had said "no", as a matter of scope creep, but
this rename doesn't seem to be any more disruptive than
base_http_server.

On the other hand, I'm used to

    >>> from package.subpackage.module import Class1 as C1, func_abcde as myfunc

If I correctly read a recent note on google standards, then there are
people who can't rename on import, and might be stuck using several
dots (or worse, writing the semi-generic "server.base") on every
reference.

-jJ

From brett at python.org  Fri Jan  5 23:30:21 2007
From: brett at python.org (Brett Cannon)
Date: Fri, 5 Jan 2007 14:30:21 -0800
Subject: [Python-3000] [Python-checkins] r53276 - peps/trunk/pep-3108.txt
In-Reply-To: <fb6fbf560701051416u615cbbbcj1c44937834747091@mail.gmail.com>
References: <20070105202359.49D491E4002@bag.python.org>
	<fb6fbf560701051416u615cbbbcj1c44937834747091@mail.gmail.com>
Message-ID: <bbaeab100701051430y1da403a8yba837a8233b621e8@mail.gmail.com>

On 1/5/07, Jim Jewett <jimjjewett at gmail.com> wrote:
>
> > +PEP 8 specifies that modules "should have short, all-lowercase names"
> > +where "underscores can be used ... if it improves readability"
> > +[#pep-0008]_.  The use of underscores is discouraged in package names.
>
> ...
>
> > -BaseHTTPServer      basehttpserver
> > +BaseHTTPServer      base_http_server
> > -CGIHTTPServer       cgihttpserver
> > +CGIHTTPServer       cgi_http_server
>
> And so we circle round to (namespace) packages, because is a dot
> really any worse than an underscore?
>
>     http.server.base
>     http.server.cgi
>     http.server.simple
>
> I had thought Guido had said "no", as a matter of scope creep,


Basically.  Very shallow packaging is all he has ever said he would allow.

but
> this rename doesn't seem to be any more disruptive than
> base_http_server.


Depends on your perspective.  Granted the refactoring will be no different,
but if you view the hierarchy it just suddenly got deeper with your
packaging.

On the other hand, I'm used to
>
>     >>> from package.subpackage.module import Class1 as C1, func_abcde as
> myfunc
>
> If I correctly read a recent note on google standards, then there are
> people who can't rename on import, and might be stuck using several
> dots (or worse, writing the semi-generic "server.base") on every
> reference.


Well, people might need to change their standards for code that they have
automatically refactored for Py3K.  Plus I am not going to worry about any
one person or company's coding standards.  I am only worried about Python's
coding standard.

The renaming discussion can happen after any new packaging of modules is
agreed upon.  So don't worry too much just yet, Jim.  =)

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20070105/382a9b5d/attachment.htm 

From paul at boddie.org.uk  Fri Jan  5 23:26:42 2007
From: paul at boddie.org.uk (Paul Boddie)
Date: Fri, 5 Jan 2007 23:26:42 +0100
Subject: [Python-3000]  PEP 3108: Standard Library Reorganization
Message-ID: <200701052326.42627.paul@boddie.org.uk>

Brett Cannon wrote:
> As part of my New Years resolution to get all of my current and planned PEPs
> actually written, accepted, and implemented for 2007, here is the stdlib
> reorg PEP.  I have already checked it into svn but I have inlined it below
> for comments.

There's a document on the python.org Wiki which goes into some of the same 
areas as those addressed in the PEP, although things like hierarchy 
reorganisation are presumably beyond what is desired for Python 3.0. See here 
for more:

http://wiki.python.org/moin/CodingProjectIdeas/StandardLibrary/RestructuredStandardLibrary

Paul

From brett at python.org  Sat Jan  6 00:04:39 2007
From: brett at python.org (Brett Cannon)
Date: Fri, 5 Jan 2007 15:04:39 -0800
Subject: [Python-3000] PEP 3108: Standard Library Reorganization
In-Reply-To: <200701052326.42627.paul@boddie.org.uk>
References: <200701052326.42627.paul@boddie.org.uk>
Message-ID: <bbaeab100701051504h29f39fe4la217d6d25d60ae2@mail.gmail.com>

On 1/5/07, Paul Boddie <paul at boddie.org.uk> wrote:
>
> Brett Cannon wrote:
> > As part of my New Years resolution to get all of my current and planned
> PEPs
> > actually written, accepted, and implemented for 2007, here is the stdlib
> > reorg PEP.  I have already checked it into svn but I have inlined it
> below
> > for comments.
>
> There's a document on the python.org Wiki which goes into some of the same
> areas as those addressed in the PEP, although things like hierarchy
> reorganisation are presumably beyond what is desired for Python 3.0. See
> here
> for more:
>
>
> http://wiki.python.org/moin/CodingProjectIdeas/StandardLibrary/RestructuredStandardLibrary


I added the link to the PEP.  I will have a look, but you guys may have been
too thorough.  =)  Plus there are renaming suggestions that I don't want to
get into (e.g., renaming grp to groups).  But I will have a look and see
which ones make sense to add a possibilities.  I do want to make sure,
though, the the new packages have at least a few modules in it and not just
two.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20070105/7182607a/attachment.htm 

From janssen at parc.com  Sat Jan  6 00:10:48 2007
From: janssen at parc.com (Bill Janssen)
Date: Fri, 5 Jan 2007 15:10:48 PST
Subject: [Python-3000] [Python-checkins] r53276 - peps/trunk/pep-3108.txt
In-Reply-To: <fb6fbf560701051416u615cbbbcj1c44937834747091@mail.gmail.com> 
References: <20070105202359.49D491E4002@bag.python.org>
	<fb6fbf560701051416u615cbbbcj1c44937834747091@mail.gmail.com>
Message-ID: <07Jan5.151053pst."58648"@synergy1.parc.xerox.com>

> > -BaseHTTPServer      basehttpserver
> > +BaseHTTPServer      base_http_server
> > -CGIHTTPServer       cgihttpserver
> > +CGIHTTPServer       cgi_http_server
> 
> And so we circle round to (namespace) packages, because is a dot
> really any worse than an underscore?
> 
>     http.server.base
>     http.server.cgi
>     http.server.simple
> 

We should probably scrap either Base or Simple for Py3K.

Bill

From steven.bethard at gmail.com  Sat Jan  6 03:54:50 2007
From: steven.bethard at gmail.com (Steven Bethard)
Date: Fri, 5 Jan 2007 19:54:50 -0700
Subject: [Python-3000] PEP 3108 and modules to be removed (current list)
In-Reply-To: <bbaeab100701051214h54c0e1f9w73c2e059fafbb04d@mail.gmail.com>
References: <bbaeab100701051214h54c0e1f9w73c2e059fafbb04d@mail.gmail.com>
Message-ID: <d11dcfba0701051854h4c4c7083g54a2a2dd176942f5@mail.gmail.com>

On 1/5/07, Brett Cannon <brett at python.org> wrote:
> * getopt
>
>   + optparse provides better functionality.
>   + But ...
>
>     - getopt is still widely used.
>     - optparse does not handle the issue of when people set
>       incompatible options very well.
>     - optparse does not allow option arguments to be optional.

Just wanted to mention that at the `request of c.l.py`_ I'm planning
to write a Python 3000 PEP to either replace or merge optparse with
argparse_. The argparse module is an optparse-like library that
handles both optional and positional arguments, and among other things
supports "optional option arguments". Both optparse and argparse
support incompatible options pretty well AFAICT -- see my `response in
Brett's blog`_ -- so I'm not sure what the middle point above is all
about.

So maybe PEP 3108 should just leave both getopt and optparse around,
and I should add a section in my PEP about how to update getopt code
to optparse/argparse code?

.. _request of c.l.py:
http://mail.python.org/pipermail/python-list/2007-January/420952.html
.. _argparse: http://argparse.python-hosting.com/
.. _response in Brett's blog:
http://sayspy.blogspot.com/2007/01/stdlib-reorg-pep-has-been-added.html#comment-2017147891520627677

STeVe
-- 
I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a
tiny blip on the distant coast of sanity.
        --- Bucky Katt, Get Fuzzy

From jjl at pobox.com  Fri Jan  5 21:12:20 2007
From: jjl at pobox.com (John J Lee)
Date: Fri, 5 Jan 2007 20:12:20 +0000 (GMT Standard Time)
Subject: [Python-3000] Comments on PEP 3108
Message-ID: <Pine.WNT.4.64.0701051951090.4224@shaolin>

(I'm not subscribed to this list)

> Consolidate dependent modules together into a single module or package?
> -----------------------------------------------------------------------
[...]
> Cookie/cookielib

There is no dependency in either direction between these two, and they do 
quite different things (as ISTR is documented in one or both modules).


>     + urlparse?

Since RFC 3986 and 3987 have now been published, this should be deprecated 
in favour of module uriparse (once it's been through the review process 
and various known issues with it are resolved).  I think uriparse.py is on 
Python SF patch tracker somewhere (I forget who contributed it, sorry).

(I'm -1 on the name "uriparse", FWIW, because of the implication that the 
difference between urlparse and uriparse is something to do with the 
difference between URLs and URIs (it is not).  But if urlparse goes away 
fairly quickly, maybe that's not a big deal.)


>     + httplib?

Greg Stein pointed out on the web-sig some years back that some of the 
functionality of urllib2 (handling redirections, auth, etc.) could/should 
be added to httplib.  I suspect one could devise a similar system to 
urllib2 handlers (as of Python 2.4) that would nicely allow the redirect 
etc. functionality to be implemented only once, while preserving backwards 
compatibility for both modules.  I may (or may not) work on this myself.


John

From tony at pagedna.com  Sat Jan  6 20:05:51 2007
From: tony at pagedna.com (Tony Lownds)
Date: Sat, 6 Jan 2007 11:05:51 -0800
Subject: [Python-3000] PEP 3107 - Function Annotations
In-Reply-To: <ee2a432c0701050001j33822fat28ec543b9590b34a@mail.gmail.com>
References: <ca471dc20612271731w53e708e7nea0d21bad77eb316@mail.gmail.com>
	<6B99BFD7-F305-4A69-B7F3-04BAE9A8BC82@PageDNA.com>
	<ee2a432c0612272103w6801f235o26a3859155680d15@mail.gmail.com>
	<ee2a432c0612272253s5ca735ebl9d4f0643f888e4b4@mail.gmail.com>
	<582BC187-758E-4367-B8E3-3755BD5F75D9@pagedna.com>
	<ee2a432c0701050001j33822fat28ec543b9590b34a@mail.gmail.com>
Message-ID: <2CE34B80-FED7-492F-9E26-AE72722A6D3B@pagedna.com>


On Jan 5, 2007, at 12:01 AM, Neal Norwitz wrote:

> On 12/28/06, Tony Lownds <tony at pagedna.com> wrote:
>>
>> >  * With the modification of MAKE_FUNCTION to be a 32-bit value,  
>> this
>> > means that EXTENDED_ARG is now used.  This means that the peephole
>> > optimizer won't work for the outer function.  I think we should
>> > correct this.
>>
>> Ok. Something like this should fix it in peephople.c. I can post a
>> patch.
>>
>> @@ -535,8 +535,13 @@
>>                                  break;
>>                          case EXTENDED_ARG:
>> -                               goto exitUnchanged;
>> +                               if (codestr[i+3] != MAKE_FUNCTION)
>> +                                       goto exitUnchanged;
>> +                               /* don't visit MAKE_FUNCTION as
>> GETARG will be wrong */
>> +                               i += 3;
>> +                               break;
>> +
>
> I'm not sure if it's not that simple or not.  The problem is the jump
> fix up code.  There are some assumptions that the code sizes are only
> 1 or 2.  It's possible that this is a very limited circumstance that
> doesn't screw up any of the assumptions.  IIRC there would be problems
> if a jump changed from 2 to 4 bytes (or vica versa).  Since the size
> can't change for MAKE_FUNCTION, it may be ok in this limited case, but
> I'm not sure without reviewing the code in detail.  (It probably is
> ok, but I'm not sure.)
>

The peephole code bails on code larger than 32767 . All of the peephole
optimizations make the code smaller. In practice that means EXTENDED_ARG
will not be encountered before jump opcodes. I believe the logic is  
fine even if
EXTENDED_ARG did pop up on any other opcode.

> Hmm, what would happen if the nested function contained more than 256
> args?  Might that mess up the annotation handling?

There is a limit for non-nested arguments, but apparently not for
nested arguments, and the handling does mess up:

 >>> s = "def f((%s)): pass"
 >>> s %= ', '.join('a%d' % i for i in xrange(70000))
 >>> d = {}
 >>> exec(s, d)
 >>> s = "def f((%s)): pass"
 >>> s %= ', '.join('a%d:%d' % (i,i) for i in xrange(70000))
 >>> exec(s, d)
python: Python/ceval.c:2307: PyEval_EvalFrameEx: Assertion  
`num_annotations == name_ix+1' failed.
Abort (core dumped)

compile.c happily compiled 70000 annotations; I think it should  
complain (SyntaxError) at 2^16-1.
I'll post a patch.

Thanks
-Tony



From tony at pagedna.com  Sat Jan  6 20:13:13 2007
From: tony at pagedna.com (Tony Lownds)
Date: Sat, 6 Jan 2007 11:13:13 -0800
Subject: [Python-3000] PEP 3107 - Function Annotations
In-Reply-To: <5.1.1.6.0.20070105121614.04dbe3f8@sparrow.telecommunity.com>
References: <ee2a432c0612272103w6801f235o26a3859155680d15@mail.gmail.com>
	<ca471dc20612271731w53e708e7nea0d21bad77eb316@mail.gmail.com>
	<6B99BFD7-F305-4A69-B7F3-04BAE9A8BC82@PageDNA.com>
	<ee2a432c0612272103w6801f235o26a3859155680d15@mail.gmail.com>
	<5.1.1.6.0.20070105121614.04dbe3f8@sparrow.telecommunity.com>
Message-ID: <A5A502ED-22A3-407C-89D1-488007E2C7C8@pagedna.com>


On Jan 5, 2007, at 9:18 AM, Phillip J. Eby wrote:

> At 10:53 PM 12/27/2006 -0800, Neal Norwitz wrote:
>>  * With the modification of MAKE_FUNCTION to be a 32-bit value, this
>> means that EXTENDED_ARG is now used.  This means that the peephole
>> optimizer won't work for the outer function.  I think we should
>> correct this.
>
> Why not just follow the MAKE_FUNCTION with the calculation of the  
> annotations, followed by a STORE_ATTR to attach them to the  
> function?  Why do we have to change MAKE_FUNCTION for this?
> (I.e., can't we just compute the annotation values *after* the  
> function is created, but before it's bound to its name?)

Yes, it's possible, I tried the approach and it was more C code, more  
bytecode, and it ran slower.

-Tony



From pje at telecommunity.com  Sat Jan  6 20:45:20 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sat, 06 Jan 2007 14:45:20 -0500
Subject: [Python-3000] PEP 3107 - Function Annotations
In-Reply-To: <A5A502ED-22A3-407C-89D1-488007E2C7C8@pagedna.com>
References: <5.1.1.6.0.20070105121614.04dbe3f8@sparrow.telecommunity.com>
	<ee2a432c0612272103w6801f235o26a3859155680d15@mail.gmail.com>
	<ca471dc20612271731w53e708e7nea0d21bad77eb316@mail.gmail.com>
	<6B99BFD7-F305-4A69-B7F3-04BAE9A8BC82@PageDNA.com>
	<ee2a432c0612272103w6801f235o26a3859155680d15@mail.gmail.com>
	<5.1.1.6.0.20070105121614.04dbe3f8@sparrow.telecommunity.com>
Message-ID: <5.1.1.6.0.20070106143827.028db758@sparrow.telecommunity.com>

At 11:13 AM 1/6/2007 -0800, Tony Lownds wrote:

>On Jan 5, 2007, at 9:18 AM, Phillip J. Eby wrote:
>
>>At 10:53 PM 12/27/2006 -0800, Neal Norwitz wrote:
>>>  * With the modification of MAKE_FUNCTION to be a 32-bit value, this
>>>means that EXTENDED_ARG is now used.  This means that the peephole
>>>optimizer won't work for the outer function.  I think we should
>>>correct this.
>>
>>Why not just follow the MAKE_FUNCTION with the calculation of the
>>annotations, followed by a STORE_ATTR to attach them to the
>>function?  Why do we have to change MAKE_FUNCTION for this?
>>(I.e., can't we just compute the annotation values *after* the
>>function is created, but before it's bound to its name?)
>
>Yes, it's possible, I tried the approach and it was more C code, more
>bytecode, and it ran slower.

...and doesn't require EXTENDED_ARG, doesn't introduce a new 
backward-compatibility issue for PyPy, Psyco, or other bytecode-based tools 
(I'm assuming here we'll want to backport annotations to 2.x) or require 
changes to the optimizer.

The additional bytecode and slowdown, IIUC, amounts to a few stack 
manipulations per annotation; the computation of the annotation values 
themselves should still be the bulk of the time used, unless the values are 
constants.


From tony at pagedna.com  Sat Jan  6 20:57:31 2007
From: tony at pagedna.com (Tony Lownds)
Date: Sat, 6 Jan 2007 11:57:31 -0800
Subject: [Python-3000] PEP 3107 - Function Annotations
In-Reply-To: <5.1.1.6.0.20070106143827.028db758@sparrow.telecommunity.com>
References: <5.1.1.6.0.20070105121614.04dbe3f8@sparrow.telecommunity.com>
	<ee2a432c0612272103w6801f235o26a3859155680d15@mail.gmail.com>
	<ca471dc20612271731w53e708e7nea0d21bad77eb316@mail.gmail.com>
	<6B99BFD7-F305-4A69-B7F3-04BAE9A8BC82@PageDNA.com>
	<ee2a432c0612272103w6801f235o26a3859155680d15@mail.gmail.com>
	<5.1.1.6.0.20070105121614.04dbe3f8@sparrow.telecommunity.com>
	<5.1.1.6.0.20070106143827.028db758@sparrow.telecommunity.com>
Message-ID: <BD7E2C17-F55B-403E-9928-AC5BCC9CA572@pagedna.com>


On Jan 6, 2007, at 11:45 AM, Phillip J. Eby wrote:

> At 11:13 AM 1/6/2007 -0800, Tony Lownds wrote:
>
>> On Jan 5, 2007, at 9:18 AM, Phillip J. Eby wrote:
>>
>>> At 10:53 PM 12/27/2006 -0800, Neal Norwitz wrote:
>>>>  * With the modification of MAKE_FUNCTION to be a 32-bit value,  
>>>> this
>>>> means that EXTENDED_ARG is now used.  This means that the peephole
>>>> optimizer won't work for the outer function.  I think we should
>>>> correct this.
>>>
>>> Why not just follow the MAKE_FUNCTION with the calculation of the
>>> annotations, followed by a STORE_ATTR to attach them to the
>>> function?  Why do we have to change MAKE_FUNCTION for this?
>>> (I.e., can't we just compute the annotation values *after* the
>>> function is created, but before it's bound to its name?)
>>
>> Yes, it's possible, I tried the approach and it was more C code, more
>> bytecode, and it ran slower.
>
> ...and doesn't require EXTENDED_ARG,

Is that a problem in and of itself?

> doesn't introduce a new
> backward-compatibility issue for PyPy, Psyco, or other bytecode- 
> based tools
> (I'm assuming here we'll want to backport annotations to 2.x)

Keyword-only arguments also introduces a backward-compatibility  
issue; so
the additional work to understand the annotations portion of  
Python-3000's
MAKE_FUNCTION is not large.

> or require
> changes to the optimizer.

A pretty small change.

> The additional bytecode and slowdown, IIUC, amounts to a few stack
> manipulations per annotation; the computation of the annotation values
> themselves should still be the bulk of the time used, unless the  
> values are
> constants.

Yes, that's probably the case. Also, MAKE_FUNCTION is not often executed
in a loop, so the runtime speed of that opcode probably doesn't  
matter much.

-Tony



From rasky at develer.com  Sun Jan  7 05:03:38 2007
From: rasky at develer.com (Giovanni Bajo)
Date: Sun, 07 Jan 2007 05:03:38 +0100
Subject: [Python-3000] PEP 3108: Standard Library Reorganization
In-Reply-To: <200701060049.19655.anthony@interlink.com.au>
References: <bbaeab100701011614l27a87f45pb7e782c1d78a0725@mail.gmail.com>	<200701051723.20998.anthony@interlink.com.au>	<459E0365.8000201@acm.org>
	<200701060049.19655.anthony@interlink.com.au>
Message-ID: <enprep$mhn$1@sea.gmane.org>

Anthony Baxter wrote:

> Considerable for the open-source stuff. Windows users, in 
> particular, have great difficulty with downloading extensions.

easy_install makes Windows users very happy, though.

> Right now, I don't even know that it's possible to build a binary 
> extension without spending money - it needs a version of the MS 
> compiler that's no longer available. Last I looked, there wasn't a 
> free solution that worked (hopefully that's changed now). 

Yes, you can happily use MinGW nowadays with the standard Python distribution. 
There's an awfully lot of outdated pages about it on Internet, but the real 
truth is that since 2.4 you just need to install MinGW, and tell distutils 
about it (eg: in setup.cfg).

I agree with the general point of your message, though.
-- 
Giovanni Bajo


From rasky at develer.com  Sun Jan  7 05:07:37 2007
From: rasky at develer.com (Giovanni Bajo)
Date: Sun, 07 Jan 2007 05:07:37 +0100
Subject: [Python-3000] PEP 3108: Standard Library Reorganization
In-Reply-To: <07Jan3.091158pst."58648"@synergy1.parc.xerox.com>
References: <20070101174925.82FB.JCARLSON@uci.edu>	<20070101225107.8315.JCARLSON@uci.edu>	<bbaeab100701021547l2eddd412wc11330c4cfcb46dd@mail.gmail.com>	<200701031453.17348.anthony@interlink.com.au>	<dcbbbb410701030623i555f1b27t5f030d4415fb56e@mail.gmail.com>
	<07Jan3.091158pst."58648"@synergy1.parc.xerox.com>
Message-ID: <enprm8$mhn$2@sea.gmane.org>

Bill Janssen wrote:

> Will this work properly in Python 3K?  I tend to use base64, for
> example, to turn byte sequences into safe strings, so I need a way of
> doing "encode" on byte sequences, not just strings.
> 
> In general, base64.encode() should operate on byte sequences and
> return strings, and decode() should operate on strings and return byte
> sequences.

Yes. Not all encoders are unicode->str, and not all decoders are str->unicode. 
This is a point which is often forgotten. Having different functions for 
encoders ("foo".encode.base64()) like someone was proposing in this thread 
would also make it slightly less confusing.
-- 
Giovanni Bajo


From collinw at gmail.com  Mon Jan  8 04:14:15 2007
From: collinw at gmail.com (Collin Winter)
Date: Sun, 7 Jan 2007 21:14:15 -0600
Subject: [Python-3000] self-contained exceptions
In-Reply-To: <ca471dc20701041023q4fabfc76h34ed4a7b491774f4@mail.gmail.com>
References: <1d85506f0701011438q6d8b65bfi4ef1e8f9fa509650@mail.gmail.com>
	<fb6fbf560701021220h4cb1b656u21c83d1b8b916fe5@mail.gmail.com>
	<ca471dc20701021228k592aced9g2244040f7ebc5b00@mail.gmail.com>
	<aac2c7cb0701030941o7e7d72dai76861554011fd3a8@mail.gmail.com>
	<Pine.LNX.4.58.0701032052370.16811@server1.LFW.org>
	<5.1.1.6.0.20070103225854.042a3868@sparrow.telecommunity.com>
	<5.1.1.6.0.20070104004429.02a19820@sparrow.telecommunity.com>
	<Pine.LNX.4.58.0701040139090.16811@server1.LFW.org>
	<5.1.1.6.0.20070104095412.029577a8@sparrow.telecommunity.com>
	<ca471dc20701041023q4fabfc76h34ed4a7b491774f4@mail.gmail.com>
Message-ID: <43aa6ff70701071914j2959f683ncc979bba8be9dc91@mail.gmail.com>

On 1/4/07, Guido van Rossum <guido at python.org> wrote:
> On 1/4/07, Phillip J. Eby <pje at telecommunity.com> wrote:
> > How about:
> >
> >       except ExcType, e:
> >           try:
> >               # body
> >           finally:
> >               e = None
> >               del e
> >
> > Then we get the best of all three worlds: a clean explanation, a clean
> > implementation, and a pure source-to-source transformation.
>
> Great! We need (a) an update to PEP 3100 and (b) a patch to implement
> this. (Once we have this we can talk about the rest of PEP 344;
> perhaps that PEP should also target Py3k only?)

I've just uploaded patch #1630248 to SF. This implements the
transformation (including test cases for the new behavior) and patches
all places in the stdlib that relied on the old behavior.

The transformation is as Phillip outlined above, with extra logic to
handle the case where e is a tuple or list.

Thanks,
Collin Winter

From brett at python.org  Mon Jan  8 06:28:43 2007
From: brett at python.org (Brett Cannon)
Date: Sun, 7 Jan 2007 21:28:43 -0800
Subject: [Python-3000] self-contained exceptions
In-Reply-To: <43aa6ff70701071914j2959f683ncc979bba8be9dc91@mail.gmail.com>
References: <1d85506f0701011438q6d8b65bfi4ef1e8f9fa509650@mail.gmail.com>
	<ca471dc20701021228k592aced9g2244040f7ebc5b00@mail.gmail.com>
	<aac2c7cb0701030941o7e7d72dai76861554011fd3a8@mail.gmail.com>
	<Pine.LNX.4.58.0701032052370.16811@server1.LFW.org>
	<5.1.1.6.0.20070103225854.042a3868@sparrow.telecommunity.com>
	<5.1.1.6.0.20070104004429.02a19820@sparrow.telecommunity.com>
	<Pine.LNX.4.58.0701040139090.16811@server1.LFW.org>
	<5.1.1.6.0.20070104095412.029577a8@sparrow.telecommunity.com>
	<ca471dc20701041023q4fabfc76h34ed4a7b491774f4@mail.gmail.com>
	<43aa6ff70701071914j2959f683ncc979bba8be9dc91@mail.gmail.com>
Message-ID: <bbaeab100701072128m6a08cc5ey9488ffcefca1a23d@mail.gmail.com>

On 1/7/07, Collin Winter <collinw at gmail.com> wrote:
>
> On 1/4/07, Guido van Rossum <guido at python.org> wrote:
> > On 1/4/07, Phillip J. Eby <pje at telecommunity.com> wrote:
> > > How about:
> > >
> > >       except ExcType, e:
> > >           try:
> > >               # body
> > >           finally:
> > >               e = None
> > >               del e
> > >
> > > Then we get the best of all three worlds: a clean explanation, a clean
> > > implementation, and a pure source-to-source transformation.
> >
> > Great! We need (a) an update to PEP 3100 and (b) a patch to implement
> > this. (Once we have this we can talk about the rest of PEP 344;
> > perhaps that PEP should also target Py3k only?)
>
> I've just uploaded patch #1630248 to SF. This implements the
> transformation (including test cases for the new behavior) and patches
> all places in the stdlib that relied on the old behavior.
>
> The transformation is as Phillip outlined above, with extra logic to
> handle the case where e is a tuple or list.


Having 'e' be a tuple or list in Py3K will be rather uncommon?  If you look
at PEP 352 you will notice that the __getitem__ method for BaseException is
going to be gone so the unpacking of an exception will not be something you
can normally do.  If you want to add the support for unpacking when someone
subclasses BaseException and adds __getitem__ that's fine, but in the
general case it won't be used very much I suspect.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20070107/955c00a1/attachment.htm 

From anthony at interlink.com.au  Mon Jan  8 06:50:16 2007
From: anthony at interlink.com.au (Anthony Baxter)
Date: Mon, 8 Jan 2007 16:50:16 +1100
Subject: [Python-3000] Warning for 2.6 and greater
Message-ID: <200701081650.17353.anthony@interlink.com.au>

I've been thinking a little about how and where we'd add warnings to 
2.6 and later for things that will break in 3.0. My first idea is 
to add a command line option '-3' (or maybe '-warn3') implemented 
as "from __future__ import py3k". We can then put code in that 
optionally generates a warning if this is set. 

I'm concerned that we don't make this check too expensive, though - 
but it should be doable. A good proof-of-concept for someone keen 
(I'm totally out of time for the rest of this month) would be to 
whip up something that works and spits out a warning for, say, 
dictobject.c:dict_has_key(), but only when the flag is specified.

Anyone else have alternate ideas?

Anthony
-- 
Anthony Baxter     <anthony at interlink.com.au>
It's never too late to have a happy childhood.

From nnorwitz at gmail.com  Mon Jan  8 07:07:51 2007
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Sun, 7 Jan 2007 22:07:51 -0800
Subject: [Python-3000] Warning for 2.6 and greater
In-Reply-To: <200701081650.17353.anthony@interlink.com.au>
References: <200701081650.17353.anthony@interlink.com.au>
Message-ID: <ee2a432c0701072207s5a69e125s3ab75a931e5188db@mail.gmail.com>

On 1/7/07, Anthony Baxter <anthony at interlink.com.au> wrote:
> I've been thinking a little about how and where we'd add warnings to
> 2.6 and later for things that will break in 3.0. My first idea is
> to add a command line option '-3' (or maybe '-warn3') implemented
> as "from __future__ import py3k". We can then put code in that
> optionally generates a warning if this is set.
>
> I'm concerned that we don't make this check too expensive, though -

I have re-implemented warnings in C which should speed things up a bit
(including start up).  I need to polish it off and up load it.

n

From pje at telecommunity.com  Mon Jan  8 07:21:27 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 08 Jan 2007 01:21:27 -0500
Subject: [Python-3000] self-contained exceptions
In-Reply-To: <43aa6ff70701071914j2959f683ncc979bba8be9dc91@mail.gmail.co
 m>
References: <ca471dc20701041023q4fabfc76h34ed4a7b491774f4@mail.gmail.com>
	<1d85506f0701011438q6d8b65bfi4ef1e8f9fa509650@mail.gmail.com>
	<fb6fbf560701021220h4cb1b656u21c83d1b8b916fe5@mail.gmail.com>
	<ca471dc20701021228k592aced9g2244040f7ebc5b00@mail.gmail.com>
	<aac2c7cb0701030941o7e7d72dai76861554011fd3a8@mail.gmail.com>
	<Pine.LNX.4.58.0701032052370.16811@server1.LFW.org>
	<5.1.1.6.0.20070103225854.042a3868@sparrow.telecommunity.com>
	<5.1.1.6.0.20070104004429.02a19820@sparrow.telecommunity.com>
	<Pine.LNX.4.58.0701040139090.16811@server1.LFW.org>
	<5.1.1.6.0.20070104095412.029577a8@sparrow.telecommunity.com>
	<ca471dc20701041023q4fabfc76h34ed4a7b491774f4@mail.gmail.com>
Message-ID: <5.1.1.6.0.20070108012059.04121fd0@sparrow.telecommunity.com>

At 09:14 PM 1/7/2007 -0600, Collin Winter wrote:
>On 1/4/07, Guido van Rossum <guido at python.org> wrote:
> > On 1/4/07, Phillip J. Eby <pje at telecommunity.com> wrote:
> > > How about:
> > >
> > >       except ExcType, e:
> > >           try:
> > >               # body
> > >           finally:
> > >               e = None
> > >               del e
> > >
> > > Then we get the best of all three worlds: a clean explanation, a clean
> > > implementation, and a pure source-to-source transformation.
> >
> > Great! We need (a) an update to PEP 3100 and (b) a patch to implement
> > this. (Once we have this we can talk about the rest of PEP 344;
> > perhaps that PEP should also target Py3k only?)
>
>I've just uploaded patch #1630248 to SF. This implements the
>transformation (including test cases for the new behavior) and patches
>all places in the stdlib that relied on the old behavior.
>
>The transformation is as Phillip outlined above, with extra logic to
>handle the case where e is a tuple or list.

In the tuple or list case, there's no need to reset the variables, because 
then the traceback won't be present any more; the exception object will 
have been discarded after unpacking.


From anthony at interlink.com.au  Mon Jan  8 08:43:43 2007
From: anthony at interlink.com.au (Anthony Baxter)
Date: Mon, 8 Jan 2007 18:43:43 +1100
Subject: [Python-3000] Warning for 2.6 and greater
In-Reply-To: <200701081650.17353.anthony@interlink.com.au>
References: <200701081650.17353.anthony@interlink.com.au>
Message-ID: <200701081843.43631.anthony@interlink.com.au>

On Monday 08 January 2007 16:50, Anthony Baxter wrote:
> I've been thinking a little about how and where we'd add warnings
> to 2.6 and later for things that will break in 3.0. My first idea
> is to add a command line option '-3' (or maybe '-warn3')
> implemented as "from __future__ import py3k". We can then put
> code in that optionally generates a warning if this is set.

Ok, I lied - I implemented this pretty easily just now (patch 
attached). As a first pass, I put in warnings for dict.has_key and 
the builtin apply() function. 

I've changed my mind about the from future statement. I think that
'from __future__ import py3k' says "please use whatever backwards 
incompatible 3.x features that you support". I guess we could have 
something like 'from __future__ import warn3k', perhaps. But this 
wouldn't take effect for just the current source file, so could be 
confusing. 

If people are happy with this, I'll check it into the trunk, then 
people can go crazy adding other warnings for 3.0-isms. 

beaker% ./python
Python 2.6a0 (trunk:53285:53286M, Jan  8 2007, 18:34:47) 
[GCC 4.1.2 20060928 (prerelease) (Ubuntu 4.1.1-13ubuntu5)] on linux2
Type "help", "copyright", "credits" or "license" for more 
information.
>>> 1 in {}
False
>>> {}.has_key(1)
False
>>> apply(lambda x:None, (1,))
>>> 
beaker% ./python -3 
Python 2.6a0 (trunk:53285:53286M, Jan  8 2007, 18:34:47) 
[GCC 4.1.2 20060928 (prerelease) (Ubuntu 4.1.1-13ubuntu5)] on linux2
Type "help", "copyright", "credits" or "license" for more 
information.
>>> 1 in {}
False
>>> {}.has_key(1)
__main__:1: DeprecationWarning: dict.has_key not supported in 3.x
False
>>> apply(lambda x:None, (1,))
__main__:1: DeprecationWarning: apply() not supported in 3.x
>>> 

Anthony
-------------- next part --------------
A non-text attachment was scrubbed...
Name: warn3.patch
Type: text/x-diff
Size: 4039 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-3000/attachments/20070108/a6169da6/attachment.bin 

From thomas at python.org  Mon Jan  8 12:53:11 2007
From: thomas at python.org (Thomas Wouters)
Date: Mon, 8 Jan 2007 03:53:11 -0800
Subject: [Python-3000] Warning for 2.6 and greater
In-Reply-To: <200701081650.17353.anthony@interlink.com.au>
References: <200701081650.17353.anthony@interlink.com.au>
Message-ID: <9e804ac0701080353x6e84bd55q72848de26783635d@mail.gmail.com>

On 1/7/07, Anthony Baxter <anthony at interlink.com.au> wrote:
>
> I've been thinking a little about how and where we'd add warnings to
> 2.6 and later for things that will break in 3.0. My first idea is
> to add a command line option '-3' (or maybe '-warn3') implemented
> as "from __future__ import py3k". We can then put code in that
> optionally generates a warning if this is set.


I would prefer -Wpy3k, or the corresponding warnings.filterwarnings call,
and just not warn in too many (used) places. Also, there are a number of
things I'd like to warn for, regardless of -3/-warn3/-Wp3yk, that will be
errors in py3k, but are already a bad idea: mixing tabs and spaces
(basically make -t the default), using `` (the repr syntax), using input().

(Also on my warn-TODO for 2.6 are coerce(), __get/set/delslice__, integer
division, __cmp__, passing floats to getargs.c's integer functions,
exceptions not deriving from BaseException, use of file.xreadlines(), and
implicit relative imports, but those should all (probably) be optional,
default off.)

-- 
Thomas Wouters <thomas at python.org>

Hi! I'm a .signature virus! copy me into your .signature file to help me
spread!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20070108/56b2bf58/attachment.html 

From anthony at interlink.com.au  Mon Jan  8 13:29:42 2007
From: anthony at interlink.com.au (Anthony Baxter)
Date: Mon, 8 Jan 2007 23:29:42 +1100
Subject: [Python-3000] Warning for 2.6 and greater
In-Reply-To: <9e804ac0701080353x6e84bd55q72848de26783635d@mail.gmail.com>
References: <200701081650.17353.anthony@interlink.com.au>
	<9e804ac0701080353x6e84bd55q72848de26783635d@mail.gmail.com>
Message-ID: <200701082329.44458.anthony@interlink.com.au>

On Monday 08 January 2007 22:53, Thomas Wouters wrote:
> I would prefer -Wpy3k, or the corresponding
> warnings.filterwarnings call, and just not warn in too many
> (used) places.

-W seems like an option. We can add other -W options later
(for instance, -W tabs). Hm. -W already has a meaning, though.
Should we also add a PYTHONWARNINGS environment variable?

> Also, there are a number of things I'd like to 
> warn for, regardless of -3/-warn3/-Wp3yk, that will be errors in
> py3k, but are already a bad idea: mixing tabs and spaces
> (basically make -t the default), using `` (the repr syntax),
> using input().

I haven't looked into hooking into the compiler yet to detect ``. 
The others seem OK to add. 

> (Also on my warn-TODO for 2.6 are coerce(), __get/set/delslice__,
> integer division, 

I'll just make -W py3k also enable -Qwarn (and -t, for the tabs).
Or do you want to make -t the default for 2.6? In that case, should 
we add an option to turn it off? Hm. -T, maybe?

I'm not sure whether we should try and warn about 'print' becoming a 
function. We should also consider things like intern() and 
compile() (if they are to move to sys).
> __cmp__, passing floats to getargs.c's integer 
> functions, exceptions not deriving from BaseException, use of
> file.xreadlines(), and implicit relative imports, but those
> should all (probably) be optional, default off.)

See, I'd like to make -W py3k enable _all_ of these warnings. If we 
also want to add finer-grained warnings, we could do that, too (but 
I'm not so sure about that). 

Do we also want to add an option to promote these warnings to actual 
errors? I think leaving that for 2.7 is a good idea...

Anthony
-- 
Anthony Baxter     <anthony at interlink.com.au>
It's never too late to have a happy childhood.

From thomas at python.org  Mon Jan  8 14:08:24 2007
From: thomas at python.org (Thomas Wouters)
Date: Mon, 8 Jan 2007 05:08:24 -0800
Subject: [Python-3000] Warning for 2.6 and greater
In-Reply-To: <200701082329.44458.anthony@interlink.com.au>
References: <200701081650.17353.anthony@interlink.com.au>
	<9e804ac0701080353x6e84bd55q72848de26783635d@mail.gmail.com>
	<200701082329.44458.anthony@interlink.com.au>
Message-ID: <9e804ac0701080508w7773e689q53fc00d4d9317d0a@mail.gmail.com>

On 1/8/07, Anthony Baxter <anthony at interlink.com.au> wrote:
>
> On Monday 08 January 2007 22:53, Thomas Wouters wrote:
> > I would prefer -Wpy3k, or the corresponding
> > warnings.filterwarnings call, and just not warn in too many
> > (used) places.
>
> -W seems like an option. We can add other -W options later
> (for instance, -W tabs). Hm. -W already has a meaning, though.


Well, yes, that's why I want to add it to -W :) I mean to suggest that
'py3k' should just be a warning category. And not a special case.

Should we also add a PYTHONWARNINGS environment variable?


Makes sense, I suppose.

> Also, there are a number of things I'd like to
> > warn for, regardless of -3/-warn3/-Wp3yk, that will be errors in
> > py3k, but are already a bad idea: mixing tabs and spaces
> > (basically make -t the default), using `` (the repr syntax),
> > using input().
>
> I haven't looked into hooking into the compiler yet to detect ``.
> The others seem OK to add.
>
> > (Also on my warn-TODO for 2.6 are coerce(), __get/set/delslice__,
> > integer division,
>
> I'll just make -W py3k also enable -Qwarn (and -t, for the tabs).
> Or do you want to make -t the default for 2.6? In that case, should
> we add an option to turn it off? Hm. -T, maybe?


Yes, I mean for the warnings for ``, tabmixing and input() to be enabled
always, and the rest just when enabling py3k warnings. And yes, we'd need a
way to turn the tabmix-warnings off; in the p3yk branch, I made '-t' and
'-tt' be a no-op (since it errors by default), and introduces '-ttt' to turn
tab-errors into warnings. The main reason I've not made tabmix-warnings the
default in the trunk is that I couldn't think of the right option to turn it
back off :) It doesn't use the warnings mechanism (mostly because it has to
warn during parsing, when usually no python code is running) so '-W
tabs:ignore' doesn't work. Then again, maybe with Neal's warning framework
in C, we can make it work for the parser, too. Would be luvly for
SyntaxWarnings too :)

I'm not sure whether we should try and warn about 'print' becoming a
> function. We should also consider things like intern() and
> compile() (if they are to move to sys).


For p3yk-warnings, we should warn about both, but I'm not sure about the
usefulness of warning about print... there's no upgrade path, as such. At
least not for all print's features.

> __cmp__, passing floats to getargs.c's integer
> > functions, exceptions not deriving from BaseException, use of
> > file.xreadlines(), and implicit relative imports, but those
> > should all (probably) be optional, default off.)
>
> See, I'd like to make -W py3k enable _all_ of these warnings.


Me, too. That's the 'optional, default off' thing. I was talking about
making things warn in absense of -Wpy3k too.

Do we also want to add an option to promote these warnings to actual
> errors? I think leaving that for 2.7 is a good idea...


We have that: -W error :)

-- 
Thomas Wouters <thomas at python.org>

Hi! I'm a .signature virus! copy me into your .signature file to help me
spread!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20070108/bb4a59e8/attachment.htm 

From collinw at gmail.com  Mon Jan  8 17:25:17 2007
From: collinw at gmail.com (Collin Winter)
Date: Mon, 8 Jan 2007 10:25:17 -0600
Subject: [Python-3000] self-contained exceptions
In-Reply-To: <5.1.1.6.0.20070108012059.04121fd0@sparrow.telecommunity.com>
References: <1d85506f0701011438q6d8b65bfi4ef1e8f9fa509650@mail.gmail.com>
	<ca471dc20701021228k592aced9g2244040f7ebc5b00@mail.gmail.com>
	<aac2c7cb0701030941o7e7d72dai76861554011fd3a8@mail.gmail.com>
	<Pine.LNX.4.58.0701032052370.16811@server1.LFW.org>
	<5.1.1.6.0.20070103225854.042a3868@sparrow.telecommunity.com>
	<5.1.1.6.0.20070104004429.02a19820@sparrow.telecommunity.com>
	<Pine.LNX.4.58.0701040139090.16811@server1.LFW.org>
	<5.1.1.6.0.20070104095412.029577a8@sparrow.telecommunity.com>
	<ca471dc20701041023q4fabfc76h34ed4a7b491774f4@mail.gmail.com>
	<5.1.1.6.0.20070108012059.04121fd0@sparrow.telecommunity.com>
Message-ID: <43aa6ff70701080825n4af8c375y74cc8ad3367d3564@mail.gmail.com>

On 1/8/07, Phillip J. Eby <pje at telecommunity.com> wrote:
> At 09:14 PM 1/7/2007 -0600, Collin Winter wrote:
> >The transformation is as Phillip outlined above, with extra logic to
> >handle the case where e is a tuple or list.
>
> In the tuple or list case, there's no need to reset the variables, because
> then the traceback won't be present any more; the exception object will
> have been discarded after unpacking.

Ah, good point. I'll update the patch to strip out that extra code.

Thanks,
Collin Winter

From pje at telecommunity.com  Mon Jan  8 18:25:43 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 08 Jan 2007 12:25:43 -0500
Subject: [Python-3000] self-contained exceptions
In-Reply-To: <43aa6ff70701080825n4af8c375y74cc8ad3367d3564@mail.gmail.co
 m>
References: <5.1.1.6.0.20070108012059.04121fd0@sparrow.telecommunity.com>
	<1d85506f0701011438q6d8b65bfi4ef1e8f9fa509650@mail.gmail.com>
	<ca471dc20701021228k592aced9g2244040f7ebc5b00@mail.gmail.com>
	<aac2c7cb0701030941o7e7d72dai76861554011fd3a8@mail.gmail.com>
	<Pine.LNX.4.58.0701032052370.16811@server1.LFW.org>
	<5.1.1.6.0.20070103225854.042a3868@sparrow.telecommunity.com>
	<5.1.1.6.0.20070104004429.02a19820@sparrow.telecommunity.com>
	<Pine.LNX.4.58.0701040139090.16811@server1.LFW.org>
	<5.1.1.6.0.20070104095412.029577a8@sparrow.telecommunity.com>
	<ca471dc20701041023q4fabfc76h34ed4a7b491774f4@mail.gmail.com>
	<5.1.1.6.0.20070108012059.04121fd0@sparrow.telecommunity.com>
Message-ID: <5.1.1.6.0.20070108122108.04b73328@sparrow.telecommunity.com>

At 10:25 AM 1/8/2007 -0600, Collin Winter wrote:
>On 1/8/07, Phillip J. Eby <pje at telecommunity.com> wrote:
>>At 09:14 PM 1/7/2007 -0600, Collin Winter wrote:
>> >The transformation is as Phillip outlined above, with extra logic to
>> >handle the case where e is a tuple or list.
>>
>>In the tuple or list case, there's no need to reset the variables, because
>>then the traceback won't be present any more; the exception object will
>>have been discarded after unpacking.
>
>Ah, good point. I'll update the patch to strip out that extra code.

Hm... what happens with things like:

      except Exception, x[y]:

and

      except Exception, x(y).z:

Are these currently allowed?  Should they be allowed?  And should we clear 
them?

My inclination is to say that we should only clear in the simple local 
variable case, as anything more advanced is up to the author to sort out.

Actually, I think we should probably just define the syntax in Py3K as:

      except <expr> [as <name>]:

And to heck with the rest of the complexity.  :)  For migration 
convenience, we could offer this syntax as an option in 2.x, preferably 
with the same name-clearing behavior, even though it's not needed.



From collinw at gmail.com  Mon Jan  8 18:34:54 2007
From: collinw at gmail.com (Collin Winter)
Date: Mon, 8 Jan 2007 11:34:54 -0600
Subject: [Python-3000] self-contained exceptions
In-Reply-To: <5.1.1.6.0.20070108122108.04b73328@sparrow.telecommunity.com>
References: <1d85506f0701011438q6d8b65bfi4ef1e8f9fa509650@mail.gmail.com>
	<aac2c7cb0701030941o7e7d72dai76861554011fd3a8@mail.gmail.com>
	<Pine.LNX.4.58.0701032052370.16811@server1.LFW.org>
	<5.1.1.6.0.20070103225854.042a3868@sparrow.telecommunity.com>
	<5.1.1.6.0.20070104004429.02a19820@sparrow.telecommunity.com>
	<Pine.LNX.4.58.0701040139090.16811@server1.LFW.org>
	<5.1.1.6.0.20070104095412.029577a8@sparrow.telecommunity.com>
	<ca471dc20701041023q4fabfc76h34ed4a7b491774f4@mail.gmail.com>
	<5.1.1.6.0.20070108012059.04121fd0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070108122108.04b73328@sparrow.telecommunity.com>
Message-ID: <43aa6ff70701080934w3d66b48dw17733c2f5a64b140@mail.gmail.com>

On 1/8/07, Phillip J. Eby <pje at telecommunity.com> wrote:
> At 10:25 AM 1/8/2007 -0600, Collin Winter wrote:
> >On 1/8/07, Phillip J. Eby <pje at telecommunity.com> wrote:
> >>At 09:14 PM 1/7/2007 -0600, Collin Winter wrote:
> >> >The transformation is as Phillip outlined above, with extra logic to
> >> >handle the case where e is a tuple or list.
> >>
> >>In the tuple or list case, there's no need to reset the variables, because
> >>then the traceback won't be present any more; the exception object will
> >>have been discarded after unpacking.
> >
> >Ah, good point. I'll update the patch to strip out that extra code.
>
> Hm... what happens with things like:
>
>       except Exception, x[y]:
>
> and
>
>       except Exception, x(y).z:
>
> Are these currently allowed?  Should they be allowed?  And should we clear
> them?

They are currently allowed, and they are currently being cleared.

> My inclination is to say that we should only clear in the simple local
> variable case, as anything more advanced is up to the author to sort out.
>
> Actually, I think we should probably just define the syntax in Py3K as:
>
>       except <expr> [as <name>]:
>

FWIW, I wasn't aware that things like "except T, (a, b, c):" were
possible until I started work on this patch; I thought <name> was all
you _could_ use.

Collin Winter

From pje at telecommunity.com  Mon Jan  8 18:48:28 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 08 Jan 2007 12:48:28 -0500
Subject: [Python-3000] self-contained exceptions
In-Reply-To: <43aa6ff70701080934w3d66b48dw17733c2f5a64b140@mail.gmail.co
 m>
References: <5.1.1.6.0.20070108122108.04b73328@sparrow.telecommunity.com>
	<1d85506f0701011438q6d8b65bfi4ef1e8f9fa509650@mail.gmail.com>
	<aac2c7cb0701030941o7e7d72dai76861554011fd3a8@mail.gmail.com>
	<Pine.LNX.4.58.0701032052370.16811@server1.LFW.org>
	<5.1.1.6.0.20070103225854.042a3868@sparrow.telecommunity.com>
	<5.1.1.6.0.20070104004429.02a19820@sparrow.telecommunity.com>
	<Pine.LNX.4.58.0701040139090.16811@server1.LFW.org>
	<5.1.1.6.0.20070104095412.029577a8@sparrow.telecommunity.com>
	<ca471dc20701041023q4fabfc76h34ed4a7b491774f4@mail.gmail.com>
	<5.1.1.6.0.20070108012059.04121fd0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070108122108.04b73328@sparrow.telecommunity.com>
Message-ID: <5.1.1.6.0.20070108124331.04b80240@sparrow.telecommunity.com>

At 11:34 AM 1/8/2007 -0600, Collin Winter wrote:
>On 1/8/07, Phillip J. Eby <pje at telecommunity.com> wrote:
>>At 10:25 AM 1/8/2007 -0600, Collin Winter wrote:
>> >On 1/8/07, Phillip J. Eby <pje at telecommunity.com> wrote:
>> >>At 09:14 PM 1/7/2007 -0600, Collin Winter wrote:
>> >> >The transformation is as Phillip outlined above, with extra logic to
>> >> >handle the case where e is a tuple or list.
>> >>
>> >>In the tuple or list case, there's no need to reset the variables, because
>> >>then the traceback won't be present any more; the exception object will
>> >>have been discarded after unpacking.
>> >
>> >Ah, good point. I'll update the patch to strip out that extra code.
>>
>>Hm... what happens with things like:
>>
>>       except Exception, x[y]:
>>
>>and
>>
>>       except Exception, x(y).z:
>>
>>Are these currently allowed?  Should they be allowed?  And should we clear
>>them?
>
>They are currently allowed, and they are currently being cleared.

Which is a bit weird, in that x(y).z isn't necessarily even reachable from 
the frame, although x[y] certainly is.  However, there are no guarantees 
that these targets will still be valid at the time of clearing, either, or 
that they allow delattr/delitem to work, so this really sinks the whole 
clearing idea, unless we move to an 'as name' format.  I seem to recall 
that Guido wanted to move to an 'as' clause for Py3K anyway, to avoid the 
"except FooExc, BarExc:" anti-pattern.  So now seems as good a time "as" 
any.  :)


From eopadoan at altavix.com  Mon Jan  8 19:13:25 2007
From: eopadoan at altavix.com (Eduardo "EdCrypt" O. Padoan)
Date: Mon, 8 Jan 2007 16:13:25 -0200
Subject: [Python-3000] self-contained exceptions
In-Reply-To: <5.1.1.6.0.20070108124331.04b80240@sparrow.telecommunity.com>
References: <1d85506f0701011438q6d8b65bfi4ef1e8f9fa509650@mail.gmail.com>
	<Pine.LNX.4.58.0701032052370.16811@server1.LFW.org>
	<5.1.1.6.0.20070103225854.042a3868@sparrow.telecommunity.com>
	<5.1.1.6.0.20070104004429.02a19820@sparrow.telecommunity.com>
	<Pine.LNX.4.58.0701040139090.16811@server1.LFW.org>
	<5.1.1.6.0.20070104095412.029577a8@sparrow.telecommunity.com>
	<ca471dc20701041023q4fabfc76h34ed4a7b491774f4@mail.gmail.com>
	<5.1.1.6.0.20070108012059.04121fd0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070108122108.04b73328@sparrow.telecommunity.com>
	<5.1.1.6.0.20070108124331.04b80240@sparrow.telecommunity.com>
Message-ID: <dea92f560701081013h5382bd34m30c4d40a6191053e@mail.gmail.com>

  I seem to recall
> that Guido wanted to move to an 'as' clause for Py3K anyway, to avoid the
> "except FooExc, BarExc:" anti-pattern.  So now seems as good a time "as"
> any.  :)


http://mail.python.org/pipermail/python-dev/2006-March/062449.html
http://mail.python.org/pipermail/python-dev/2006-March/062640.html

-- 
EduardoOPadoan (eopadoan->altavix::com)
Bookmarks: http://del.icio.us/edcrypt
Blog: http://edcrypt.blogspot.com
Jabber: edcrypt at jabber dot org
ICQ: 161480283
GTalk: eduardo dot padoan at gmail dot com
MSN: eopadoan at altavix dot com

From jcarlson at uci.edu  Mon Jan  8 20:17:56 2007
From: jcarlson at uci.edu (Josiah Carlson)
Date: Mon, 08 Jan 2007 11:17:56 -0800
Subject: [Python-3000] Lazy strings (was Re: Py3k release schedule
	worries)
In-Reply-To: <ca471dc20612311043y52ff10aeu5db878ae48156efa@mail.gmail.com>
References: <459732CF.3020803@acm.org>
	<ca471dc20612311043y52ff10aeu5db878ae48156efa@mail.gmail.com>
Message-ID: <20070108111412.8D40.JCARLSON@uci.edu>


"Guido van Rossum" <guido at python.org> wrote:
> On 12/30/06, Talin <talin at acm.org> wrote:
> > Maybe this would be a good time to review, or at least restate, the
> > specific plans for strings in Py3K? I know that there's been a great
> > deal of discussion on this, but a lot of that discussion took place
> > *before* Larry's work (specifically, before a number of people in this
> > group drank the lazy-strings KoolAid.)
> 
> The *only* think that I care about is that we end up with a single
> string type named 'str' that has the same semantics (and some of the
> same performance) as the current Unicode strings. (I mention
> performance because s[i] should remain an O(1) operation.)

Could I get some clarification on this point?

Were you referring to the structure I had offered that would allow the
proper handling of surrogate pairs in the case of s[i], which would
result in access time being a function of the number of surrogates in
the string (in the worst case O(log num_surrogates) )?

Was this in reference to delayed construction time overhead as in the
case of lazy strings?

Both?  Neither?  Something else that I've forgotten?


 - Josiah


From rasky at develer.com  Tue Jan  9 00:38:43 2007
From: rasky at develer.com (Giovanni Bajo)
Date: Tue, 09 Jan 2007 00:38:43 +0100
Subject: [Python-3000] Warning for 2.6 and greater
In-Reply-To: <200701082329.44458.anthony@interlink.com.au>
References: <200701081650.17353.anthony@interlink.com.au>	<9e804ac0701080353x6e84bd55q72848de26783635d@mail.gmail.com>
	<200701082329.44458.anthony@interlink.com.au>
Message-ID: <enukm1$p72$1@sea.gmane.org>

Anthony Baxter wrote:

> I'm not sure whether we should try and warn about 'print' becoming a 
> function. We should also consider things like intern() and 
> compile() (if they are to move to sys).

I think it's fine if we prepare an upgrade path: for instance, we could add 
sys.intern() and sys.compile() to 2.6, and warn only if the builtin version is 
used.

In other words, I think it's important always to provide a way to "fix the 
code" so that -W py3k is happy *and* it still works on the Python 2.x serie.
-- 
Giovanni Bajo


From anthony at interlink.com.au  Tue Jan  9 01:52:47 2007
From: anthony at interlink.com.au (Anthony Baxter)
Date: Tue, 9 Jan 2007 11:52:47 +1100
Subject: [Python-3000] Warning for 2.6 and greater
In-Reply-To: <9e804ac0701080353x6e84bd55q72848de26783635d@mail.gmail.com>
References: <200701081650.17353.anthony@interlink.com.au>
	<9e804ac0701080353x6e84bd55q72848de26783635d@mail.gmail.com>
Message-ID: <200701091152.48048.anthony@interlink.com.au>

On Monday 08 January 2007 22:53, Thomas Wouters wrote:
> (Also on my warn-TODO for 2.6 are coerce(), __get/set/delslice__

Should we warn on use of the __slice__ methods and the __cmp__ 
method, or hook into the object code to warn when they're defined?
I guess the latter would be more proactive. Or we could do both.

Do we also want to warn about use of sq_slice and sq_ass_slice with 
C-code objects? (And yes, my inner 12 year old still giggles about 
sq_ass_slice)

On Tuesday 09 January 2007 00:08, Thomas Wouters wrote:
> Well, yes, that's why I want to add it to -W :) I mean to suggest
> that 'py3k' should just be a warning category. And not a special
> case.

My concern is that having to dive into the warnings code could cause 
terrible slowdowns in places that are heavily used. The major 
concern I have is dict.has_key() - a quick audit of a bunch of code 
shows that it's used heavily inside inner loops. This is mostly for 
2.6 - this is the first time we're going to be (optionally) issuing 
these warnings, and it seems a little user-hostile to suddenly make 
their code suck. Checking for a single global C variable is going 
to add a very slight overhead (particularly compared to the method 
call cost). Making an additional call into the warnings code is 
going to be a lot heavier...

> Yes, I mean for the warnings for ``, tabmixing and input() to be
> enabled always, and the rest just when enabling py3k warnings.
> And yes, we'd need a way to turn the tabmix-warnings off; in the
> p3yk branch, I made '-t' and '-tt' be a no-op (since it errors by
> default), and introduces '-ttt' to turn tab-errors into warnings.

Why not -T for "suppress the errors"? 
So for 2.6, we'd make -t the default (and a single -t be a no-op).

> It doesn't use the warnings mechanism (mostly because
> it has to warn during parsing, when usually no python code is
> running) so '-W tabs:ignore' doesn't work. Then again, maybe with
> Neal's warning framework in C, we can make it work for the
> parser, too. Would be luvly for SyntaxWarnings too :)

I'm not sure whether reimplementing warnings in C is going to add 
this - I guess we can ask Neal nicely to add this functionality. Hi 
Neal.

> > I'm not sure whether we should try and warn about 'print'
> > becoming a function. We should also consider things like
> > intern() and compile() (if they are to move to sys).
>
> For p3yk-warnings, we should warn about both, but I'm not sure
> about the usefulness of warning about print... there's no upgrade
> path, as such. At least not for all print's features.

There's also the problem that 
print ("hello world") 
is valid both for print-as-function and print-as-statement.

But let's leave print until 2.7. Hopefully by then we'll have some 
vague sense of an upgrade path, perhaps. 

> > Do we also want to add an option to promote these warnings to
> > actual errors? I think leaving that for 2.7 is a good idea...
>
> We have that: -W error :)

Well, yes, except that -W error would need some special magic to not 
just fall over on all Warnings. Maybe we could make it only raise 
for DeprecationWarnings. ("I break for DeprecationWarnings")

-- 
Anthony Baxter     <anthony at interlink.com.au>
It's never too late to have a happy childhood.

From aahz at pythoncraft.com  Tue Jan  9 06:26:10 2007
From: aahz at pythoncraft.com (Aahz)
Date: Mon, 8 Jan 2007 21:26:10 -0800
Subject: [Python-3000] Warning for 2.6 and greater
In-Reply-To: <200701091152.48048.anthony@interlink.com.au>
References: <200701081650.17353.anthony@interlink.com.au>
	<9e804ac0701080353x6e84bd55q72848de26783635d@mail.gmail.com>
	<200701091152.48048.anthony@interlink.com.au>
Message-ID: <20070109052610.GA7620@panix.com>

On Tue, Jan 09, 2007, Anthony Baxter wrote:
> On Monday 08 January 2007 22:53, Thomas Wouters wrote:
>>
>> (Also on my warn-TODO for 2.6 are coerce(), __get/set/delslice__
> 
> Should we warn on use of the __slice__ methods and the __cmp__ 
> method, or hook into the object code to warn when they're defined?
> I guess the latter would be more proactive. Or we could do both.

Hey!  What's this about __cmp__?!
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"Plus ca change, plus c'est la meme chose."

From nnorwitz at gmail.com  Tue Jan  9 07:35:26 2007
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Mon, 8 Jan 2007 22:35:26 -0800
Subject: [Python-3000] Warning for 2.6 and greater
In-Reply-To: <200701091152.48048.anthony@interlink.com.au>
References: <200701081650.17353.anthony@interlink.com.au>
	<9e804ac0701080353x6e84bd55q72848de26783635d@mail.gmail.com>
	<200701091152.48048.anthony@interlink.com.au>
Message-ID: <ee2a432c0701082235m431ad06el9a1932414d87fa0f@mail.gmail.com>

On 1/8/07, Anthony Baxter <anthony at interlink.com.au> wrote:
>
> Do we also want to warn about use of sq_slice and sq_ass_slice with
> C-code objects? (And yes, my inner 12 year old still giggles about
> sq_ass_slice)

Sometimes it's nice to know you aren't alone. :-)

> My concern is that having to dive into the warnings code could cause
> terrible slowdowns in places that are heavily used. The major
> concern I have is dict.has_key() - a quick audit of a bunch of code
> shows that it's used heavily inside inner loops. This is mostly for
> 2.6 - this is the first time we're going to be (optionally) issuing
> these warnings, and it seems a little user-hostile to suddenly make
> their code suck. Checking for a single global C variable is going
> to add a very slight overhead (particularly compared to the method
> call cost). Making an additional call into the warnings code is
> going to be a lot heavier...

Sure, for really heavy places we can also use a static int (bool)
check.  Not really sure of the best way to handle this.  There was a
warning for apply() that was removed because it was too slow.

My general preference as a user is to have more warnings.  My
preference as a maintainer is to have less.

> > It doesn't use the warnings mechanism (mostly because
> > it has to warn during parsing, when usually no python code is
> > running) so '-W tabs:ignore' doesn't work. Then again, maybe with
> > Neal's warning framework in C, we can make it work for the
> > parser, too. Would be luvly for SyntaxWarnings too :)
>
> I'm not sure whether reimplementing warnings in C is going to add
> this - I guess we can ask Neal nicely to add this functionality. Hi
> Neal.

http://python.org/sf/1631171

I'll be gone for a few days, so probably won't get a chance to respond
until this weekend.

n

From thomas at python.org  Tue Jan  9 10:49:08 2007
From: thomas at python.org (Thomas Wouters)
Date: Tue, 9 Jan 2007 10:49:08 +0100
Subject: [Python-3000] Warning for 2.6 and greater
In-Reply-To: <20070109052610.GA7620@panix.com>
References: <200701081650.17353.anthony@interlink.com.au>
	<9e804ac0701080353x6e84bd55q72848de26783635d@mail.gmail.com>
	<200701091152.48048.anthony@interlink.com.au>
	<20070109052610.GA7620@panix.com>
Message-ID: <9e804ac0701090149p5552034dgc8c21c81e867b229@mail.gmail.com>

On 1/9/07, Aahz <aahz at pythoncraft.com> wrote:

> Hey!  What's this about __cmp__?!


Python 3.0x (p3yk:53295, Jan  8 2007, 09:32:17)
[GCC 4.1.2 20061115 (prerelease) (Debian 4.1.1-21)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> object.__cmp__
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: type object 'object' has no attribute '__cmp__'

E's dead. Stone dead, miss.

-- 
Thomas Wouters <thomas at python.org>

Hi! I'm a .signature virus! copy me into your .signature file to help me
spread!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20070109/9b5f68ae/attachment.html 

From guido at python.org  Tue Jan  9 18:41:46 2007
From: guido at python.org (Guido van Rossum)
Date: Tue, 9 Jan 2007 09:41:46 -0800
Subject: [Python-3000] self-contained exceptions
In-Reply-To: <43aa6ff70701080934w3d66b48dw17733c2f5a64b140@mail.gmail.com>
References: <1d85506f0701011438q6d8b65bfi4ef1e8f9fa509650@mail.gmail.com>
	<Pine.LNX.4.58.0701032052370.16811@server1.LFW.org>
	<5.1.1.6.0.20070103225854.042a3868@sparrow.telecommunity.com>
	<5.1.1.6.0.20070104004429.02a19820@sparrow.telecommunity.com>
	<Pine.LNX.4.58.0701040139090.16811@server1.LFW.org>
	<5.1.1.6.0.20070104095412.029577a8@sparrow.telecommunity.com>
	<ca471dc20701041023q4fabfc76h34ed4a7b491774f4@mail.gmail.com>
	<5.1.1.6.0.20070108012059.04121fd0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070108122108.04b73328@sparrow.telecommunity.com>
	<43aa6ff70701080934w3d66b48dw17733c2f5a64b140@mail.gmail.com>
Message-ID: <ca471dc20701090941x17110b56y193110bad6c6fd53@mail.gmail.com>

On 1/8/07, Collin Winter <collinw at gmail.com> wrote:
> FWIW, I wasn't aware that things like "except T, (a, b, c):" were
> possible until I started work on this patch; I thought <name> was all
> you _could_ use.

And, given the secial semantics for this position, I would support
changing the syntax to *only* allow a single simple name there -- not
even x.y.z. Thsh should handle 99.9% of existing code anyway.

(And yes, if someone wants to add the 'as' keyword at the same time
that would be great!

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Tue Jan  9 18:47:02 2007
From: guido at python.org (Guido van Rossum)
Date: Tue, 9 Jan 2007 09:47:02 -0800
Subject: [Python-3000] Lazy strings (was Re: Py3k release schedule
	worries)
In-Reply-To: <20070108111412.8D40.JCARLSON@uci.edu>
References: <459732CF.3020803@acm.org>
	<ca471dc20612311043y52ff10aeu5db878ae48156efa@mail.gmail.com>
	<20070108111412.8D40.JCARLSON@uci.edu>
Message-ID: <ca471dc20701090947q4696828dv48375049d47b59ec@mail.gmail.com>

> > On 12/30/06, Talin <talin at acm.org> wrote:
> > > Maybe this would be a good time to review, or at least restate, the
> > > specific plans for strings in Py3K? I know that there's been a great
> > > deal of discussion on this, but a lot of that discussion took place
> > > *before* Larry's work (specifically, before a number of people in this
> > > group drank the lazy-strings KoolAid.)

> "Guido van Rossum" <guido at python.org> wrote:
> > The *only* think that I care about is that we end up with a single
> > string type named 'str' that has the same semantics (and some of the
> > same performance) as the current Unicode strings. (I mention
> > performance because s[i] should remain an O(1) operation.)

On 1/8/07, Josiah Carlson <jcarlson at uci.edu> wrote:
> Could I get some clarification on this point?
>
> Were you referring to the structure I had offered that would allow the
> proper handling of surrogate pairs in the case of s[i], which would
> result in access time being a function of the number of surrogates in
> the string (in the worst case O(log num_surrogates) )?

I wasn't referring to anything in particular; I have neither looked at
any of the discussion not at any patches. I was just stating the
expectation that indexing a sequence should take O(1) time. (I'd be
okay with an *amortized* O(1) since that is if course still O(1), as
long as it really is.)

> Was this in reference to delayed construction time overhead as in the
> case of lazy strings?

Not in particular; I was just arguing from a clean slate. Though I
guess you could argue that since construction time is curerntly O(N),
if you could win some on construction you'd have that to play with to
meet the O(1) access goal. Although of curse then you run the risk of
implementing something that doesn't have any performance benefits yet
is much more complicated, which is exactly the basis of my arguments
against all this kind of cleverness.

> Both?  Neither?  Something else that I've forgotten?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From collinw at gmail.com  Tue Jan  9 18:48:53 2007
From: collinw at gmail.com (Collin Winter)
Date: Tue, 9 Jan 2007 11:48:53 -0600
Subject: [Python-3000] self-contained exceptions
In-Reply-To: <ca471dc20701090941x17110b56y193110bad6c6fd53@mail.gmail.com>
References: <1d85506f0701011438q6d8b65bfi4ef1e8f9fa509650@mail.gmail.com>
	<5.1.1.6.0.20070103225854.042a3868@sparrow.telecommunity.com>
	<5.1.1.6.0.20070104004429.02a19820@sparrow.telecommunity.com>
	<Pine.LNX.4.58.0701040139090.16811@server1.LFW.org>
	<5.1.1.6.0.20070104095412.029577a8@sparrow.telecommunity.com>
	<ca471dc20701041023q4fabfc76h34ed4a7b491774f4@mail.gmail.com>
	<5.1.1.6.0.20070108012059.04121fd0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070108122108.04b73328@sparrow.telecommunity.com>
	<43aa6ff70701080934w3d66b48dw17733c2f5a64b140@mail.gmail.com>
	<ca471dc20701090941x17110b56y193110bad6c6fd53@mail.gmail.com>
Message-ID: <43aa6ff70701090948r33a779a8ie56b58ef6e011a27@mail.gmail.com>

On 1/9/07, Guido van Rossum <guido at python.org> wrote:
> And, given the secial semantics for this position, I would support
> changing the syntax to *only* allow a single simple name there -- not
> even x.y.z. Thsh should handle 99.9% of existing code anyway.
>
> (And yes, if someone wants to add the 'as' keyword at the same time
> that would be great!

I'm going to roll the next except...as...: syntax into the next
version of my exception cleanup patch.

Collin Winter

From tcdelaney at optusnet.com.au  Tue Jan  9 22:10:15 2007
From: tcdelaney at optusnet.com.au (Tim Delaney)
Date: Wed, 10 Jan 2007 08:10:15 +1100
Subject: [Python-3000] self-contained exceptions
References: <1d85506f0701011438q6d8b65bfi4ef1e8f9fa509650@mail.gmail.com><Pine.LNX.4.58.0701032052370.16811@server1.LFW.org><5.1.1.6.0.20070103225854.042a3868@sparrow.telecommunity.com><5.1.1.6.0.20070104004429.02a19820@sparrow.telecommunity.com><Pine.LNX.4.58.0701040139090.16811@server1.LFW.org><5.1.1.6.0.20070104095412.029577a8@sparrow.telecommunity.com><ca471dc20701041023q4fabfc76h34ed4a7b491774f4@mail.gmail.com><5.1.1.6.0.20070108012059.04121fd0@sparrow.telecommunity.com><5.1.1.6.0.20070108122108.04b73328@sparrow.telecommunity.com><43aa6ff70701080934w3d66b48dw17733c2f5a64b140@mail.gmail.com>
	<ca471dc20701090941x17110b56y193110bad6c6fd53@mail.gmail.com>
Message-ID: <008601c73432$8f46f5f0$f100a8c0@ryoko>

Guido van Rossum wrote:

> On 1/8/07, Collin Winter <collinw at gmail.com> wrote:
>> FWIW, I wasn't aware that things like "except T, (a, b, c):" were
>> possible until I started work on this patch; I thought <name> was all
>> you _could_ use.
>
> And, given the secial semantics for this position, I would support
> changing the syntax to *only* allow a single simple name there -- not
> even x.y.z. Thsh should handle 99.9% of existing code anyway.
>
> (And yes, if someone wants to add the 'as' keyword at the same time
> that would be great!

With such changed semantics (only allow simple name, simple name gets 
deleted at the end of the except: block) I would be strongly in favour of 
only having the changed semantics when the 'as' keyword is used.

This would definitely make the warning about usage of old-style exceptions 
much easier - no need to try to determine if the exception name is used 
outside of the block - just check for an exception clause that doesn't use 
'as'.

Haven't checked the patch, but does it successfully delete the name in the 
case that another exception is thrown inside the except block?

    try:
        raise Exception()
    except Exception as e:
        try:
            raise Exception()
        except Exception as e2:
            pass

Also, what are the semantics of:

    try:
        raise Exception()
    except Exception as e:
        try:
            raise Exception()
        except Exception as e:
            pass

        print e

and

    try:
        raise Exception()
    except Exception as e:
        try:
            pass
        except Exception as e:
            pass

        print e

As I understand the proposed semantics, in the first case a NameError will 
be thrown at the 'print e' line in the first case, but not in the second 
case, because we never enter the second except: block. For consistency, 
maybe instead the semantics be that any name declared for catching an 
exception be deleted at the end of the try/except/else/finally clause i.e.

    try:
        pass
    except AttributeError as e1:
        pass
    except NameError as e2:
        pass

is equivalent to:

    try:
        try:
            pass
        except AttributeError, e1:
            pass
        except NameError, e2:
            pass
    finally:
        e1 = None
        e2 = None
        del e1
        del e2

Tim Delaney 


From pje at telecommunity.com  Tue Jan  9 22:41:03 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue, 09 Jan 2007 16:41:03 -0500
Subject: [Python-3000] self-contained exceptions
In-Reply-To: <008601c73432$8f46f5f0$f100a8c0@ryoko>
References: <1d85506f0701011438q6d8b65bfi4ef1e8f9fa509650@mail.gmail.com>
	<Pine.LNX.4.58.0701032052370.16811@server1.LFW.org>
	<5.1.1.6.0.20070103225854.042a3868@sparrow.telecommunity.com>
	<5.1.1.6.0.20070104004429.02a19820@sparrow.telecommunity.com>
	<Pine.LNX.4.58.0701040139090.16811@server1.LFW.org>
	<5.1.1.6.0.20070104095412.029577a8@sparrow.telecommunity.com>
	<ca471dc20701041023q4fabfc76h34ed4a7b491774f4@mail.gmail.com>
	<5.1.1.6.0.20070108012059.04121fd0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070108122108.04b73328@sparrow.telecommunity.com>
	<43aa6ff70701080934w3d66b48dw17733c2f5a64b140@mail.gmail.com>
	<ca471dc20701090941x17110b56y193110bad6c6fd53@mail.gmail.com>
Message-ID: <5.1.1.6.0.20070109163200.043a1240@sparrow.telecommunity.com>

At 08:10 AM 1/10/2007 +1100, Tim Delaney wrote:
>Guido van Rossum wrote:
>
> > On 1/8/07, Collin Winter <collinw at gmail.com> wrote:
> >> FWIW, I wasn't aware that things like "except T, (a, b, c):" were
> >> possible until I started work on this patch; I thought <name> was all
> >> you _could_ use.
> >
> > And, given the secial semantics for this position, I would support
> > changing the syntax to *only* allow a single simple name there -- not
> > even x.y.z. Thsh should handle 99.9% of existing code anyway.
> >
> > (And yes, if someone wants to add the 'as' keyword at the same time
> > that would be great!
>
>With such changed semantics (only allow simple name, simple name gets
>deleted at the end of the except: block) I would be strongly in favour of
>only having the changed semantics when the 'as' keyword is used.

That seems fine for 2.x, but 3.x should have Only One Way To Do It.  2.x 
needs both syntaxes to support writing forward-compatible code.


>Haven't checked the patch, but does it successfully delete the name in the
>case that another exception is thrown inside the except block?
>
>     try:
>         raise Exception()
>     except Exception as e:
>         try:
>             raise Exception()
>         except Exception as e2:
>             pass

Remmeber, the semantics are a source transform from:

    except <expr> as <name>:
        <suite>

to:

    except <expr>:
        <name> = <current exception value>
        try:
            <suite>
        finally:
            <name> = None
            del <name>

So just apply that expansion to all your examples to get your answers.  :)


>As I understand the proposed semantics, in the first case a NameError will
>be thrown at the 'print e' line in the first case, but not in the second
>case, because we never enter the second except: block. For consistency,
>maybe instead the semantics be that any name declared for catching an
>exception be deleted at the end of the try/except/else/finally clause i.e.
>
>     try:
>         pass
>     except AttributeError as e1:
>         pass
>     except NameError as e2:
>         pass
>
>is equivalent to:
>
>     try:
>         try:
>             pass
>         except AttributeError, e1:
>             pass
>         except NameError, e2:
>             pass
>     finally:
>         e1 = None
>         e2 = None
>         del e1
>         del e2

I find this explanation harder to explain.  The current model can be 
explained as:

"""The "as <name>" clause binds the exception value to <name>, then unbinds 
<name> when the <suite> is exited for any reason.  Any previous value bound 
to <name> is lost."""

I haven't thought of as simple a way to explain what you propose above.


From brett at python.org  Tue Jan  9 23:27:53 2007
From: brett at python.org (Brett Cannon)
Date: Tue, 9 Jan 2007 14:27:53 -0800
Subject: [Python-3000] possible new packages (PEP 3108)
Message-ID: <bbaeab100701091427ob6cc8c2jd45feb9ee22b9889@mail.gmail.com>

Since the threads on what modules to remove has puttered out (which I
take it to mean people no longer are having any fits over the list
which makes me happy =), I figured we can move on to the open issue of
new packages.  Lots of people seem to pick up on it when I posted the
full PEP so I am hoping we can actually agree on some shallow packages
to add to the stdlib to group some common stuff together.

Below is the list I currently have in the PEP.  The criteria I had was
that there must be three or more modules to go into a possible package
to bother listing it.  This is to prevent putting *everything* in a
package somehow and have that big headache come up.  Plus we are not
going to go with deep packages as Guido has stated he wants to keep
things shallow.

In the list below anything with a question mark next to it means I am
not totally sure that the module should go in the proposed package.  I
should also point out that their are no explicit package names being
suggested, just how to group stuff (although the group names could
easily act as package names in some cases).  Lastly, I used the
module's current name instead of any renaming since any approved
packaging will get shifted into the renaming section of the PEP and
can be dealt with there.

Anyway, since this is an Open Issue and slightly controversial I need
explicit voting from people on whether they like any of the proposed
packages.  So please don't be silent if there is a package you think
that should (or should not) happen.

-----------------------------------------

* 'collections' package

  + heapq
  + Queue
  + sets
  + UserDist (?)
  + UserList (?)

* Python implementations of built-in data types

  + sets
  + UserDict
  + UserList
  + UserString

* Platform-specific

  + windows

    - _winreg
    - msilib
    - msvcrt
    - winsound

  + unix

    - crypt
    - dbm
    - fcntl
    - fpectl
    - gdbm
    - grp
    - nis
    - pipes
    - pwd
    - readline (?)
    - resource
    - spwd
    - syslog
    - termios (?)
    - tty (?)

  - mac (all Mac-specific modules)

* Profiling

  + cProfile
  + profile
  + hotshot
  + pstats

* 'email' package

  + imaplib
  + mailbox
  + mailcap
  + mhlib
  + poplib

* Databases

  + anydbm
  + dbhash
  + dbm
  + bsddb
  + dumbdbm
  + gdbm
  + whichdb

* Audio

  + aifc
  + audioop
  + chunk
  + ossaudiodev
  + sndhdr
  + sunau
  + wave
  + winsound

* XML-RPC

  + DocXMLRPCServer
  + SimpleXMLRPCServer
  + xmlrpclib

* Internet (leaving out all questionable modules leads to a Web
  grouping)

  + BaseHTTPServer
  + cgi
  + CGIHTTPServer
  + cgitb
  + Cookie
  + cookielib
  + ftplib (?)
  + htmlentitydefs
  + htmllib
  + HTMLParser
  + httplib
  + netrc (?)
  + nntplib (?)
  + sgmllib (?)
  + SimpleHTTPServer
  + stringprep
  + urllib
  + urllib2
  + urlparse
  + webbrowser
  + wsgiref

* Servers

  + HTTP

    - BaseHTTPServer
    - CGIHTTPServer
    - DocXMLRPCServer
    - SimpleHTTPServer
    - SimpleXMLRPCServer
    - wsgiref

  + Socket

    - asynchat
    - asyncore
    - SocketServer

* Bytecode

  + codeop
  + compiler
  + dis
  + py_compile
  + compileall

* Compression

  + bz2
  + gzip
  + zlib

* Date & time

  + calendar
  + datetime
  + time

From guido at python.org  Tue Jan  9 23:43:07 2007
From: guido at python.org (Guido van Rossum)
Date: Tue, 9 Jan 2007 14:43:07 -0800
Subject: [Python-3000] self-contained exceptions
In-Reply-To: <5.1.1.6.0.20070109163200.043a1240@sparrow.telecommunity.com>
References: <1d85506f0701011438q6d8b65bfi4ef1e8f9fa509650@mail.gmail.com>
	<Pine.LNX.4.58.0701040139090.16811@server1.LFW.org>
	<5.1.1.6.0.20070104095412.029577a8@sparrow.telecommunity.com>
	<ca471dc20701041023q4fabfc76h34ed4a7b491774f4@mail.gmail.com>
	<5.1.1.6.0.20070108012059.04121fd0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070108122108.04b73328@sparrow.telecommunity.com>
	<43aa6ff70701080934w3d66b48dw17733c2f5a64b140@mail.gmail.com>
	<ca471dc20701090941x17110b56y193110bad6c6fd53@mail.gmail.com>
	<008601c73432$8f46f5f0$f100a8c0@ryoko>
	<5.1.1.6.0.20070109163200.043a1240@sparrow.telecommunity.com>
Message-ID: <ca471dc20701091443p641799b6w7a0ee42cb0f5d063@mail.gmail.com>

On 1/9/07, Phillip J. Eby <pje at telecommunity.com> wrote:
> That seems fine for 2.x, but 3.x should have Only One Way To Do It.  2.x
> needs both syntaxes to support writing forward-compatible code.

Really? I personally expect that that's a lost cause. A much more
modest goal would be to write code that doesn't need additional
changes after being run through the refactoring tool (sandbox/2to3).

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From collinw at gmail.com  Tue Jan  9 23:55:06 2007
From: collinw at gmail.com (Collin Winter)
Date: Tue, 9 Jan 2007 16:55:06 -0600
Subject: [Python-3000] PEP 3108 and modules to be removed (current list)
In-Reply-To: <bbaeab100701051214h54c0e1f9w73c2e059fafbb04d@mail.gmail.com>
References: <bbaeab100701051214h54c0e1f9w73c2e059fafbb04d@mail.gmail.com>
Message-ID: <43aa6ff70701091455w616f6503s1ff64a729aab821a@mail.gmail.com>

On 1/5/07, Brett Cannon <brett at python.org> wrote:
> Obsolete
> --------
>
> Becoming obsolete signifies that either another module in the stdlib
> or a widely distributed third-party library provides a better solution
> for what the module is meant for.

How about the sets module? The set and frozenset types would seem to
have made a separate module obsolete.

I've just refreshed SF patch #1500611, which removes all usages of the
sets module from the stdlib and test suite.

Collin Winter

From brett at python.org  Wed Jan 10 00:08:15 2007
From: brett at python.org (Brett Cannon)
Date: Tue, 9 Jan 2007 15:08:15 -0800
Subject: [Python-3000] PEP 3108 and modules to be removed (current list)
In-Reply-To: <43aa6ff70701091455w616f6503s1ff64a729aab821a@mail.gmail.com>
References: <bbaeab100701051214h54c0e1f9w73c2e059fafbb04d@mail.gmail.com>
	<43aa6ff70701091455w616f6503s1ff64a729aab821a@mail.gmail.com>
Message-ID: <bbaeab100701091508nbab73a5t4ab19b96432914a1@mail.gmail.com>

On 1/9/07, Collin Winter <collinw at gmail.com> wrote:
> On 1/5/07, Brett Cannon <brett at python.org> wrote:
> > Obsolete
> > --------
> >
> > Becoming obsolete signifies that either another module in the stdlib
> > or a widely distributed third-party library provides a better solution
> > for what the module is meant for.
>
> How about the sets module? The set and frozenset types would seem to
> have made a separate module obsolete.
>
> I've just refreshed SF patch #1500611, which removes all usages of the
> sets module from the stdlib and test suite.
>

Well, I have no issue removing it, but I know some people really like
having the Python version of the built-ins around (e.g., User*).  Plus
I don't know if PyPy or IronPython use it for their set
implementation.

What do other people think?

-Brett

From fdrake at acm.org  Wed Jan 10 00:24:36 2007
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Tue, 9 Jan 2007 18:24:36 -0500
Subject: [Python-3000] possible new packages (PEP 3108)
In-Reply-To: <bbaeab100701091427ob6cc8c2jd45feb9ee22b9889@mail.gmail.com>
References: <bbaeab100701091427ob6cc8c2jd45feb9ee22b9889@mail.gmail.com>
Message-ID: <200701091824.37024.fdrake@acm.org>

On Tuesday 09 January 2007 17:27, Brett Cannon wrote:
 > ? ? - syslog

There's nothing Unix-specific about syslog (other than history), but the 
syslog module itself is Unix-specific.  Given that there's a Python 
implementation of a syslog client in the logging module, would it make sense 
to refactor that into a separate syslog module (wherever it lands in the 
packages) and remove the Unix-specific module?


  -Fred

-- 
Fred L. Drake, Jr.   <fdrake at acm.org>

From mike.klaas at gmail.com  Wed Jan 10 00:50:40 2007
From: mike.klaas at gmail.com (Mike Klaas)
Date: Tue, 9 Jan 2007 15:50:40 -0800
Subject: [Python-3000] possible new packages (PEP 3108)
In-Reply-To: <bbaeab100701091427ob6cc8c2jd45feb9ee22b9889@mail.gmail.com>
References: <bbaeab100701091427ob6cc8c2jd45feb9ee22b9889@mail.gmail.com>
Message-ID: <3d2ce8cb0701091550p69531191xa1604784e568d912@mail.gmail.com>

On 1/9/07, Brett Cannon <brett at python.org> wrote:
>
> * 'collections' package
>
>   + heapq
>   + Queue

Putting this in a collections package might exacerbate the confusion
over its intended use.  It really is more of a threading
synchronization tool than a general queue datastructure (though it is
certainly possible to use in that fashion).

Perhaps the module can be renamed to conform to pep 8.  threadqueue?
Or, move into the threading module?

>   + sets

Does the sets module serve any purpose other than backward
compatibility?  Why not remove those module for py3k?

Especially considering:
In [6]: isinstance(set(), sets.BaseSet)
Out[6]: False

regards,
-Mike

From brett at python.org  Wed Jan 10 01:10:20 2007
From: brett at python.org (Brett Cannon)
Date: Tue, 9 Jan 2007 16:10:20 -0800
Subject: [Python-3000] possible new packages (PEP 3108)
In-Reply-To: <200701091824.37024.fdrake@acm.org>
References: <bbaeab100701091427ob6cc8c2jd45feb9ee22b9889@mail.gmail.com>
	<200701091824.37024.fdrake@acm.org>
Message-ID: <bbaeab100701091610k7cc3a844ta7f5ce022da50f15@mail.gmail.com>

On 1/9/07, Fred L. Drake, Jr. <fdrake at acm.org> wrote:
> On Tuesday 09 January 2007 17:27, Brett Cannon wrote:
>  > - syslog
>
> There's nothing Unix-specific about syslog (other than history), but the
> syslog module itself is Unix-specific.  Given that there's a Python
> implementation of a syslog client in the logging module, would it make sense
> to refactor that into a separate syslog module (wherever it lands in the
> packages) and remove the Unix-specific module?
>

If someone wants to put the time in to do it then great; less code to
maintain (which is point of removing these modules).

-Brett

From brett at python.org  Wed Jan 10 01:14:02 2007
From: brett at python.org (Brett Cannon)
Date: Tue, 9 Jan 2007 16:14:02 -0800
Subject: [Python-3000] possible new packages (PEP 3108)
In-Reply-To: <3d2ce8cb0701091550p69531191xa1604784e568d912@mail.gmail.com>
References: <bbaeab100701091427ob6cc8c2jd45feb9ee22b9889@mail.gmail.com>
	<3d2ce8cb0701091550p69531191xa1604784e568d912@mail.gmail.com>
Message-ID: <bbaeab100701091614h603bcc7s5d433ba9f3b56d9@mail.gmail.com>

On 1/9/07, Mike Klaas <mike.klaas at gmail.com> wrote:
> On 1/9/07, Brett Cannon <brett at python.org> wrote:
> >
> > * 'collections' package
> >
> >   + heapq
> >   + Queue
>
> Putting this in a collections package might exacerbate the confusion
> over its intended use.  It really is more of a threading
> synchronization tool than a general queue datastructure (though it is
> certainly possible to use in that fashion).
>
> Perhaps the module can be renamed to conform to pep 8.  threadqueue?
> Or, move into the threading module?
>

Moving it to the threading module has been discussed before (I think).
 I am fine with that if people really want it to be solely associated
with threading.  But renaming it threadqueue instead of adding it to
the 'threading' module seems a little silly if it is to be considered
so thread-specific.

> >   + sets
>
> Does the sets module serve any purpose other than backward
> compatibility?  Why not remove those module for py3k?
>
> Especially considering:
> In [6]: isinstance(set(), sets.BaseSet)
> Out[6]: False
>

Same reason that the User* modules still exist I suppose.  Plus I
don't know if PyPy or anyone else rely on it for their set/frozentset
implementation.

-Brett

From collinw at gmail.com  Wed Jan 10 01:20:03 2007
From: collinw at gmail.com (Collin Winter)
Date: Tue, 9 Jan 2007 18:20:03 -0600
Subject: [Python-3000] possible new packages (PEP 3108)
In-Reply-To: <bbaeab100701091614h603bcc7s5d433ba9f3b56d9@mail.gmail.com>
References: <bbaeab100701091427ob6cc8c2jd45feb9ee22b9889@mail.gmail.com>
	<3d2ce8cb0701091550p69531191xa1604784e568d912@mail.gmail.com>
	<bbaeab100701091614h603bcc7s5d433ba9f3b56d9@mail.gmail.com>
Message-ID: <43aa6ff70701091620p6f637a7k5f91a6f775ad7c29@mail.gmail.com>

On 1/9/07, Brett Cannon <brett at python.org> wrote:
> On 1/9/07, Mike Klaas <mike.klaas at gmail.com> wrote:
> > Does the sets module serve any purpose other than backward
> > compatibility?  Why not remove those module for py3k?
>
> Same reason that the User* modules still exist I suppose.  Plus I
> don't know if PyPy or anyone else rely on it for their set/frozentset
> implementation.

What's the concern with PyPy and IronPython? Don't they already have
copies of the sets module?

Collin Winter

From brett at python.org  Wed Jan 10 02:06:39 2007
From: brett at python.org (Brett Cannon)
Date: Tue, 9 Jan 2007 17:06:39 -0800
Subject: [Python-3000] possible new packages (PEP 3108)
In-Reply-To: <43aa6ff70701091620p6f637a7k5f91a6f775ad7c29@mail.gmail.com>
References: <bbaeab100701091427ob6cc8c2jd45feb9ee22b9889@mail.gmail.com>
	<3d2ce8cb0701091550p69531191xa1604784e568d912@mail.gmail.com>
	<bbaeab100701091614h603bcc7s5d433ba9f3b56d9@mail.gmail.com>
	<43aa6ff70701091620p6f637a7k5f91a6f775ad7c29@mail.gmail.com>
Message-ID: <bbaeab100701091706s4cfcb41cpfb93d39a409d42e9@mail.gmail.com>

On 1/9/07, Collin Winter <collinw at gmail.com> wrote:
> On 1/9/07, Brett Cannon <brett at python.org> wrote:
> > On 1/9/07, Mike Klaas <mike.klaas at gmail.com> wrote:
> > > Does the sets module serve any purpose other than backward
> > > compatibility?  Why not remove those module for py3k?
> >
> > Same reason that the User* modules still exist I suppose.  Plus I
> > don't know if PyPy or anyone else rely on it for their set/frozentset
> > implementation.
>
> What's the concern with PyPy and IronPython? Don't they already have
> copies of the sets module?
>

It's not a worry of whether they have a copy in their current version,
but whether we care to continue to be the official maintainers of the
module if they use it.  We could say they have to maintain it
themselves and remove the module, but there might be duplicated work
if they both use it.

But I am fine with removing it.  If no one speaks up to save it you
can consider it gone for Py3K.

-Brett

From rhamph at gmail.com  Wed Jan 10 02:24:28 2007
From: rhamph at gmail.com (Adam Olsen)
Date: Tue, 9 Jan 2007 18:24:28 -0700
Subject: [Python-3000] possible new packages (PEP 3108)
In-Reply-To: <bbaeab100701091427ob6cc8c2jd45feb9ee22b9889@mail.gmail.com>
References: <bbaeab100701091427ob6cc8c2jd45feb9ee22b9889@mail.gmail.com>
Message-ID: <aac2c7cb0701091724k679b56bkb98ad5d987f17aa8@mail.gmail.com>

On 1/9/07, Brett Cannon <brett at python.org> wrote:
> * Profiling
>
>   + cProfile
>   + profile
>   + hotshot
>   + pstats

Perhaps 'diagnostics' instead, with pdb thrown in?  Hopefully there
will be more tools in the future.

-- 
Adam Olsen, aka Rhamphoryncus

From collinw at gmail.com  Wed Jan 10 03:17:00 2007
From: collinw at gmail.com (Collin Winter)
Date: Tue, 9 Jan 2007 20:17:00 -0600
Subject: [Python-3000] self-contained exceptions
In-Reply-To: <ca471dc20701090941x17110b56y193110bad6c6fd53@mail.gmail.com>
References: <1d85506f0701011438q6d8b65bfi4ef1e8f9fa509650@mail.gmail.com>
	<5.1.1.6.0.20070103225854.042a3868@sparrow.telecommunity.com>
	<5.1.1.6.0.20070104004429.02a19820@sparrow.telecommunity.com>
	<Pine.LNX.4.58.0701040139090.16811@server1.LFW.org>
	<5.1.1.6.0.20070104095412.029577a8@sparrow.telecommunity.com>
	<ca471dc20701041023q4fabfc76h34ed4a7b491774f4@mail.gmail.com>
	<5.1.1.6.0.20070108012059.04121fd0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070108122108.04b73328@sparrow.telecommunity.com>
	<43aa6ff70701080934w3d66b48dw17733c2f5a64b140@mail.gmail.com>
	<ca471dc20701090941x17110b56y193110bad6c6fd53@mail.gmail.com>
Message-ID: <43aa6ff70701091817w7ef85d55l7ec4ed35640fb84e@mail.gmail.com>

On 1/9/07, Guido van Rossum <guido at python.org> wrote:
> On 1/8/07, Collin Winter <collinw at gmail.com> wrote:
> > FWIW, I wasn't aware that things like "except T, (a, b, c):" were
> > possible until I started work on this patch; I thought <name> was all
> > you _could_ use.
>
> And, given the secial semantics for this position, I would support
> changing the syntax to *only* allow a single simple name there -- not
> even x.y.z. Thsh should handle 99.9% of existing code anyway.
>
> (And yes, if someone wants to add the 'as' keyword at the same time
> that would be great!

Ask and you shall receive: patch #1631942 implements both the 'as'
keyword for exceptions and the new exception cleanup semantics.

Thanks,
Collin Winter

From janssen at parc.com  Wed Jan 10 03:18:21 2007
From: janssen at parc.com (Bill Janssen)
Date: Tue, 9 Jan 2007 18:18:21 PST
Subject: [Python-3000] possible new packages (PEP 3108)
In-Reply-To: <bbaeab100701091427ob6cc8c2jd45feb9ee22b9889@mail.gmail.com> 
References: <bbaeab100701091427ob6cc8c2jd45feb9ee22b9889@mail.gmail.com>
Message-ID: <07Jan9.181829pst."58648"@synergy1.parc.xerox.com>

OK, let me repeat myself.

I see no point in grouping modules just because they're servers.

I'd suggest a Web module containing:

  html:
    htmlentitydefs
    htmllib
    HTMLParser
    sgmllib (?)

  server:

    BaseHTTPServer
    cgi
    CGIHTTPServer
    Cookie
    wsgiref

  client:

    cookielib
    httplib
    urllib, urllib2, urlparse

  browser:  (or these could just be part of "client")

    webbrowser

"cgitb" is generic functionality, and should be merged into "traceback".

The classes in SimpleHTTPServer should be merged into BaseHTTPServer.

"urllib" and "urllib2" should be merged.  "urlparse" should be merged into urllib.

What's in SocketServer should be merged into "socket".

Perhaps "web" could be part of a higher-level "internet" package,
which would also include "email", and things like nntplib and
stringprep.

Bill

From brett at python.org  Wed Jan 10 04:34:05 2007
From: brett at python.org (Brett Cannon)
Date: Tue, 9 Jan 2007 19:34:05 -0800
Subject: [Python-3000] possible new packages (PEP 3108)
In-Reply-To: <284183686114447699@unknownmsgid>
References: <bbaeab100701091427ob6cc8c2jd45feb9ee22b9889@mail.gmail.com>
	<284183686114447699@unknownmsgid>
Message-ID: <bbaeab100701091934v4c99b32bn11b0be402218efca@mail.gmail.com>

On 1/9/07, Bill Janssen <janssen at parc.com> wrote:
> OK, let me repeat myself.
>

OK.

> I see no point in grouping modules just because they're servers.
>

That's fine.  But they do server a similar purpose and so it is a
legit suggestion.  If no one else likes it then it won't go anywhere.

I will take this as a -1 on any server-specific grouping.

> I'd suggest a Web module containing:
>
>   html:
>     htmlentitydefs
>     htmllib
>     HTMLParser
>     sgmllib (?)
>
>   server:
>
>     BaseHTTPServer
>     cgi
>     CGIHTTPServer
>     Cookie
>     wsgiref
>
>   client:
>
>     cookielib
>     httplib
>     urllib, urllib2, urlparse
>
>   browser:  (or these could just be part of "client")
>
>     webbrowser
>

I already have that package but without the subpackging which I am not
going to do.  As I said, I am not going to push for anything more than
a shallow (i.e., one level deep) package introduction.

> "cgitb" is generic functionality, and should be merged into "traceback".
>
> The classes in SimpleHTTPServer should be merged into BaseHTTPServer.
>
> "urllib" and "urllib2" should be merged.  "urlparse" should be merged into urllib.
>
> What's in SocketServer should be merged into "socket".
>

I am not dealing with any merging unless I have to.  Perhaps I should
make that a basic rule that I am not handling merges into a package
and that can be done separately.

> Perhaps "web" could be part of a higher-level "internet" package,
> which would also include "email", and things like nntplib and
> stringprep.
>

As I said, I am not going to try to make any package deep.

-Brett

From brett at python.org  Wed Jan 10 04:36:10 2007
From: brett at python.org (Brett Cannon)
Date: Tue, 9 Jan 2007 19:36:10 -0800
Subject: [Python-3000] possible new packages (PEP 3108)
In-Reply-To: <43aa6ff70701091620p6f637a7k5f91a6f775ad7c29@mail.gmail.com>
References: <bbaeab100701091427ob6cc8c2jd45feb9ee22b9889@mail.gmail.com>
	<3d2ce8cb0701091550p69531191xa1604784e568d912@mail.gmail.com>
	<bbaeab100701091614h603bcc7s5d433ba9f3b56d9@mail.gmail.com>
	<43aa6ff70701091620p6f637a7k5f91a6f775ad7c29@mail.gmail.com>
Message-ID: <bbaeab100701091936q77383303h642b3fb6d55eb4d3@mail.gmail.com>

On 1/9/07, Collin Winter <collinw at gmail.com> wrote:
> On 1/9/07, Brett Cannon <brett at python.org> wrote:
> > On 1/9/07, Mike Klaas <mike.klaas at gmail.com> wrote:
> > > Does the sets module serve any purpose other than backward
> > > compatibility?  Why not remove those module for py3k?
> >
> > Same reason that the User* modules still exist I suppose.  Plus I
> > don't know if PyPy or anyone else rely on it for their set/frozentset
> > implementation.
>
> What's the concern with PyPy and IronPython? Don't they already have
> copies of the sets module?
>

Well, it no longer matters as Guido just removed the module thanks to
your patch.  =)

-Brett

From anthony at interlink.com.au  Wed Jan 10 05:24:05 2007
From: anthony at interlink.com.au (Anthony Baxter)
Date: Wed, 10 Jan 2007 15:24:05 +1100
Subject: [Python-3000] Warning for 2.6 and greater
In-Reply-To: <200701091152.48048.anthony@interlink.com.au>
References: <200701081650.17353.anthony@interlink.com.au>
	<9e804ac0701080353x6e84bd55q72848de26783635d@mail.gmail.com>
	<200701091152.48048.anthony@interlink.com.au>
Message-ID: <200701101524.06515.anthony@interlink.com.au>

cc'ing python-dev - followups should probably go there, rather than 
the p3yk list.

So here's my latest plan:

- Add a Py3KDeprecationWarning, as a subclass of DeprecationWarning 
(or maybe just of Warning)
- Add a -W py3k shortcut, which is the same as 
"-W once:Py3kDeprecationWarning"
- Add a C int (as in previous patch) that is also set to 1 if the 
command line flag is specified. This is because there's no easy or 
quick way to check that a warning has been set - and calling into 
the warnings code is expensive, even if the C code warnings module 
is done. We can revisit this if the C version of warnings grows 
some extra features to make this less awful. 

For 2.6, we make a single -t (mixing tabs and spaces) the default, 
and maybe add -T to suppress this.

DeprecationWarning for 2.6:
- `foo` (backticks)
- input()

The following are the list of things that get 
Py3kDeprecationWarnings (so far):

integer division - make "-Q warn" the default when -W py3k specified
coerce() - going away
apply() - going away
intern() - moving to sys (we should also move it to sys, and make 
intern() - call sys.intern())
file.xreadlines() - going away
dict.has_key() - use 'in dict'
<> - use != (aka "MakeBarryCryDeprecationWarning")
__(get|set|del)slice__
__coerce__ - gone
__(div|idiv|rdiv)__ - no replacement?
__nonzero__ - we should add __bool__ to 2.6, and then warn people. 
  Not sure we can rename the nb_nonzero slot, though.

For the various __foo__ slots, it might be nice to warn when they're 
defined, rather than used, but I've not looked into how hard this 
would be to do, or whether it would still work with .pyc files and 
the like. On the other hand, warning on use might also let us pick 
up when C code modules use the same functions. 

There's other changes that are probably too hard to warn about, 
because there's not an easy replacement - the exec and print 
statements come to mind here. 

Comments? What else should get warnings?
Anthony

From guido at python.org  Wed Jan 10 06:31:31 2007
From: guido at python.org (Guido van Rossum)
Date: Tue, 9 Jan 2007 21:31:31 -0800
Subject: [Python-3000] test_hotshot failing -- what's up?
Message-ID: <ca471dc20701092131x43371332mda607ef78d57d6a2@mail.gmail.com>

In the p3yk branch, test_hotshot seems to be failing. Does anybody
have any idea why? Is this an artefact of the latest merge? (BTW
thanks for doing another merge, Thomas -- it's greatly appreciated!)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From jcarlson at uci.edu  Wed Jan 10 06:35:37 2007
From: jcarlson at uci.edu (Josiah Carlson)
Date: Tue, 09 Jan 2007 21:35:37 -0800
Subject: [Python-3000] possible new packages (PEP 3108)
In-Reply-To: <bbaeab100701091427ob6cc8c2jd45feb9ee22b9889@mail.gmail.com>
References: <bbaeab100701091427ob6cc8c2jd45feb9ee22b9889@mail.gmail.com>
Message-ID: <20070109212841.8D6A.JCARLSON@uci.edu>


"Brett Cannon" <brett at python.org> wrote:
> * Databases
>   + anydbm
>   + dbhash
>   + dbm
>   + bsddb
>   + dumbdbm
>   + gdbm
>   + whichdb

What about sqlite3?


> * Internet (leaving out all questionable modules leads to a Web
>   grouping)

I'm not a real big fan of the Internet package, but you forgot about
smtpd (an asyncore subclass that is an smtp server).


> * Servers
>   + HTTP
>     - BaseHTTPServer
>     - CGIHTTPServer
>     - DocXMLRPCServer
>     - SimpleHTTPServer
>     - SimpleXMLRPCServer
>     - wsgiref

Why not just call the above http_servers?


>   + Socket
> 
>     - asynchat
>     - asyncore
>     - SocketServer

And the above server_frameworks?  It may also make sense to merge the
functionality of asynchat into asyncore (there is already
dispatcher_with_send in asyncore).


 - Josiah


From jcarlson at uci.edu  Wed Jan 10 07:00:08 2007
From: jcarlson at uci.edu (Josiah Carlson)
Date: Tue, 09 Jan 2007 22:00:08 -0800
Subject: [Python-3000] self-contained exceptions
In-Reply-To: <43aa6ff70701091817w7ef85d55l7ec4ed35640fb84e@mail.gmail.com>
References: <ca471dc20701090941x17110b56y193110bad6c6fd53@mail.gmail.com>
	<43aa6ff70701091817w7ef85d55l7ec4ed35640fb84e@mail.gmail.com>
Message-ID: <20070109213607.8D6D.JCARLSON@uci.edu>


"Collin Winter" <collinw at gmail.com> wrote:
> On 1/9/07, Guido van Rossum <guido at python.org> wrote:
> > On 1/8/07, Collin Winter <collinw at gmail.com> wrote:
> > > FWIW, I wasn't aware that things like "except T, (a, b, c):" were
> > > possible until I started work on this patch; I thought <name> was all
> > > you _could_ use.
> >
> > And, given the secial semantics for this position, I would support
> > changing the syntax to *only* allow a single simple name there -- not
> > even x.y.z. Thsh should handle 99.9% of existing code anyway.
> >
> > (And yes, if someone wants to add the 'as' keyword at the same time
> > that would be great!
> 
> Ask and you shall receive: patch #1631942 implements both the 'as'
> keyword for exceptions and the new exception cleanup semantics.

I know that one of the points of all of this is so that the name that
the exc_info() stuff gets in the local scope doesn't propagate past the
except clause, but I can't help feeling as if removing the name isn't a
little overenthusiastic.

Here's an alternate semantic...

    try:
        ...
    except <exceptions> as <name>:
        <body>

... would become ...

    try:
        try:
            ...
        except <exceptions>:
            <name> = sys.exc_info()
            <body>
    finally:
        if '<name>' in locals():
            <name> = None

The benefit to the above is that the name only disappears from the
namespace if the user deletes it, otherwise the name is bound to None. 
We could do some trickery to see if the name is bound to the exception,
but that may be a bit of overkill.


 - Josiah


From anthony at interlink.com.au  Wed Jan 10 07:14:05 2007
From: anthony at interlink.com.au (Anthony Baxter)
Date: Wed, 10 Jan 2007 17:14:05 +1100
Subject: [Python-3000] test_hotshot failing -- what's up?
In-Reply-To: <ca471dc20701092131x43371332mda607ef78d57d6a2@mail.gmail.com>
References: <ca471dc20701092131x43371332mda607ef78d57d6a2@mail.gmail.com>
Message-ID: <200701101714.06542.anthony@interlink.com.au>

On Wednesday 10 January 2007 16:31, Guido van Rossum wrote:
> In the p3yk branch, test_hotshot seems to be failing. Does
> anybody have any idea why? Is this an artefact of the latest
> merge? (BTW thanks for doing another merge, Thomas -- it's
> greatly appreciated!)

It works fine for me...


From phd at phd.pp.ru  Wed Jan 10 08:47:10 2007
From: phd at phd.pp.ru (Oleg Broytmann)
Date: Wed, 10 Jan 2007 10:47:10 +0300
Subject: [Python-3000] possible new packages (PEP 3108)
In-Reply-To: <07Jan9.181829pst."58648"@synergy1.parc.xerox.com>
References: <bbaeab100701091427ob6cc8c2jd45feb9ee22b9889@mail.gmail.com>
	<07Jan9.181829pst."58648"@synergy1.parc.xerox.com>
Message-ID: <20070110074710.GB2867@phd.pp.ru>

On Tue, Jan 09, 2007 at 06:18:21PM -0800, Bill Janssen wrote:
>   server:
>     Cookie
> 
>   client:
>     cookielib

server:
   cookie

client:
   cookie

   ?

Oleg.
-- 
     Oleg Broytmann            http://phd.pp.ru/            phd at phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.

From thomas at python.org  Wed Jan 10 10:49:06 2007
From: thomas at python.org (Thomas Wouters)
Date: Wed, 10 Jan 2007 10:49:06 +0100
Subject: [Python-3000] test_hotshot failing -- what's up?
In-Reply-To: <ca471dc20701092131x43371332mda607ef78d57d6a2@mail.gmail.com>
References: <ca471dc20701092131x43371332mda607ef78d57d6a2@mail.gmail.com>
Message-ID: <9e804ac0701100149u24ea4be5sc73bce5430e49c9b@mail.gmail.com>

On 1/10/07, Guido van Rossum <guido at python.org> wrote:
>
> In the p3yk branch, test_hotshot seems to be failing. Does anybody
> have any idea why? Is this an artefact of the latest merge? (BTW
> thanks for doing another merge, Thomas -- it's greatly appreciated!)


No tests were failing for me, after the merge, or I wouldn't have checked it
in :) It looks like test_hotshot was run normally. It doesn't fail now,
either.

-- 
Thomas Wouters <thomas at python.org>

Hi! I'm a .signature virus! copy me into your .signature file to help me
spread!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20070110/2f9bf5e1/attachment.htm 

From pje at telecommunity.com  Wed Jan 10 15:28:03 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 10 Jan 2007 09:28:03 -0500
Subject: [Python-3000] self-contained exceptions
In-Reply-To: <20070109213607.8D6D.JCARLSON@uci.edu>
References: <43aa6ff70701091817w7ef85d55l7ec4ed35640fb84e@mail.gmail.com>
	<ca471dc20701090941x17110b56y193110bad6c6fd53@mail.gmail.com>
	<43aa6ff70701091817w7ef85d55l7ec4ed35640fb84e@mail.gmail.com>
Message-ID: <5.1.1.6.0.20070110092310.04def958@sparrow.telecommunity.com>

At 10:00 PM 1/9/2007 -0800, Josiah Carlson wrote:
>I know that one of the points of all of this is so that the name that
>the exc_info() stuff gets in the local scope doesn't propagate past the
>except clause,

That is the *entire* point of this, actualy.  :)  (Well, the "as" change 
was supposed to happen anyway.)


>but I can't help feeling as if removing the name isn't a
>little overenthusiastic.
>
>Here's an alternate semantic...
>
>     try:
>         ...
>     except <exceptions> as <name>:
>         <body>
>
>... would become ...
>
>     try:
>         try:
>             ...
>         except <exceptions>:
>             <name> = sys.exc_info()
>             <body>
>     finally:
>         if '<name>' in locals():
>             <name> = None

Please provide what you'd put in the reference manual as an explanation of 
this behavior.  (Remember: if the implementation is hard to explain...)


>The benefit to the above is that the name only disappears from the
>namespace if the user deletes it, otherwise the name is bound to None.

Please explain how this is a benefit.  By making any use of the name raise 
an error, we ensure that you will detect your mistake early, whereas 
binding the name to None may allow the value to be passed, stored, 
returned, or otherwise used, with the problem not being detected until 
later, when it may be much harder to trace back to the source.

In other words, eliminating the error is *not* a benefit here.  (Errors 
should not pass silently.)


From aahz at pythoncraft.com  Wed Jan 10 16:39:45 2007
From: aahz at pythoncraft.com (Aahz)
Date: Wed, 10 Jan 2007 07:39:45 -0800
Subject: [Python-3000] Warning for 2.6 and greater
In-Reply-To: <9e804ac0701090149p5552034dgc8c21c81e867b229@mail.gmail.com>
References: <200701081650.17353.anthony@interlink.com.au>
	<9e804ac0701080353x6e84bd55q72848de26783635d@mail.gmail.com>
	<200701091152.48048.anthony@interlink.com.au>
	<20070109052610.GA7620@panix.com>
	<9e804ac0701090149p5552034dgc8c21c81e867b229@mail.gmail.com>
Message-ID: <20070110153945.GA27437@panix.com>

On Tue, Jan 09, 2007, Thomas Wouters wrote:
> On 1/9/07, Aahz <aahz at pythoncraft.com> wrote:
>> 
>>Hey!  What's this about __cmp__?!
> 
> Python 3.0x (p3yk:53295, Jan  8 2007, 09:32:17)
> [GCC 4.1.2 20061115 (prerelease) (Debian 4.1.1-21)] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
> >>>object.__cmp__
> Traceback (most recent call last):
>  File "<stdin>", line 1, in <module>
> AttributeError: type object 'object' has no attribute '__cmp__'
> 
> E's dead. Stone dead, miss.

Unfortunately, I haven't set up a checkout of py3k yet, could you please
test what this does:

    class CheapCompare:
        def __init__(self, value):
            self.value = value
        def __cmp__(self, other):
            x = self.value
            y = other.value
            if x == y:
                return 0
            elif x > y:
                return 1
            else:
                return -1

    print cmp(CheapCompare(1), CheapCompare(2))

If that doesn't work, I expect there to be some unhappy Numeric people.
OTOH, if that does work, what exactly will you be warning about?
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"Plus ca change, plus c'est la meme chose."

From guido at python.org  Wed Jan 10 16:43:33 2007
From: guido at python.org (Guido van Rossum)
Date: Wed, 10 Jan 2007 07:43:33 -0800
Subject: [Python-3000] test_hotshot failing -- what's up?
In-Reply-To: <9e804ac0701100149u24ea4be5sc73bce5430e49c9b@mail.gmail.com>
References: <ca471dc20701092131x43371332mda607ef78d57d6a2@mail.gmail.com>
	<9e804ac0701100149u24ea4be5sc73bce5430e49c9b@mail.gmail.com>
Message-ID: <ca471dc20701100743y4a238124y293e2c8c38bfcdd8@mail.gmail.com>

Hm, then maybe it started failing after I applied Collin Winter's
'except' patches. Collin, does test_hotshot pass or fail for you, with
or without your patches?

On 1/10/07, Thomas Wouters <thomas at python.org> wrote:
>
>
> On 1/10/07, Guido van Rossum <guido at python.org> wrote:
> > In the p3yk branch, test_hotshot seems to be failing. Does anybody
> > have any idea why? Is this an artefact of the latest merge? (BTW
> > thanks for doing another merge, Thomas -- it's greatly appreciated!)
>
>
> No tests were failing for me, after the merge, or I wouldn't have checked it
> in :) It looks like test_hotshot was run normally. It doesn't fail now,
> either.
>
> --
> Thomas Wouters < thomas at python.org>
>
> Hi! I'm a .signature virus! copy me into your .signature file to help me
> spread!


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Wed Jan 10 16:47:40 2007
From: guido at python.org (Guido van Rossum)
Date: Wed, 10 Jan 2007 07:47:40 -0800
Subject: [Python-3000] self-contained exceptions
In-Reply-To: <20070109213607.8D6D.JCARLSON@uci.edu>
References: <ca471dc20701090941x17110b56y193110bad6c6fd53@mail.gmail.com>
	<43aa6ff70701091817w7ef85d55l7ec4ed35640fb84e@mail.gmail.com>
	<20070109213607.8D6D.JCARLSON@uci.edu>
Message-ID: <ca471dc20701100747i5bf31745h3547a7a16dcdcb3c@mail.gmail.com>

On 1/9/07, Josiah Carlson <jcarlson at uci.edu> wrote:
> I know that one of the points of all of this is so that the name that
> the exc_info() stuff gets in the local scope doesn't propagate past the
> except clause, but I can't help feeling as if removing the name isn't a
> little overenthusiastic.

What's your use case, except for not breaking existing code?  Apart
from the latter, I can't think of a single use case that isn't served
just as well by *copying* the object reference into another variable.

BTW perhaps we should add the same semantics and syntax to 'with expr
as var'? I can't think of a reasonable use case for keeping var alive
after the with-statement terminates either. (Nor for using anything
more complex than a local variable, BTW.)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From jimjjewett at gmail.com  Wed Jan 10 17:32:03 2007
From: jimjjewett at gmail.com (Jim Jewett)
Date: Wed, 10 Jan 2007 11:32:03 -0500
Subject: [Python-3000] possible new packages (PEP 3108)
In-Reply-To: <bbaeab100701091427ob6cc8c2jd45feb9ee22b9889@mail.gmail.com>
References: <bbaeab100701091427ob6cc8c2jd45feb9ee22b9889@mail.gmail.com>
Message-ID: <fb6fbf560701100832r2d5b2fc3o79534362530f3da9@mail.gmail.com>

On 1/9/07, Brett Cannon <brett at python.org> wrote:
> Anyway, since this is an Open Issue and slightly controversial I need
> explicit voting from people on whether they like any of the proposed
> packages.  So please don't be silent if there is a package you think
> that should (or should not) happen.

I explicitly like reducing the the number of top-level modules in
almost *any* sensible manner.

For me (and I doubt I'm alone), the most obvious way to find something
is to look in the Lib directory (which misses C modules), followed by
the global module index.  With over 200 choices (windows), this
doesn't work as well as it should.

> * 'collections' package

>   + heapq
>   + Queue
>   + sets
>   + UserDist (?)
>   + UserList (?)

Except for queue (which should move to threads), this is the most clearcut.

If it is rejected, then the "builtin data types" or even a more
generic datatypes (for modules that mostly let you work with a certain
type of data, like csv or datetime) would be better than nothing.

> * Platform-specific

Yes, please.  While some are there for reasons of historical accident,
the fact remains that they don't work on all platforms, and are
therefore only a source of confusion to beginners.

The platform-specfic implementations of path should also move there.

If all builds end up distributing some of the platform-specific
packages for compatibility ("but I want to treat it as a Mac path"),
that's OK -- at least people will understand what it is there for, and
why it acts "funny".

> * Profiling

Should this be combined with debugging?

> * 'email' package
(yes)

> * Databases
>
>   + anydbm
>   + dbhash
>   + dbm
>   + bsddb
>   + dumbdbm
>   + gdbm
>   + whichdb

also shelve, sqlite3

And this reopens the question of whether there should be a way to
separate the "standard" databases from those that were added locally.

> * Audio
>
>   + aifc
>   + audioop
>   + chunk
>   + ossaudiodev
>   + sndhdr
>   + sunau
>   + wave
>   + winsound

This at least gets them out of my non-audio hair, but I'm not sure how
much sense it makes to actual users.  Should there be an audio module
(or the package __init__.py) which imports the "right"
platform-specific module for simple uses?

> * XML-RPC

>   + DocXMLRPCServer
>   + SimpleXMLRPCServer
>   + xmlrpclib

Somehow, this grouping seems artificial.  I also wonder whether the
Doc server ought to go in a docs package, or the debug package.  That
said, I still like it better than no change.

> * Internet (leaving out all questionable modules leads to a Web
>   grouping)

I would like this if we weren't worried about hierarchy depth.

I would also like it if web were pulled out separately, the way email is.

> * Servers

I would most prefer to see these as subgroupings under net (or web).

I would rather see them lumped in with the rest of web than broken out
into a separate servers package.

Though even just this -- grouping them as "servers", is an improvement
over the status quo.


> * Bytecode
> * Compression
> * Date & time

yes

From jcarlson at uci.edu  Wed Jan 10 17:36:01 2007
From: jcarlson at uci.edu (Josiah Carlson)
Date: Wed, 10 Jan 2007 08:36:01 -0800
Subject: [Python-3000] self-contained exceptions
In-Reply-To: <ca471dc20701100747i5bf31745h3547a7a16dcdcb3c@mail.gmail.com>
References: <20070109213607.8D6D.JCARLSON@uci.edu>
	<ca471dc20701100747i5bf31745h3547a7a16dcdcb3c@mail.gmail.com>
Message-ID: <20070110083355.8D70.JCARLSON@uci.edu>


"Guido van Rossum" <guido at python.org> wrote:
> 
> On 1/9/07, Josiah Carlson <jcarlson at uci.edu> wrote:
> > I know that one of the points of all of this is so that the name that
> > the exc_info() stuff gets in the local scope doesn't propagate past the
> > except clause, but I can't help feeling as if removing the name isn't a
> > little overenthusiastic.
> 
> What's your use case, except for not breaking existing code?  Apart
> from the latter, I can't think of a single use case that isn't served
> just as well by *copying* the object reference into another variable.
> 
> BTW perhaps we should add the same semantics and syntax to 'with expr
> as var'? I can't think of a reasonable use case for keeping var alive
> after the with-statement terminates either. (Nor for using anything
> more complex than a local variable, BTW.)

I suppose when compared with the 'with' syntax, which is also supposed
to handle error conditions and cleanup, I am convinced that propagating
the name post-context/exception management should be avoided.  Consider
the alternate semantics withdrawn (if it wasn't considered rejected ;) ).


 - Josiah


From janssen at parc.com  Wed Jan 10 17:43:39 2007
From: janssen at parc.com (Bill Janssen)
Date: Wed, 10 Jan 2007 08:43:39 PST
Subject: [Python-3000] possible new packages (PEP 3108)
In-Reply-To: <bbaeab100701091934v4c99b32bn11b0be402218efca@mail.gmail.com> 
References: <bbaeab100701091427ob6cc8c2jd45feb9ee22b9889@mail.gmail.com>
	<284183686114447699@unknownmsgid>
	<bbaeab100701091934v4c99b32bn11b0be402218efca@mail.gmail.com>
Message-ID: <07Jan10.084345pst."58648"@synergy1.parc.xerox.com>

> I already have that package but without the subpackging which I am not
> going to do.  As I said, I am not going to push for anything more than
> a shallow (i.e., one level deep) package introduction.

> I am not dealing with any merging unless I have to.  Perhaps I should
> make that a basic rule that I am not handling merges into a package
> and that can be done separately.

Then I don't see the point of re-organizing at all.  Just drop the old
modules, and leave well enough alone.  -1 on the reorg.

Bill

From nnorwitz at gmail.com  Wed Jan 10 17:44:09 2007
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Wed, 10 Jan 2007 08:44:09 -0800
Subject: [Python-3000] test_hotshot failing -- what's up?
In-Reply-To: <ca471dc20701100743y4a238124y293e2c8c38bfcdd8@mail.gmail.com>
References: <ca471dc20701092131x43371332mda607ef78d57d6a2@mail.gmail.com>
	<9e804ac0701100149u24ea4be5sc73bce5430e49c9b@mail.gmail.com>
	<ca471dc20701100743y4a238124y293e2c8c38bfcdd8@mail.gmail.com>
Message-ID: <ee2a432c0701100844t1cba19c3w3593a671b1830aab@mail.gmail.com>

Did the import magic number get changed?  That was the cause of my
last crash in test_hotshot (not related to this though).

n

On 1/10/07, Guido van Rossum <guido at python.org> wrote:
> Hm, then maybe it started failing after I applied Collin Winter's
> 'except' patches. Collin, does test_hotshot pass or fail for you, with
> or without your patches?
>
> On 1/10/07, Thomas Wouters <thomas at python.org> wrote:
> >
> >
> > On 1/10/07, Guido van Rossum <guido at python.org> wrote:
> > > In the p3yk branch, test_hotshot seems to be failing. Does anybody
> > > have any idea why? Is this an artefact of the latest merge? (BTW
> > > thanks for doing another merge, Thomas -- it's greatly appreciated!)
> >
> >
> > No tests were failing for me, after the merge, or I wouldn't have checked it
> > in :) It looks like test_hotshot was run normally. It doesn't fail now,
> > either.
> >
> > --
> > Thomas Wouters < thomas at python.org>
> >
> > Hi! I'm a .signature virus! copy me into your .signature file to help me
> > spread!
>
>
> --
> --Guido van Rossum (home page: http://www.python.org/~guido/)
> _______________________________________________
> Python-3000 mailing list
> Python-3000 at python.org
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe: http://mail.python.org/mailman/options/python-3000/nnorwitz%40gmail.com
>

From janssen at parc.com  Wed Jan 10 17:45:47 2007
From: janssen at parc.com (Bill Janssen)
Date: Wed, 10 Jan 2007 08:45:47 PST
Subject: [Python-3000] possible new packages (PEP 3108)
In-Reply-To: <fb6fbf560701100832r2d5b2fc3o79534362530f3da9@mail.gmail.com> 
References: <bbaeab100701091427ob6cc8c2jd45feb9ee22b9889@mail.gmail.com>
	<fb6fbf560701100832r2d5b2fc3o79534362530f3da9@mail.gmail.com>
Message-ID: <07Jan10.084550pst."58648"@synergy1.parc.xerox.com>

> > * XML-RPC
> 
> >   + DocXMLRPCServer
> >   + SimpleXMLRPCServer
> >   + xmlrpclib
> 
> Somehow, this grouping seems artificial.  I also wonder whether the
> Doc server ought to go in a docs package, or the debug package.  That
> said, I still like it better than no change.

How about putting these into an web.xmlrpc package?

Bill

From jcarlson at uci.edu  Wed Jan 10 17:53:46 2007
From: jcarlson at uci.edu (Josiah Carlson)
Date: Wed, 10 Jan 2007 08:53:46 -0800
Subject: [Python-3000] self-contained exceptions
In-Reply-To: <5.1.1.6.0.20070110092310.04def958@sparrow.telecommunity.com>
References: <20070109213607.8D6D.JCARLSON@uci.edu>
	<5.1.1.6.0.20070110092310.04def958@sparrow.telecommunity.com>
Message-ID: <20070110083609.8D73.JCARLSON@uci.edu>


"Phillip J. Eby" <pje at telecommunity.com> wrote:
> At 10:00 PM 1/9/2007 -0800, Josiah Carlson wrote:
[snip]
> Please provide what you'd put in the reference manual as an explanation of 
> this behavior.  (Remember: if the implementation is hard to explain...)

If the name to the right of the 'as' keyword perpetuates beyond the end
of the try/except/finally clause, it will be rebound to None before user
code is executed further.


> >The benefit to the above is that the name only disappears from the
> >namespace if the user deletes it, otherwise the name is bound to None.
> 
> Please explain how this is a benefit.  By making any use of the name raise 
> an error, we ensure that you will detect your mistake early, whereas 
> binding the name to None may allow the value to be passed, stored, 
> returned, or otherwise used, with the problem not being detected until 
> later, when it may be much harder to trace back to the source.
> 
> In other words, eliminating the error is *not* a benefit here.  (Errors 
> should not pass silently.)

I'm not sure that the error would pass silently for very long, as None
can't really do a whole lot of things.  Then again, any duration is
probably too much, as you say.


 - Josiah


From veloso at verylowsodium.com  Wed Jan 10 18:05:59 2007
From: veloso at verylowsodium.com (Greg Falcon)
Date: Wed, 10 Jan 2007 12:05:59 -0500
Subject: [Python-3000] self-contained exceptions
In-Reply-To: <ca471dc20701100747i5bf31745h3547a7a16dcdcb3c@mail.gmail.com>
References: <ca471dc20701090941x17110b56y193110bad6c6fd53@mail.gmail.com>
	<43aa6ff70701091817w7ef85d55l7ec4ed35640fb84e@mail.gmail.com>
	<20070109213607.8D6D.JCARLSON@uci.edu>
	<ca471dc20701100747i5bf31745h3547a7a16dcdcb3c@mail.gmail.com>
Message-ID: <3cdcefb80701100905p1e98c12dke0e21890e2b984c@mail.gmail.com>

Well, now I have to delurk.

I've been watching with growing excitement.  Between the changes to
the scope rules for list comprehensions, and this change for except
blocks, I wondered if Python might have been moving in the direction
of block scoped variables where appropriate as a philosophy.

I almost posted about this yesterday, but didn't.  I thought that the
fact these two changes had to do with scope was pure coincidence.
(After all, one change is meant to add a symmetry to the language, and
one is meant to fix potential gc problems.)

But, on 1/10/07, Guido van Rossum <guido at python.org> wrote:
> BTW perhaps we should add the same semantics and syntax to 'with expr
> as var'? I can't think of a reasonable use case for keeping var alive
> after the with-statement terminates either. (Nor for using anything
> more complex than a local variable, BTW.)

+1, though I'm probably +1 for the wrong reasons. :)  If it is
desirable for Python to delete local variables in the face of certain
syntactic constructs, then there simply does not exist a more
appropriate time to do so than 'with x as y'.  (Before your post, I
thought the motivation of this proposal was "we'd rather not delete
this variable at all, but the user will create cycles if we don't, so
it's a necessary evil.")

I'm coming from a philosophy, though, that believes any time a
syntactic construct creates a local binding and a new suite to use it
in, the binding shouldn't escape that suite.  Yes, I'm really talking
about 'for', and I know this is probably a controversial view.  But
don't the same arguments apply?  (It's probably an error to use the
variable outside the suite, and if you really need to, you could be
explicit and store another reference to it in a separate local.)

I'd also like to add that this "foo=none; del foo" approach to fake
block-scopes is clever, but probably deserves a bit of extra error
checking as well.  Specifically, Python should generate a
SyntaxWarning whenever one of these block-scope constructs binds to a
local that was previously assigned to.  I'm worried about this mistake
going unnoticed:

re = get_resource()
...
try:
  something()
except RareException as re:
  ... # handle it
...
re.use_resource()

Which *appears* to work fine most of the time, but when the
RareException is raised and handled, suddenly the use_resource() line
generates a surprising UnboundLocalError.  These are the sort of
things that should be caught ahead of time if possible.

Greg F

From jcarlson at uci.edu  Wed Jan 10 18:32:57 2007
From: jcarlson at uci.edu (Josiah Carlson)
Date: Wed, 10 Jan 2007 09:32:57 -0800
Subject: [Python-3000] self-contained exceptions
In-Reply-To: <3cdcefb80701100905p1e98c12dke0e21890e2b984c@mail.gmail.com>
References: <ca471dc20701100747i5bf31745h3547a7a16dcdcb3c@mail.gmail.com>
	<3cdcefb80701100905p1e98c12dke0e21890e2b984c@mail.gmail.com>
Message-ID: <20070110092555.8D76.JCARLSON@uci.edu>


"Greg Falcon" <veloso at verylowsodium.com> wrote:
[snip]
> I'm coming from a philosophy, though, that believes any time a
> syntactic construct creates a local binding and a new suite to use it
> in, the binding shouldn't escape that suite.  Yes, I'm really talking
> about 'for', and I know this is probably a controversial view.  But
> don't the same arguments apply?  (It's probably an error to use the
> variable outside the suite, and if you really need to, you could be
> explicit and store another reference to it in a separate local.)

Yeah, I like reusing the variable names bound in a for loop.  And unless
we are going to attempt to simplify the <assignment target> in...

    for <assignment target> in ...:
        ...

to only be a bare name, and not things like 'i, (j, k)', then removing
names is going to be difficult.  I would be -1 on any changes to the for
loop syntax (value unpacking is so darn convenient), and without syntax
change, then variable bleeding semantics would be difficult to change 
(which I am also -1 on).


> re = get_resource()
> ...
> try:
>   something()
> except RareException as re:
>   ... # handle it
> ...
> re.use_resource()
> 
> Which *appears* to work fine most of the time, but when the
> RareException is raised and handled, suddenly the use_resource() line
> generates a surprising UnboundLocalError.  These are the sort of
> things that should be caught ahead of time if possible.

According to the translation rules Colin has previously provided...

    re = get_resource()
    ...
    try:
        something()
    except RareException as re:
        ...
    ...
    re.use_resource()

is translated into...

    re = get_resource()
    ...
    try:
        try:
            something()
        except RareException:
            re = <current exception>
            ...
    finally:
        re = None
        del re
    ...
    re.use_resource()

That is, the 're = None; del re' stuff is executed regardless of whether
the except body is executed.  The user would be notified right away.


 - Josiah


From jcarlson at uci.edu  Wed Jan 10 18:36:52 2007
From: jcarlson at uci.edu (Josiah Carlson)
Date: Wed, 10 Jan 2007 09:36:52 -0800
Subject: [Python-3000] possible new packages (PEP 3108)
In-Reply-To: <07Jan10.084550pst."58648"@synergy1.parc.xerox.com>
References: <fb6fbf560701100832r2d5b2fc3o79534362530f3da9@mail.gmail.com>
	<07Jan10.084550pst."58648"@synergy1.parc.xerox.com>
Message-ID: <20070110093333.8D79.JCARLSON@uci.edu>


Bill Janssen <janssen at parc.com> wrote:
> 
> > > * XML-RPC
> > 
> > >   + DocXMLRPCServer
> > >   + SimpleXMLRPCServer
> > >   + xmlrpclib
> > 
> > Somehow, this grouping seems artificial.  I also wonder whether the
> > Doc server ought to go in a docs package, or the debug package.  That
> > said, I still like it better than no change.
> 
> How about putting these into an web.xmlrpc package?

Too deep?  Also, from what I understand, XML-RPC is a specialized web
server, so the two Servers might fit within the http server package, and
xmlrpclib fitting with the http client stuff.

 - Josiah


From collinw at gmail.com  Wed Jan 10 18:46:31 2007
From: collinw at gmail.com (Collin Winter)
Date: Wed, 10 Jan 2007 11:46:31 -0600
Subject: [Python-3000] self-contained exceptions
In-Reply-To: <20070110092555.8D76.JCARLSON@uci.edu>
References: <ca471dc20701100747i5bf31745h3547a7a16dcdcb3c@mail.gmail.com>
	<3cdcefb80701100905p1e98c12dke0e21890e2b984c@mail.gmail.com>
	<20070110092555.8D76.JCARLSON@uci.edu>
Message-ID: <43aa6ff70701100946l1e5913d4i4b6e047736376c65@mail.gmail.com>

On 1/10/07, Josiah Carlson <jcarlson at uci.edu> wrote:
> According to the translation rules Colin has previously provided...

Phillip came up the translation, I just implemented it.

>     re = get_resource()
>     ...
>     try:
>         something()
>     except RareException as re:
>         ...
>     ...
>     re.use_resource()
>
> is translated into...
>
>     re = get_resource()
>     ...
>     try:
>         try:
>             something()
>         except RareException:
>             re = <current exception>
>             ...
>     finally:
>         re = None
>         del re
>     ...
>     re.use_resource()
>
> That is, the 're = None; del re' stuff is executed regardless of whether
> the except body is executed.  The user would be notified right away.

The correct translation is

    re = get_resource()
    ...
    try:
        something()
    except RareException as re:
         try:
             ...
         finally:
             re = None
             del re
    ...
    re.use_resource()


So 're = None; del re;' is *only* executed if the except body is executed.

Collin Winter

From pje at telecommunity.com  Wed Jan 10 18:47:34 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 10 Jan 2007 12:47:34 -0500
Subject: [Python-3000] self-contained exceptions
In-Reply-To: <ca471dc20701100747i5bf31745h3547a7a16dcdcb3c@mail.gmail.co
 m>
References: <20070109213607.8D6D.JCARLSON@uci.edu>
	<ca471dc20701090941x17110b56y193110bad6c6fd53@mail.gmail.com>
	<43aa6ff70701091817w7ef85d55l7ec4ed35640fb84e@mail.gmail.com>
	<20070109213607.8D6D.JCARLSON@uci.edu>
Message-ID: <5.1.1.6.0.20070110124448.0299ac90@sparrow.telecommunity.com>

At 07:47 AM 1/10/2007 -0800, Guido van Rossum wrote:
>BTW perhaps we should add the same semantics and syntax to 'with expr
>as var'? I can't think of a reasonable use case
[snip]
>... for using anything
>more complex than a local variable, BTW.)

with multi(ctx1, ctx2, ...)  as var1, var2, ...:

(if multi is a function that returns a context manager wrapping the other 
contexts and returning the tuple their __enter__ values).


From collinw at gmail.com  Wed Jan 10 18:49:37 2007
From: collinw at gmail.com (Collin Winter)
Date: Wed, 10 Jan 2007 11:49:37 -0600
Subject: [Python-3000] test_hotshot failing -- what's up?
In-Reply-To: <ee2a432c0701100844t1cba19c3w3593a671b1830aab@mail.gmail.com>
References: <ca471dc20701092131x43371332mda607ef78d57d6a2@mail.gmail.com>
	<9e804ac0701100149u24ea4be5sc73bce5430e49c9b@mail.gmail.com>
	<ca471dc20701100743y4a238124y293e2c8c38bfcdd8@mail.gmail.com>
	<ee2a432c0701100844t1cba19c3w3593a671b1830aab@mail.gmail.com>
Message-ID: <43aa6ff70701100949s2e081121m17860139910f803e@mail.gmail.com>

On 1/10/07, Neal Norwitz <nnorwitz at gmail.com> wrote:
> Did the import magic number get changed?  That was the cause of my
> last crash in test_hotshot (not related to this though).

I don't think so (at least, I didn't change it).

> On 1/10/07, Guido van Rossum <guido at python.org> wrote:
> > Hm, then maybe it started failing after I applied Collin Winter's
> > 'except' patches. Collin, does test_hotshot pass or fail for you, with
> > or without your patches?

test_hotshot passes both with and without the patches for me. The
attached patch may fix things for you, though; it correctly restricts
the N in "except E as N" to a NAME token (I had accidentally left it
as <test>), plus patches the compiler package.

Thanks,
Collin Winter
-------------- next part --------------
A non-text attachment was scrubbed...
Name: fix_compiler_parser.patch
Type: text/x-diff
Size: 8538 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-3000/attachments/20070110/fe689d5b/attachment-0001.bin 

From jimjjewett at gmail.com  Wed Jan 10 18:50:30 2007
From: jimjjewett at gmail.com (Jim Jewett)
Date: Wed, 10 Jan 2007 12:50:30 -0500
Subject: [Python-3000] possible new packages (PEP 3108)
In-Reply-To: <-4021698297383678739@unknownmsgid>
References: <bbaeab100701091427ob6cc8c2jd45feb9ee22b9889@mail.gmail.com>
	<284183686114447699@unknownmsgid>
	<bbaeab100701091934v4c99b32bn11b0be402218efca@mail.gmail.com>
	<-4021698297383678739@unknownmsgid>
Message-ID: <fb6fbf560701100950n160aeaf3w9abe69dc6b1ff26c@mail.gmail.com>

On 1/10/07, Bill Janssen <janssen at parc.com> wrote:

> > I am not dealing with any merging unless I have to.  Perhaps I should
> > make that a basic rule that I am not handling merges into a package
> > and that can be done separately.

> Then I don't see the point of re-organizing at all.  Just drop the old
> modules, and leave well enough alone.  -1 on the reorg.

I think he meant "This PEP is not dealing with anything other than
complete existing modules."

If you want to combine modules, or move code to a more appropriate
place, that might be good too, but it isn't in Brett's scope.  All he
is doing is

(1)  Getting rid of stuff that has already been replaced, and
(2)  (Maybe) moving modules closer together so that it will be easier
to see what  refactoring is needed.

Eliminating some additional modules afterwards would be good too, but
there may not be time, and step (2) is worthwhile on its own.

-jJ

From veloso at verylowsodium.com  Wed Jan 10 18:53:23 2007
From: veloso at verylowsodium.com (Greg Falcon)
Date: Wed, 10 Jan 2007 12:53:23 -0500
Subject: [Python-3000] self-contained exceptions
In-Reply-To: <20070110092555.8D76.JCARLSON@uci.edu>
References: <ca471dc20701100747i5bf31745h3547a7a16dcdcb3c@mail.gmail.com>
	<3cdcefb80701100905p1e98c12dke0e21890e2b984c@mail.gmail.com>
	<20070110092555.8D76.JCARLSON@uci.edu>
Message-ID: <3cdcefb80701100953q338e2f1fk96459cded3929ebd@mail.gmail.com>

On 1/10/07, Josiah Carlson <jcarlson at uci.edu> wrote:
> According to the translation rules Colin has previously provided...
>
>     re = get_resource()
>     ...
>     try:
>         something()
>     except RareException as re:
>         ...
>     ...
>     re.use_resource()
>
> is translated into...
>
>     re = get_resource()
>     ...
>     try:
>         try:
>             something()
>         except RareException:
>             re = <current exception>
>             ...
>     finally:
>         re = None
>         del re
>     ...
>     re.use_resource()

That's not what I understand the transformation to be.  I thought the
transformation we were talking about was:

    re = get_resource()
    ...
    try:
        something()
    except RareException, re:
        try:
            ...
        finally:
            re = None
            del re
    ...
    re.use_resource()

Per these rules:
On 1/7/07, Collin Winter <collinw at gmail.com> wrote:
> > On 1/4/07, Phillip J. Eby <pje at telecommunity.com> wrote:
> > >       except ExcType, e:
> > >           try:
> > >               # body
> > >           finally:
> > >               e = None
> > >               del e
> The transformation is as Phillip outlined above, with extra logic to
> handle the case where e is a tuple or list.

Perhaps the example transformation you provided is better, in that at
least you can't make the mistake I'm worried about.  But it seems a
bit gratuitous (and expensive) to delete N variables when there are N
except clauses.

In any event, if I was mistaken about the transformation that occurs,
I apologize and withdraw my SyntaxWarning suggestion.

Greg F

From jimjjewett at gmail.com  Wed Jan 10 18:56:34 2007
From: jimjjewett at gmail.com (Jim Jewett)
Date: Wed, 10 Jan 2007 12:56:34 -0500
Subject: [Python-3000] self-contained exceptions
In-Reply-To: <43aa6ff70701100946l1e5913d4i4b6e047736376c65@mail.gmail.com>
References: <ca471dc20701100747i5bf31745h3547a7a16dcdcb3c@mail.gmail.com>
	<3cdcefb80701100905p1e98c12dke0e21890e2b984c@mail.gmail.com>
	<20070110092555.8D76.JCARLSON@uci.edu>
	<43aa6ff70701100946l1e5913d4i4b6e047736376c65@mail.gmail.com>
Message-ID: <fb6fbf560701100956x616e2ed2pd8267299fc98fc6@mail.gmail.com>

On 1/10/07, Collin Winter <collinw at gmail.com> wrote:
> Phillip came up the translation, I just implemented it. ...

>     re = get_resource()
...
>     except RareException as re:
...
>     re.use_resource()

> So 're = None; del re;' is *only* executed if the except body is executed.

So re.use_resource() will usually work (as it does today).

The difference is that when it doesn't work, you'll see something
about NoneType instead the exception type.

(1)  So the warning might be useful even now.

(2)  It may become more important after the patch, since "Where did I
get an Exception?" is easier to figure out than "How did I get a
None?"

-jJ

From pje at telecommunity.com  Wed Jan 10 18:57:23 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 10 Jan 2007 12:57:23 -0500
Subject: [Python-3000] self-contained exceptions
In-Reply-To: <3cdcefb80701100905p1e98c12dke0e21890e2b984c@mail.gmail.com
 >
References: <ca471dc20701100747i5bf31745h3547a7a16dcdcb3c@mail.gmail.com>
	<ca471dc20701090941x17110b56y193110bad6c6fd53@mail.gmail.com>
	<43aa6ff70701091817w7ef85d55l7ec4ed35640fb84e@mail.gmail.com>
	<20070109213607.8D6D.JCARLSON@uci.edu>
	<ca471dc20701100747i5bf31745h3547a7a16dcdcb3c@mail.gmail.com>
Message-ID: <5.1.1.6.0.20070110124925.0422cb40@sparrow.telecommunity.com>

At 12:05 PM 1/10/2007 -0500, Greg Falcon wrote:
>I'm coming from a philosophy, though, that believes any time a
>syntactic construct creates a local binding and a new suite to use it
>in, the binding shouldn't escape that suite.  Yes, I'm really talking
>about 'for', and I know this is probably a controversial view.  But
>don't the same arguments apply?  (It's probably an error to use the
>variable outside the suite, and if you really need to, you could be
>explicit and store another reference to it in a separate local.)

Breaking out of a loop with a found value is way too common a use case to 
want to delete the iteration variable.


>I'd also like to add that this "foo=none; del foo" approach to fake
>block-scopes is clever, but probably deserves a bit of extra error
>checking as well.  Specifically, Python should generate a
>SyntaxWarning whenever one of these block-scope constructs binds to a
>local that was previously assigned to.  I'm worried about this mistake
>going unnoticed:
>
>re = get_resource()
>...
>try:
>   something()
>except RareException as re:
>   ... # handle it
>...
>re.use_resource()
>
>Which *appears* to work fine most of the time, but when the
>RareException is raised and handled, suddenly the use_resource() line
>generates a surprising UnboundLocalError.  These are the sort of
>things that should be caught ahead of time if possible.

We could go even further, and say it's a syntax error to bind or refer to 
<name> outside the except suite where it's bound, unless it's being used in 
another except suite.

Or, we could look at it this way: "except <expr> as <tmpvar>" and "with 
<expr> as <tmpvar> (,<tmpvar>)*", and say that a tmpvar is a normal local 
variable, *except* that it may only be used within the block where it is 
defined, or in another block that uses it as a <tmpvar>.

So, you can do as many "as" bindings of the same name as you want, but you 
can't use that name for any "non-as" use.

We could also, for that matter, prohibit:

    with x as y:
        with z as y:

since when the inner block exits, y will no longer be bound.

But I'm not sure whether we really care about any of these points.


From janssen at parc.com  Wed Jan 10 19:00:02 2007
From: janssen at parc.com (Bill Janssen)
Date: Wed, 10 Jan 2007 10:00:02 PST
Subject: [Python-3000] possible new packages (PEP 3108)
In-Reply-To: <20070110093333.8D79.JCARLSON@uci.edu> 
References: <fb6fbf560701100832r2d5b2fc3o79534362530f3da9@mail.gmail.com>
	<07Jan10.084550pst."58648"@synergy1.parc.xerox.com>
	<20070110093333.8D79.JCARLSON@uci.edu>
Message-ID: <07Jan10.100006pst."58648"@synergy1.parc.xerox.com>

> Also, from what I understand, XML-RPC is a specialized web
> server, so the two Servers might fit within the http server package, and
> xmlrpclib fitting with the http client stuff.

Bill

From janssen at parc.com  Wed Jan 10 19:02:00 2007
From: janssen at parc.com (Bill Janssen)
Date: Wed, 10 Jan 2007 10:02:00 PST
Subject: [Python-3000] possible new packages (PEP 3108)
In-Reply-To: <fb6fbf560701100950n160aeaf3w9abe69dc6b1ff26c@mail.gmail.com> 
References: <bbaeab100701091427ob6cc8c2jd45feb9ee22b9889@mail.gmail.com>
	<284183686114447699@unknownmsgid>
	<bbaeab100701091934v4c99b32bn11b0be402218efca@mail.gmail.com>
	<-4021698297383678739@unknownmsgid>
	<fb6fbf560701100950n160aeaf3w9abe69dc6b1ff26c@mail.gmail.com>
Message-ID: <07Jan10.100201pst."58648"@synergy1.parc.xerox.com>

Jim Jewett says:
> If you want to combine modules, or move code to a more appropriate
> place, that might be good too, but it isn't in Brett's scope.  All he
> is doing is
> 
> (1)  Getting rid of stuff that has already been replaced, and
> (2)  (Maybe) moving modules closer together so that it will be easier
> to see what  refactoring is needed.
> 
> Eliminating some additional modules afterwards would be good too, but
> there may not be time, and step (2) is worthwhile on its own.

Yes, it's just that I disagree with you about that.  Step (2) causes
more trouble than it's worth without really doing a reorg.

Bill

From collinw at gmail.com  Wed Jan 10 19:05:48 2007
From: collinw at gmail.com (Collin Winter)
Date: Wed, 10 Jan 2007 12:05:48 -0600
Subject: [Python-3000] self-contained exceptions
In-Reply-To: <fb6fbf560701100956x616e2ed2pd8267299fc98fc6@mail.gmail.com>
References: <ca471dc20701100747i5bf31745h3547a7a16dcdcb3c@mail.gmail.com>
	<3cdcefb80701100905p1e98c12dke0e21890e2b984c@mail.gmail.com>
	<20070110092555.8D76.JCARLSON@uci.edu>
	<43aa6ff70701100946l1e5913d4i4b6e047736376c65@mail.gmail.com>
	<fb6fbf560701100956x616e2ed2pd8267299fc98fc6@mail.gmail.com>
Message-ID: <43aa6ff70701101005x7d23ab3ck2b40604be318462@mail.gmail.com>

On 1/10/07, Jim Jewett <jimjjewett at gmail.com> wrote:
> On 1/10/07, Collin Winter <collinw at gmail.com> wrote:
> >     re = get_resource()
> ...
> >     except RareException as re:
> ...
> >     re.use_resource()
>
> > So 're = None; del re;' is *only* executed if the except body is executed.
>
> So re.use_resource() will usually work (as it does today).
>
> The difference is that when it doesn't work, you'll see something
> about NoneType instead the exception type.

You're forgetting about the 'del N'; re.use_resource() will fail with
a NameError at the call site.

Collin Winter

From veloso at verylowsodium.com  Wed Jan 10 19:13:22 2007
From: veloso at verylowsodium.com (Greg Falcon)
Date: Wed, 10 Jan 2007 13:13:22 -0500
Subject: [Python-3000] self-contained exceptions
In-Reply-To: <fb6fbf560701100956x616e2ed2pd8267299fc98fc6@mail.gmail.com>
References: <ca471dc20701100747i5bf31745h3547a7a16dcdcb3c@mail.gmail.com>
	<3cdcefb80701100905p1e98c12dke0e21890e2b984c@mail.gmail.com>
	<20070110092555.8D76.JCARLSON@uci.edu>
	<43aa6ff70701100946l1e5913d4i4b6e047736376c65@mail.gmail.com>
	<fb6fbf560701100956x616e2ed2pd8267299fc98fc6@mail.gmail.com>
Message-ID: <3cdcefb80701101013l53a79a3fu1dfdec3e2fd1a7b1@mail.gmail.com>

Some very good points made here.

On 1/10/07, Josiah Carlson <jcarlson at uci.edu> wrote:
> Yeah, I like reusing the variable names bound in a for loop.  And unless
> we are going to attempt to simplify the <assignment target> in...
>
>     for <assignment target> in ...:
>         ...
>
> to only be a bare name, and not things like 'i, (j, k)', then removing
> names is going to be difficult.  I would be -1 on any changes to the for
> loop syntax (value unpacking is so darn convenient), and without syntax
> change, then variable bleeding semantics would be difficult to change
> (which I am also -1 on).

I had overlooked that.  Good point on the complications of unpacking.

Just like for loops, though, there are probably cases where looking at
a leaked 'with x as y' variable is as useful.  Say you're trying to do
an atomic database operation, wrapped in 'with db as cursor'.  You
want the transactional behavior inside the structure, but afterwards
you might want to do some read operations to the database.  Could be
'cursor'

I'm not saying I think that sort of code is a good idea, but if
"accessing leaked variables is convenient" is an argument to keep
for-loop behavior, perhaps it's an argument for keeping with-statement
behavior as well.  (Even if you don't like my example, who knows what
sorts of things people will do with context managers?)

On 1/10/07, Jim Jewett <jimjjewett at gmail.com> wrote:
> So re.use_resource() will usually work (as it does today).
>
> The difference is that when it doesn't work, you'll see something
> about NoneType instead the exception type.

Actually, the problem won't be NoneType, it'll be an
UnboundLocalError, which could be confusing.  ("what do you mean,
unbound?  I assigned it right there.")  But the point that my code
sample was already broken is well taken.  A warning would be mildly
nice to have, but this transformation won't introduce new bugs like I
initially thought.

Thanks,
Greg F

From jcarlson at uci.edu  Wed Jan 10 19:23:26 2007
From: jcarlson at uci.edu (Josiah Carlson)
Date: Wed, 10 Jan 2007 10:23:26 -0800
Subject: [Python-3000] self-contained exceptions
In-Reply-To: <3cdcefb80701100953q338e2f1fk96459cded3929ebd@mail.gmail.com>
References: <20070110092555.8D76.JCARLSON@uci.edu>
	<3cdcefb80701100953q338e2f1fk96459cded3929ebd@mail.gmail.com>
Message-ID: <20070110102049.8D7C.JCARLSON@uci.edu>


"Greg Falcon" <veloso at verylowsodium.com> wrote:
[snip]
> Per these rules:
> On 1/7/07, Collin Winter <collinw at gmail.com> wrote:
> > > On 1/4/07, Phillip J. Eby <pje at telecommunity.com> wrote:
> > > >       except ExcType, e:
> > > >           try:
> > > >               # body
> > > >           finally:
> > > >               e = None
> > > >               del e
> > The transformation is as Phillip outlined above, with extra logic to
> > handle the case where e is a tuple or list.
> 
> Perhaps the example transformation you provided is better, in that at
> least you can't make the mistake I'm worried about.  But it seems a
> bit gratuitous (and expensive) to delete N variables when there are N
> except clauses.

You are right.  Hrm...  I see the reasoning behind putting the finally
clause inside the except suite, but I can't help feeling that it would
be better outside.  Yeah you delete the name every time, but I think
consistancy would be better in this case (also, deleting a local is
really fast, I wouldn't worry about time considerations).

 - Josiah


From guido at python.org  Wed Jan 10 19:34:19 2007
From: guido at python.org (Guido van Rossum)
Date: Wed, 10 Jan 2007 10:34:19 -0800
Subject: [Python-3000] test_hotshot failing -- what's up?
In-Reply-To: <43aa6ff70701100949s2e081121m17860139910f803e@mail.gmail.com>
References: <ca471dc20701092131x43371332mda607ef78d57d6a2@mail.gmail.com>
	<9e804ac0701100149u24ea4be5sc73bce5430e49c9b@mail.gmail.com>
	<ca471dc20701100743y4a238124y293e2c8c38bfcdd8@mail.gmail.com>
	<ee2a432c0701100844t1cba19c3w3593a671b1830aab@mail.gmail.com>
	<43aa6ff70701100949s2e081121m17860139910f803e@mail.gmail.com>
Message-ID: <ca471dc20701101034h76dba02bs44c888afbcb8f889@mail.gmail.com>

On 1/10/07, Collin Winter <collinw at gmail.com> wrote:
> On 1/10/07, Neal Norwitz <nnorwitz at gmail.com> wrote:
> > Did the import magic number get changed?  That was the cause of my
> > last crash in test_hotshot (not related to this though).
>
> I don't think so (at least, I didn't change it).

Actually, Neal meant that if the bytecode changed, you *should* have
changed the magic number (and I should have remembered when I reviewed
it). If the magic number doesn't change, bogus bytecode compiled by a
previous version of the compiler may survive and throw things off.

It looks as if the hotshot test passes after a "make clean" which is
another indication that this is indeed the cause of the failure.
(Still waiting to verify this result on a much slower machine where
the same thing happened.)

> > On 1/10/07, Guido van Rossum <guido at python.org> wrote:
> > > Hm, then maybe it started failing after I applied Collin Winter's
> > > 'except' patches. Collin, does test_hotshot pass or fail for you, with
> > > or without your patches?
>
> test_hotshot passes both with and without the patches for me. The
> attached patch may fix things for you, though; it correctly restricts
> the N in "except E as N" to a NAME token (I had accidentally left it
> as <test>), plus patches the compiler package.

Ah, thanks, I noticed that but forgot to let you know. :-(

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Wed Jan 10 19:53:31 2007
From: guido at python.org (Guido van Rossum)
Date: Wed, 10 Jan 2007 10:53:31 -0800
Subject: [Python-3000] test_hotshot failing -- what's up?
In-Reply-To: <ca471dc20701101034h76dba02bs44c888afbcb8f889@mail.gmail.com>
References: <ca471dc20701092131x43371332mda607ef78d57d6a2@mail.gmail.com>
	<9e804ac0701100149u24ea4be5sc73bce5430e49c9b@mail.gmail.com>
	<ca471dc20701100743y4a238124y293e2c8c38bfcdd8@mail.gmail.com>
	<ee2a432c0701100844t1cba19c3w3593a671b1830aab@mail.gmail.com>
	<43aa6ff70701100949s2e081121m17860139910f803e@mail.gmail.com>
	<ca471dc20701101034h76dba02bs44c888afbcb8f889@mail.gmail.com>
Message-ID: <ca471dc20701101053o4f8e6c88oaad9a0372edc05b@mail.gmail.com>

On 1/10/07, Guido van Rossum <guido at python.org> wrote:
> On 1/10/07, Collin Winter <collinw at gmail.com> wrote:
> > On 1/10/07, Neal Norwitz <nnorwitz at gmail.com> wrote:
> > > Did the import magic number get changed?  That was the cause of my
> > > last crash in test_hotshot (not related to this though).
> >
> > I don't think so (at least, I didn't change it).
>
> Actually, Neal meant that if the bytecode changed, you *should* have
> changed the magic number (and I should have remembered when I reviewed
> it). If the magic number doesn't change, bogus bytecode compiled by a
> previous version of the compiler may survive and throw things off.
>
> It looks as if the hotshot test passes after a "make clean" which is
> another indication that this is indeed the cause of the failure.
> (Still waiting to verify this result on a much slower machine where
> the same thing happened.)

It passes there too now, but I'm confused as to whether the magic
number shuold be changed. Since not too many people ought to be
affected I'm tempted to forget about the whole episode...

> > > On 1/10/07, Guido van Rossum <guido at python.org> wrote:
> > > > Hm, then maybe it started failing after I applied Collin Winter's
> > > > 'except' patches. Collin, does test_hotshot pass or fail for you, with
> > > > or without your patches?
> >
> > test_hotshot passes both with and without the patches for me. The
> > attached patch may fix things for you, though; it correctly restricts
> > the N in "except E as N" to a NAME token (I had accidentally left it
> > as <test>), plus patches the compiler package.
>
> Ah, thanks, I noticed that but forgot to let you know. :-(

And checked in. You should ask for developer privileges! I think MvL
is the one to ask.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From collinw at gmail.com  Wed Jan 10 20:17:36 2007
From: collinw at gmail.com (Collin Winter)
Date: Wed, 10 Jan 2007 13:17:36 -0600
Subject: [Python-3000] test_hotshot failing -- what's up?
In-Reply-To: <ca471dc20701101053o4f8e6c88oaad9a0372edc05b@mail.gmail.com>
References: <ca471dc20701092131x43371332mda607ef78d57d6a2@mail.gmail.com>
	<9e804ac0701100149u24ea4be5sc73bce5430e49c9b@mail.gmail.com>
	<ca471dc20701100743y4a238124y293e2c8c38bfcdd8@mail.gmail.com>
	<ee2a432c0701100844t1cba19c3w3593a671b1830aab@mail.gmail.com>
	<43aa6ff70701100949s2e081121m17860139910f803e@mail.gmail.com>
	<ca471dc20701101034h76dba02bs44c888afbcb8f889@mail.gmail.com>
	<ca471dc20701101053o4f8e6c88oaad9a0372edc05b@mail.gmail.com>
Message-ID: <43aa6ff70701101117r4ec20cc9i61bad2352e637801@mail.gmail.com>

On 1/10/07, Guido van Rossum <guido at python.org> wrote:
> On 1/10/07, Guido van Rossum <guido at python.org> wrote:
> > On 1/10/07, Collin Winter <collinw at gmail.com> wrote:
> > > On 1/10/07, Neal Norwitz <nnorwitz at gmail.com> wrote:
> > > > Did the import magic number get changed?  That was the cause of my
> > > > last crash in test_hotshot (not related to this though).
> > >
> > > I don't think so (at least, I didn't change it).
> >
> > Actually, Neal meant that if the bytecode changed, you *should* have
> > changed the magic number (and I should have remembered when I reviewed
> > it). If the magic number doesn't change, bogus bytecode compiled by a
> > previous version of the compiler may survive and throw things off.
> >
> > It looks as if the hotshot test passes after a "make clean" which is
> > another indication that this is indeed the cause of the failure.
> > (Still waiting to verify this result on a much slower machine where
> > the same thing happened.)
>
> It passes there too now, but I'm confused as to whether the magic
> number shuold be changed. Since not too many people ought to be
> affected I'm tempted to forget about the whole episode...

I thought the magic number was only supposed to change when someone
changes what the bytecode actually does (as opposed to changing what
bytecode is generated)?

Thanks,
Collin Winter

From guido at python.org  Wed Jan 10 20:26:54 2007
From: guido at python.org (Guido van Rossum)
Date: Wed, 10 Jan 2007 11:26:54 -0800
Subject: [Python-3000] test_hotshot failing -- what's up?
In-Reply-To: <43aa6ff70701101117r4ec20cc9i61bad2352e637801@mail.gmail.com>
References: <ca471dc20701092131x43371332mda607ef78d57d6a2@mail.gmail.com>
	<9e804ac0701100149u24ea4be5sc73bce5430e49c9b@mail.gmail.com>
	<ca471dc20701100743y4a238124y293e2c8c38bfcdd8@mail.gmail.com>
	<ee2a432c0701100844t1cba19c3w3593a671b1830aab@mail.gmail.com>
	<43aa6ff70701100949s2e081121m17860139910f803e@mail.gmail.com>
	<ca471dc20701101034h76dba02bs44c888afbcb8f889@mail.gmail.com>
	<ca471dc20701101053o4f8e6c88oaad9a0372edc05b@mail.gmail.com>
	<43aa6ff70701101117r4ec20cc9i61bad2352e637801@mail.gmail.com>
Message-ID: <ca471dc20701101126g905c4c9w6b11e122fd12fc9e@mail.gmail.com>

On 1/10/07, Collin Winter <collinw at gmail.com> wrote:
> > It passes there too now, but I'm confused as to whether the magic
> > number shuold be changed. Since not too many people ought to be
> > affected I'm tempted to forget about the whole episode...
>
> I thought the magic number was only supposed to change when someone
> changes what the bytecode actually does (as opposed to changing what
> bytecode is generated)?

Yeah, that's right. I really don't know what was going on. But a "make
clean" fixed it and that's good enough for me in this stage (we're not
talking "release candidate" here yet :-).

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From g.brandl at gmx.net  Wed Jan 10 20:32:02 2007
From: g.brandl at gmx.net (Georg Brandl)
Date: Wed, 10 Jan 2007 20:32:02 +0100
Subject: [Python-3000] Warning for 2.6 and greater
In-Reply-To: <200701101524.06515.anthony@interlink.com.au>
References: <200701081650.17353.anthony@interlink.com.au>	<9e804ac0701080353x6e84bd55q72848de26783635d@mail.gmail.com>	<200701091152.48048.anthony@interlink.com.au>
	<200701101524.06515.anthony@interlink.com.au>
Message-ID: <eo3evi$7dr$1@sea.gmane.org>

Anthony Baxter schrieb:
> cc'ing python-dev - followups should probably go there, rather than 
> the p3yk list.
> 
> So here's my latest plan:

[...]

> Comments? What else should get warnings?

Previously, there was some talk about making these warnings guarded by
#ifs. Is that still an option?

I guess there are quite a few people who won't start moving to Python 3.0
with 2.6, or even when 3.1 is out, as long as their program works fine with
the 2.x line. There's no need to punish them with extra overhead.

cheers,
Georg


From brett at python.org  Wed Jan 10 20:32:42 2007
From: brett at python.org (Brett Cannon)
Date: Wed, 10 Jan 2007 11:32:42 -0800
Subject: [Python-3000] possible new packages (PEP 3108)
In-Reply-To: <20070109212841.8D6A.JCARLSON@uci.edu>
References: <bbaeab100701091427ob6cc8c2jd45feb9ee22b9889@mail.gmail.com>
	<20070109212841.8D6A.JCARLSON@uci.edu>
Message-ID: <bbaeab100701101132h2049697dj70fc89026532a878@mail.gmail.com>

On 1/9/07, Josiah Carlson <jcarlson at uci.edu> wrote:
>
> "Brett Cannon" <brett at python.org> wrote:
> > * Databases
> >   + anydbm
> >   + dbhash
> >   + dbm
> >   + bsddb
> >   + dumbdbm
> >   + gdbm
> >   + whichdb
>
> What about sqlite3?
>

Oops.  =)

>
> > * Internet (leaving out all questionable modules leads to a Web
> >   grouping)
>
> I'm not a real big fan of the Internet package, but you forgot about
> smtpd (an asyncore subclass that is an smtp server).
>

Or is that more of an email thing?

>
> > * Servers
> >   + HTTP
> >     - BaseHTTPServer
> >     - CGIHTTPServer
> >     - DocXMLRPCServer
> >     - SimpleHTTPServer
> >     - SimpleXMLRPCServer
> >     - wsgiref
>
> Why not just call the above http_servers?
>

It's an option.

>
> >   + Socket
> >
> >     - asynchat
> >     - asyncore
> >     - SocketServer
>
> And the above server_frameworks?  It may also make sense to merge the
> functionality of asynchat into asyncore (there is already
> dispatcher_with_send in asyncore).
>

If you want to deprecate asynchat and move it to asyncore, go for it,
you maintain it.  =)  Just will want to do that in 2.x probably.

-Brett

From brett at python.org  Wed Jan 10 20:43:25 2007
From: brett at python.org (Brett Cannon)
Date: Wed, 10 Jan 2007 11:43:25 -0800
Subject: [Python-3000] possible new packages (PEP 3108)
In-Reply-To: <3334693148356440096@unknownmsgid>
References: <bbaeab100701091427ob6cc8c2jd45feb9ee22b9889@mail.gmail.com>
	<284183686114447699@unknownmsgid>
	<bbaeab100701091934v4c99b32bn11b0be402218efca@mail.gmail.com>
	<-4021698297383678739@unknownmsgid>
	<fb6fbf560701100950n160aeaf3w9abe69dc6b1ff26c@mail.gmail.com>
	<3334693148356440096@unknownmsgid>
Message-ID: <bbaeab100701101143q175436c6wd215f8b626868a97@mail.gmail.com>

On 1/10/07, Bill Janssen <janssen at parc.com> wrote:
> Jim Jewett says:
> > If you want to combine modules, or move code to a more appropriate
> > place, that might be good too, but it isn't in Brett's scope.  All he
> > is doing is
> >
> > (1)  Getting rid of stuff that has already been replaced, and
> > (2)  (Maybe) moving modules closer together so that it will be easier
> > to see what  refactoring is needed.
> >
> > Eliminating some additional modules afterwards would be good too, but
> > there may not be time, and step (2) is worthwhile on its own.
>
> Yes, it's just that I disagree with you about that.  Step (2) causes
> more trouble than it's worth without really doing a reorg.
>

Well, this is why it is an Open Issue.  Guido explicitly said that he
only cleared renaming for modules that broke PEP 8 standards and
ditching cruft.  I decided to take this on just because I was already
looking at the entire stdlib and so I saw some patterns in terms of
what we have support for.  But I have limited time and with Guido
having not explicitly said he would clear anything like this I didn't
plan to spend a lot of effort on it.

The enthusiasm for this is not exactly high and people are already
arguing over depth and what should go where when there is conflict,
etc.  These are the same issues that come up every time someone
attempts to do this and leads to the idea dying on the mailing list
before it even reaches the PEP stage.

I think to keep my life simple, I am going to let this Open Issue live
until Saturday morning when I get up.  If we manage to get some
consensus by then on any packages they will survive and will be
proposed to Guido.  Otherwise I am removing the Open Issue and I will
leave it up to someone else to write up a PEP and pitch it to the list
and Guido.

-Brett

From barry at python.org  Wed Jan 10 20:48:00 2007
From: barry at python.org (Barry Warsaw)
Date: Wed, 10 Jan 2007 14:48:00 -0500
Subject: [Python-3000] possible new packages (PEP 3108)
In-Reply-To: <bbaeab100701101132h2049697dj70fc89026532a878@mail.gmail.com>
References: <bbaeab100701091427ob6cc8c2jd45feb9ee22b9889@mail.gmail.com>
	<20070109212841.8D6A.JCARLSON@uci.edu>
	<bbaeab100701101132h2049697dj70fc89026532a878@mail.gmail.com>
Message-ID: <082A8692-BFA8-4CDC-8467-8491D50877D4@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Jan 10, 2007, at 2:32 PM, Brett Cannon wrote:

>>> * Internet (leaving out all questionable modules leads to a Web
>>>   grouping)
>>
>> I'm not a real big fan of the Internet package, but you forgot about
>> smtpd (an asyncore subclass that is an smtp server).
>>
>
> Or is that more of an email thing?

It depends.  I'm trying hard to stay out of this thread, but smtpd.py  
is more of a server that speaks a protocol that happens to be used  
sometimes <wink> for email.  I really don't know where best to put  
it, but another alternative is to have a 'protocols' package perhaps  
with client and server sub-modules where available.

Two other quick things: I have an adaptation of smtpd.py that  
provides LMTP support (RFC 2033) that might make it into Python 2.6.   
Also, I'd really love it if the top level 'email' package name was  
reserved for the current code.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRaVC8HEjvBPtnXfVAQKg4wQAp62flRgO8ksWDfpm1Bj/QBVQqj6gZUpo
SBrhIRGfzuT6rx6m3YzRcglrXkYFTPyJfch0LGlKqvej4hoj3B1C6nC/+F3GB1jl
kVbrkzdb4cy/v6w7cSQpjfP1gqnnbTtEibZydF6ZSZsQEDTPmaM+rFgEtHVY5JwS
8Byo8TLUNF8=
=4906
-----END PGP SIGNATURE-----

From armin.ronacher at active-4.com  Wed Jan 10 20:45:54 2007
From: armin.ronacher at active-4.com (Armin Ronacher)
Date: Wed, 10 Jan 2007 19:45:54 +0000 (UTC)
Subject: [Python-3000] =?utf-8?q?Changing_default_=5F=5Frepr=5F=5F_of_type?=
Message-ID: <loom.20070110T203322-85@post.gmane.org>

Hoi,

> gmane workaround

What drives me nuts personally is the default __repr__ of classes. (type
instances and not class instances).

Currently it looks like this::

    >>> Foo
    <class '__main__.Foo'>
    >>> mymodule.Foo
    <class 'mymodule.Foo'>

IMHO a better way is this one:

    >>> Foo
    Foo
    >>> mymodule.Foo
    mymodule.Foo

Because only classes and singletons (like True/False/None(/Ellipsis?)) have a
similar repr that's not such a big problem.

Especially for datetime this will be much more consistent then:

    >>> datetime.datetime.now()
    datetime.datetime(2007, 1, 10, 20, 43, 7, 52890)
    >>> datetime.datetime
    datetime.datetime

The advantages:

-   Smaller Output when working with the CLI
-   evalable (okay, that's not the best argument)
-   consistent (see the datetime example)

Regards,
Armin


From collinw at gmail.com  Wed Jan 10 20:56:38 2007
From: collinw at gmail.com (Collin Winter)
Date: Wed, 10 Jan 2007 13:56:38 -0600
Subject: [Python-3000] test_hotshot failing -- what's up?
In-Reply-To: <ca471dc20701101126g905c4c9w6b11e122fd12fc9e@mail.gmail.com>
References: <ca471dc20701092131x43371332mda607ef78d57d6a2@mail.gmail.com>
	<9e804ac0701100149u24ea4be5sc73bce5430e49c9b@mail.gmail.com>
	<ca471dc20701100743y4a238124y293e2c8c38bfcdd8@mail.gmail.com>
	<ee2a432c0701100844t1cba19c3w3593a671b1830aab@mail.gmail.com>
	<43aa6ff70701100949s2e081121m17860139910f803e@mail.gmail.com>
	<ca471dc20701101034h76dba02bs44c888afbcb8f889@mail.gmail.com>
	<ca471dc20701101053o4f8e6c88oaad9a0372edc05b@mail.gmail.com>
	<43aa6ff70701101117r4ec20cc9i61bad2352e637801@mail.gmail.com>
	<ca471dc20701101126g905c4c9w6b11e122fd12fc9e@mail.gmail.com>
Message-ID: <43aa6ff70701101156q7218429bq127fb80a28a02549@mail.gmail.com>

On 1/10/07, Guido van Rossum <guido at python.org> wrote:
> On 1/10/07, Collin Winter <collinw at gmail.com> wrote:
> > > It passes there too now, but I'm confused as to whether the magic
> > > number shuold be changed. Since not too many people ought to be
> > > affected I'm tempted to forget about the whole episode...
> >
> > I thought the magic number was only supposed to change when someone
> > changes what the bytecode actually does (as opposed to changing what
> > bytecode is generated)?
>
> Yeah, that's right. I really don't know what was going on. But a "make
> clean" fixed it and that's good enough for me in this stage (we're not
> talking "release candidate" here yet :-).

In that case, I'm going to go ahead and close patch #1631942, since
the test_hotshot failure was the only thing keeping it open.

Collin Winter

From jcarlson at uci.edu  Wed Jan 10 21:27:17 2007
From: jcarlson at uci.edu (Josiah Carlson)
Date: Wed, 10 Jan 2007 12:27:17 -0800
Subject: [Python-3000] possible new packages (PEP 3108)
In-Reply-To: <bbaeab100701101132h2049697dj70fc89026532a878@mail.gmail.com>
References: <20070109212841.8D6A.JCARLSON@uci.edu>
	<bbaeab100701101132h2049697dj70fc89026532a878@mail.gmail.com>
Message-ID: <20070110121612.8D8F.JCARLSON@uci.edu>


"Brett Cannon" <brett at python.org> wrote:
> On 1/9/07, Josiah Carlson <jcarlson at uci.edu> wrote:
> > "Brett Cannon" <brett at python.org> wrote:
> > > * Internet (leaving out all questionable modules leads to a Web
> > >   grouping)
> >
> > I'm not a real big fan of the Internet package, but you forgot about
> > smtpd (an asyncore subclass that is an smtp server).
> 
> Or is that more of an email thing?

The rest of the email package is email clients and email storage.  It
fits (being about email), but it is also a server.  I don't know.  If we
*wanted* to go deep, it would be easy (at least in my opinion).

clients
    http
        *
    imap
        *
    pop
        *
servers
    http
        *
    smtp
        *
storage
    dbs
        *
    email
        *


> If you want to deprecate asynchat and move it to asyncore, go for it,
> you maintain it.  =)  Just will want to do that in 2.x probably.

I have some other fixes that should go in for 2.6 first, but with
urgings like that, I'll have to seriously consider it.


 - Josiah


From jcarlson at uci.edu  Wed Jan 10 21:28:51 2007
From: jcarlson at uci.edu (Josiah Carlson)
Date: Wed, 10 Jan 2007 12:28:51 -0800
Subject: [Python-3000] possible new packages (PEP 3108)
In-Reply-To: <082A8692-BFA8-4CDC-8467-8491D50877D4@python.org>
References: <bbaeab100701101132h2049697dj70fc89026532a878@mail.gmail.com>
	<082A8692-BFA8-4CDC-8467-8491D50877D4@python.org>
Message-ID: <20070110122829.8D92.JCARLSON@uci.edu>


> Two other quick things: I have an adaptation of smtpd.py that  
> provides LMTP support (RFC 2033) that might make it into Python 2.6.   
> Also, I'd really love it if the top level 'email' package name was  
> reserved for the current code.

Agreed.

 - Josiah


From goofyheadedpunk at gmail.com  Wed Jan 10 22:03:33 2007
From: goofyheadedpunk at gmail.com (Brian L. Troutwine)
Date: Wed, 10 Jan 2007 13:03:33 -0800
Subject: [Python-3000] 2to3 pattern syntax
Message-ID: <200701101303.33577.goofyheadedpunk@gmail.com>

As the refactoring tool for the 2 to 3 transition was mentioned some time ago 
on this list I hope that this is the correct list to post questions about the 
tool. I apologize if this isn't, of course.

My questions are entirely concerned with the pattern matching syntax. The 
problem is, it would seem, I don't understand a thing about them.

Is the following incorrect for matching pass statements? If so, why?

    pass_stmt< 'pass' >

If I wanted to append to each matched pass statement, say, the string "# This 
is a comment" how would the transformation have to be structured?

If I wanted to match a function structure what would the proper pattern be? 
How about classes?
-- 
Small is beautiful.
		-- Schumacher's Dictum

From jimjjewett at gmail.com  Wed Jan 10 22:12:56 2007
From: jimjjewett at gmail.com (Jim Jewett)
Date: Wed, 10 Jan 2007 16:12:56 -0500
Subject: [Python-3000] possible new packages (PEP 3108)
In-Reply-To: <3778330928605486501@unknownmsgid>
References: <bbaeab100701091427ob6cc8c2jd45feb9ee22b9889@mail.gmail.com>
	<284183686114447699@unknownmsgid>
	<bbaeab100701091934v4c99b32bn11b0be402218efca@mail.gmail.com>
	<-4021698297383678739@unknownmsgid>
	<fb6fbf560701100950n160aeaf3w9abe69dc6b1ff26c@mail.gmail.com>
	<3778330928605486501@unknownmsgid>
Message-ID: <fb6fbf560701101312h5fd102eam8d2752de556587ca@mail.gmail.com>

On 1/10/07, Bill Janssen <janssen at parc.com> wrote:
> Jim Jewett says:

> > (2)  (Maybe) moving modules closer together so that it will be easier
> > to see what  refactoring is needed.

> > Eliminating some additional modules afterwards would be good too, but
> > there may not be time, and step (2) is worthwhile on its own.

> Yes, it's just that I disagree with you about that.  Step (2) causes
> more trouble than it's worth without really doing a reorg.

How do you figure?

The advantage of the reorg is in learnability, teachability, and
discoverability.  It is easier to think through "these are my 12 sets
of choices" than "these are my 307 choices".

The disadvantage is just a new name.  If the module contents don't
change, then this is about the lowest-cost change available.

(1)  The update is straightforward, and can be automated.
(2)  The new names can be made available in 2.x as shadows for
source-level compatibility, so package maintainers can upgrade at
theirn own convenience.

-jJ

From guido at python.org  Wed Jan 10 22:39:22 2007
From: guido at python.org (Guido van Rossum)
Date: Wed, 10 Jan 2007 13:39:22 -0800
Subject: [Python-3000] 2to3 pattern syntax
In-Reply-To: <200701101303.33577.goofyheadedpunk@gmail.com>
References: <200701101303.33577.goofyheadedpunk@gmail.com>
Message-ID: <ca471dc20701101339g6df84510sd5c72a5e624d1e5e@mail.gmail.com>

On 1/10/07, Brian L. Troutwine <goofyheadedpunk at gmail.com> wrote:
> As the refactoring tool for the 2 to 3 transition was mentioned some time ago
> on this list I hope that this is the correct list to post questions about the
> tool. I apologize if this isn't, of course.
>
> My questions are entirely concerned with the pattern matching syntax. The
> problem is, it would seem, I don't understand a thing about them.
>
> Is the following incorrect for matching pass statements? If so, why?
>
>     pass_stmt< 'pass' >

The match should be just

    'pass'

or

    any<'pass'>

The reason is that any non-leaf node in the parse tree that has
exactly one child disappears and is replaced with its (only) child. So
the pass_stmt node disappears and all you're left with is a NAME node
whose contents is 'pass'.

The optimization is done to vastly reduce the parse tree size; the
grammar as specified has lots of redundant levels. See the print fixer
for a wrinkle caused by this tree optimization.

> If I wanted to append to each matched pass statement, say, the string "# This
> is a comment" how would the transformation have to be structured?

You'd have to the parent node and set the comment as the prefix on the
NEWLINE node. The pattern would be this.

    simple_stmt <'pass' NEWLINE>

> If I wanted to match a function structure what would the proper pattern be?

   funcdef <[(not 'def') any] 'def' NAME any ':' any>

> How about classes?

    classdef <'class' NAME ['(' any ')'] ':' suite

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From janssen at parc.com  Wed Jan 10 22:52:45 2007
From: janssen at parc.com (Bill Janssen)
Date: Wed, 10 Jan 2007 13:52:45 PST
Subject: [Python-3000] possible new packages (PEP 3108)
In-Reply-To: <082A8692-BFA8-4CDC-8467-8491D50877D4@python.org> 
References: <bbaeab100701091427ob6cc8c2jd45feb9ee22b9889@mail.gmail.com>
	<20070109212841.8D6A.JCARLSON@uci.edu>
	<bbaeab100701101132h2049697dj70fc89026532a878@mail.gmail.com>
	<082A8692-BFA8-4CDC-8467-8491D50877D4@python.org>
Message-ID: <07Jan10.135252pst."58648"@synergy1.parc.xerox.com>

> It depends.  I'm trying hard to stay out of this thread, but smtpd.py  
> is more of a server that speaks a protocol that happens to be used  
> sometimes <wink> for email.  I really don't know where best to put  
> it, but another alternative is to have a 'protocols' package perhaps  
> with client and server sub-modules where available.

I could see that.

Bill

From janssen at parc.com  Wed Jan 10 23:00:40 2007
From: janssen at parc.com (Bill Janssen)
Date: Wed, 10 Jan 2007 14:00:40 PST
Subject: [Python-3000] possible new packages (PEP 3108)
In-Reply-To: <bbaeab100701101143q175436c6wd215f8b626868a97@mail.gmail.com> 
References: <bbaeab100701091427ob6cc8c2jd45feb9ee22b9889@mail.gmail.com>
	<284183686114447699@unknownmsgid>
	<bbaeab100701091934v4c99b32bn11b0be402218efca@mail.gmail.com>
	<-4021698297383678739@unknownmsgid>
	<fb6fbf560701100950n160aeaf3w9abe69dc6b1ff26c@mail.gmail.com>
	<3334693148356440096@unknownmsgid>
	<bbaeab100701101143q175436c6wd215f8b626868a97@mail.gmail.com>
Message-ID: <07Jan10.140047pst."58648"@synergy1.parc.xerox.com>

> Otherwise I am removing the Open Issue and I will
> leave it up to someone else to write up a PEP and pitch it to the list
> and Guido.

That would be my recommendation.

Bill

From anthony at interlink.com.au  Thu Jan 11 02:48:10 2007
From: anthony at interlink.com.au (Anthony Baxter)
Date: Thu, 11 Jan 2007 12:48:10 +1100
Subject: [Python-3000] Warning for 2.6 and greater
In-Reply-To: <eo3evi$7dr$1@sea.gmane.org>
References: <200701081650.17353.anthony@interlink.com.au>
	<200701101524.06515.anthony@interlink.com.au>
	<eo3evi$7dr$1@sea.gmane.org>
Message-ID: <200701111248.10791.anthony@interlink.com.au>

On Thursday 11 January 2007 06:32, Georg Brandl wrote:
> I guess there are quite a few people who won't start moving to
> Python 3.0 with 2.6, or even when 3.1 is out, as long as their
> program works fine with the 2.x line. There's no need to punish
> them with extra overhead.

Checking a single C global int is hardly going to make a huge impact 
at all.


-- 
Anthony Baxter     <anthony at interlink.com.au>
It's never too late to have a happy childhood.

From raymond.hettinger at verizon.net  Wed Jan 10 19:13:39 2007
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Wed, 10 Jan 2007 10:13:39 -0800
Subject: [Python-3000] [Python-Dev]  Warning for 2.6 and greater
References: <200701081650.17353.anthony@interlink.com.au><9e804ac0701080353x6e84bd55q72848de26783635d@mail.gmail.com><200701091152.48048.anthony@interlink.com.au>
	<200701101524.06515.anthony@interlink.com.au>
Message-ID: <00ee01c734e3$0f1e4110$ea146b0a@RaymondLaptop1>

<"Anthony Baxter">
> Comments? What else should get warnings?

It is my strong preference that we not go down this path.  
Instead, the 2.6 vs 3.0 difference analysis should go in an 
external lint utility.

The Py2.x series may live-on for some time and should do so
as if Py3.x did not exist.  Burdening the 2.x code with loads
of warnings will only clutter the source code and make maintenance 
more difficult.  There may also be some performance impact.

We should resolve that Py2.6 remain as clean as possible
and that Py3.0 be kept in its own world.  Forging a new
blade does not have to entail dulling the trusty old blade.


Raymond




From ncoghlan at gmail.com  Thu Jan 11 15:37:46 2007
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 12 Jan 2007 00:37:46 +1000
Subject: [Python-3000] self-contained exceptions
In-Reply-To: <5.1.1.6.0.20070110124448.0299ac90@sparrow.telecommunity.com>
References: <20070109213607.8D6D.JCARLSON@uci.edu>	<ca471dc20701090941x17110b56y193110bad6c6fd53@mail.gmail.com>	<43aa6ff70701091817w7ef85d55l7ec4ed35640fb84e@mail.gmail.com>	<20070109213607.8D6D.JCARLSON@uci.edu>
	<5.1.1.6.0.20070110124448.0299ac90@sparrow.telecommunity.com>
Message-ID: <45A64BBA.10603@gmail.com>

Phillip J. Eby wrote:
> At 07:47 AM 1/10/2007 -0800, Guido van Rossum wrote:
>> BTW perhaps we should add the same semantics and syntax to 'with expr
>> as var'? I can't think of a reasonable use case
> [snip]
>> ... for using anything
>> more complex than a local variable, BTW.)
> 
> with multi(ctx1, ctx2, ...)  as var1, var2, ...:
> 
> (if multi is a function that returns a context manager wrapping the other 
> contexts and returning the tuple their __enter__ values).

...a utility which is also known as contextlib.nested().

For some strange reason, I suspect you're familiar with that particular 
context manager ;)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From jimjjewett at gmail.com  Thu Jan 11 17:12:02 2007
From: jimjjewett at gmail.com (Jim Jewett)
Date: Thu, 11 Jan 2007 11:12:02 -0500
Subject: [Python-3000] Code working in both 2.x and 3.X
Message-ID: <fb6fbf560701110812p71b12567y6baae509c9747681@mail.gmail.com>

Raymond Hettinger wrote:

> Also, I'm wondering if the desire for 2.6 warnings is based on the notion that
> it will be possible to get large tools to work under both Py2.x and Py3.x.

I had certainly assumed it would be possible.

In fact, I had assumed that the 2->3 translator would have a mode
which left the code running in both.

This might not be the cleanest possible code for either line (parens
in 2.x, some extra iter calls or missed shortcuts, etc), but it should
certainly exist.  If it doesn't, then people who do want to support
both lines will themselves have to work exclusively in 2.x and
"compile" to 3.x.

> With all the module renaming/packaging,

I had believed that 2.x would support both names, typically as

    >>> # make an deprecation warning, or log entry
    >>> from oldname import *

So code using the new names may not work in 2.5, but should work in
2.6.  If you want it to work in 2.3, you either use conditional
imports or include your own forwarding modules for older pythons.

> old-style classes disappearing,

This is only a problem if you rely on specfically old-style semantics.
 Most old-style classes that I see work fine as new-style; the author
just didn't want to put in the extra boilerplate (or wanted 1.5
compatibility, or just didn't bother to make a change).

> encoded text objects,

This one I'm less sure about, but it seems that programs which work
correctly with both str and unicode in 2.x should work with 3, and
that programs which (in practice) restrict themselves to ASCII will
continue to work.

> real division

You can already get this a __future__ import, and you can explicitly
request the old form (in both lines) by truncating the result.

> and whatnot; that notion may be a pipe-dream.

I hope not.

That said, I'm not so sure about C extension modules...

-jJ

From pje at telecommunity.com  Thu Jan 11 17:18:02 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 11 Jan 2007 11:18:02 -0500
Subject: [Python-3000] self-contained exceptions
In-Reply-To: <45A64BBA.10603@gmail.com>
References: <5.1.1.6.0.20070110124448.0299ac90@sparrow.telecommunity.com>
	<20070109213607.8D6D.JCARLSON@uci.edu>
	<ca471dc20701090941x17110b56y193110bad6c6fd53@mail.gmail.com>
	<43aa6ff70701091817w7ef85d55l7ec4ed35640fb84e@mail.gmail.com>
	<20070109213607.8D6D.JCARLSON@uci.edu>
	<5.1.1.6.0.20070110124448.0299ac90@sparrow.telecommunity.com>
Message-ID: <5.1.1.6.0.20070111111705.02ea6678@sparrow.telecommunity.com>

At 12:37 AM 1/12/2007 +1000, Nick Coghlan wrote:
>Phillip J. Eby wrote:
>>At 07:47 AM 1/10/2007 -0800, Guido van Rossum wrote:
>>>BTW perhaps we should add the same semantics and syntax to 'with expr
>>>as var'? I can't think of a reasonable use case
>>[snip]
>>>... for using anything
>>>more complex than a local variable, BTW.)
>>with multi(ctx1, ctx2, ...)  as var1, var2, ...:
>>(if multi is a function that returns a context manager wrapping the other 
>>contexts and returning the tuple their __enter__ values).
>
>...a utility which is also known as contextlib.nested().
>
>For some strange reason, I suspect you're familiar with that particular 
>context manager ;)

Not that I recall, no, otherwise I'd have used the name.  I thought there 
was such a thing in the PEP, but didn't know it had gotten in the stdlib 
(which of course is an even better reason to keep tuple assignment in the 
"with" statement).


From guido at python.org  Thu Jan 11 17:36:31 2007
From: guido at python.org (Guido van Rossum)
Date: Thu, 11 Jan 2007 08:36:31 -0800
Subject: [Python-3000] self-contained exceptions
In-Reply-To: <5.1.1.6.0.20070111111705.02ea6678@sparrow.telecommunity.com>
References: <ca471dc20701090941x17110b56y193110bad6c6fd53@mail.gmail.com>
	<43aa6ff70701091817w7ef85d55l7ec4ed35640fb84e@mail.gmail.com>
	<20070109213607.8D6D.JCARLSON@uci.edu>
	<5.1.1.6.0.20070110124448.0299ac90@sparrow.telecommunity.com>
	<45A64BBA.10603@gmail.com>
	<5.1.1.6.0.20070111111705.02ea6678@sparrow.telecommunity.com>
Message-ID: <ca471dc20701110836n2fe58d44g2a393a60eeb58fa7@mail.gmail.com>

Let me clarify. I have no desire to prevent tuple assignment in with
statements. But I do think that the tuples could be limited to plain
names and other tuples (with similar constraints), and those names
should all be zapped at the end of the clause, like for except. And we
don't even have to add an extra try/finally to do so, since with
already *has* a try/finally...

I'm also curious to hear what people think of similarly limiting the
syntax of the target in all for-loops -- but WITHOUT the zapping of
the variable at block end.

I think none of these changes would affect reasonably written code;
the use cases for using variables that are part of other objects are
too obscure to care.

--Guido

On 1/11/07, Phillip J. Eby <pje at telecommunity.com> wrote:
> At 12:37 AM 1/12/2007 +1000, Nick Coghlan wrote:
> >Phillip J. Eby wrote:
> >>At 07:47 AM 1/10/2007 -0800, Guido van Rossum wrote:
> >>>BTW perhaps we should add the same semantics and syntax to 'with expr
> >>>as var'? I can't think of a reasonable use case
> >>[snip]
> >>>... for using anything
> >>>more complex than a local variable, BTW.)
> >>with multi(ctx1, ctx2, ...)  as var1, var2, ...:
> >>(if multi is a function that returns a context manager wrapping the other
> >>contexts and returning the tuple their __enter__ values).
> >
> >...a utility which is also known as contextlib.nested().
> >
> >For some strange reason, I suspect you're familiar with that particular
> >context manager ;)
>
> Not that I recall, no, otherwise I'd have used the name.  I thought there
> was such a thing in the PEP, but didn't know it had gotten in the stdlib
> (which of course is an even better reason to keep tuple assignment in the
> "with" statement).
>
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From pje at telecommunity.com  Thu Jan 11 17:52:56 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 11 Jan 2007 11:52:56 -0500
Subject: [Python-3000] self-contained exceptions
In-Reply-To: <ca471dc20701110836n2fe58d44g2a393a60eeb58fa7@mail.gmail.co
 m>
References: <5.1.1.6.0.20070111111705.02ea6678@sparrow.telecommunity.com>
	<ca471dc20701090941x17110b56y193110bad6c6fd53@mail.gmail.com>
	<43aa6ff70701091817w7ef85d55l7ec4ed35640fb84e@mail.gmail.com>
	<20070109213607.8D6D.JCARLSON@uci.edu>
	<5.1.1.6.0.20070110124448.0299ac90@sparrow.telecommunity.com>
	<45A64BBA.10603@gmail.com>
	<5.1.1.6.0.20070111111705.02ea6678@sparrow.telecommunity.com>
Message-ID: <5.1.1.6.0.20070111114501.0426e338@sparrow.telecommunity.com>

At 08:36 AM 1/11/2007 -0800, Guido van Rossum wrote:
>Let me clarify. I have no desire to prevent tuple assignment in with
>statements. But I do think that the tuples could be limited to plain
>names and other tuples (with similar constraints), and those names
>should all be zapped at the end of the clause, like for except. And we
>don't even have to add an extra try/finally to do so, since with
>already *has* a try/finally...
>
>I'm also curious to hear what people think of similarly limiting the
>syntax of the target in all for-loops -- but WITHOUT the zapping of
>the variable at block end.
>
>I think none of these changes would affect reasonably written code;
>the use cases for using variables that are part of other objects are
>too obscure to care.

Yes, they're obscure.  But the symmetry with assignment is pleasing, 
nonetheless.  It gives me the sense of a "small" language, i.e., one 
without nine jillion special cases.

I've been having to do some Perl hacking this week, and I'm being quite 
thoroughly reminded of the joys of how few special cases Python has.  :)

By comparison, Perl is nothing but special cases, as I keep discovering 
that some symbol combination I thought would work, either doesn't, or does 
something entirely different depending on context.  I keep resorting to 
using "map" and a few other tricks to find some Python-like uniformity and 
composability.

Don't get me wrong, I'm not saying that limiting the assignment targets of 
"with" and "for" will turn Python into Perl.  I'm just saying that not 
having to create different mental categories for their assignment targets 
is pleasant.  In contrast, the "as" in "except" and "import" seems to 
already be a different mental category that occurs seamlessly, whereas 
"for" and "with" seem to bunch with "assignment" in my mind.

But of course all this could just be the peculiarities of my own 
mind.  We'll see what other people think.


From guido at python.org  Thu Jan 11 18:47:37 2007
From: guido at python.org (Guido van Rossum)
Date: Thu, 11 Jan 2007 09:47:37 -0800
Subject: [Python-3000] self-contained exceptions
In-Reply-To: <5.1.1.6.0.20070111114501.0426e338@sparrow.telecommunity.com>
References: <ca471dc20701090941x17110b56y193110bad6c6fd53@mail.gmail.com>
	<43aa6ff70701091817w7ef85d55l7ec4ed35640fb84e@mail.gmail.com>
	<20070109213607.8D6D.JCARLSON@uci.edu>
	<5.1.1.6.0.20070110124448.0299ac90@sparrow.telecommunity.com>
	<45A64BBA.10603@gmail.com>
	<5.1.1.6.0.20070111111705.02ea6678@sparrow.telecommunity.com>
	<5.1.1.6.0.20070111114501.0426e338@sparrow.telecommunity.com>
Message-ID: <ca471dc20701110947l75f38833lfd37b210859c5219@mail.gmail.com>

The case for changing 'for' is weakest, for sure. But 'with' is easy
to coerce into the same category as 'import' and 'except' since it
already uses 'as'...

On 1/11/07, Phillip J. Eby <pje at telecommunity.com> wrote:
> At 08:36 AM 1/11/2007 -0800, Guido van Rossum wrote:
> >Let me clarify. I have no desire to prevent tuple assignment in with
> >statements. But I do think that the tuples could be limited to plain
> >names and other tuples (with similar constraints), and those names
> >should all be zapped at the end of the clause, like for except. And we
> >don't even have to add an extra try/finally to do so, since with
> >already *has* a try/finally...
> >
> >I'm also curious to hear what people think of similarly limiting the
> >syntax of the target in all for-loops -- but WITHOUT the zapping of
> >the variable at block end.
> >
> >I think none of these changes would affect reasonably written code;
> >the use cases for using variables that are part of other objects are
> >too obscure to care.
>
> Yes, they're obscure.  But the symmetry with assignment is pleasing,
> nonetheless.  It gives me the sense of a "small" language, i.e., one
> without nine jillion special cases.
>
> I've been having to do some Perl hacking this week, and I'm being quite
> thoroughly reminded of the joys of how few special cases Python has.  :)
>
> By comparison, Perl is nothing but special cases, as I keep discovering
> that some symbol combination I thought would work, either doesn't, or does
> something entirely different depending on context.  I keep resorting to
> using "map" and a few other tricks to find some Python-like uniformity and
> composability.
>
> Don't get me wrong, I'm not saying that limiting the assignment targets of
> "with" and "for" will turn Python into Perl.  I'm just saying that not
> having to create different mental categories for their assignment targets
> is pleasant.  In contrast, the "as" in "except" and "import" seems to
> already be a different mental category that occurs seamlessly, whereas
> "for" and "with" seem to bunch with "assignment" in my mind.
>
> But of course all this could just be the peculiarities of my own
> mind.  We'll see what other people think.
>
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From tjreedy at udel.edu  Thu Jan 11 21:33:50 2007
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 11 Jan 2007 15:33:50 -0500
Subject: [Python-3000] self-contained exceptions
References: <ca471dc20701090941x17110b56y193110bad6c6fd53@mail.gmail.com><43aa6ff70701091817w7ef85d55l7ec4ed35640fb84e@mail.gmail.com><20070109213607.8D6D.JCARLSON@uci.edu><5.1.1.6.0.20070110124448.0299ac90@sparrow.telecommunity.com><45A64BBA.10603@gmail.com><5.1.1.6.0.20070111111705.02ea6678@sparrow.telecommunity.com><5.1.1.6.0.20070111114501.0426e338@sparrow.telecommunity.com>
	<ca471dc20701110947l75f38833lfd37b210859c5219@mail.gmail.com>
Message-ID: <eo66vb$gbk$1@sea.gmane.org>


"Guido van Rossum" <guido at python.org> wrote in message 
news:ca471dc20701110947l75f38833lfd37b210859c5219 at mail.gmail.com...
| The case for changing 'for' is weakest, for sure.

As I understand the proposal, the rationale for restricting the exception 
target is the autodeletion, which is obviously bug-prone if applied to 
pre-existing structures.  But this does not apply here.  Indeed, if you 
restrict the for-loop target the same as the exception target, you will 
increase people's expection that it is block-scope limited and probably 
stimulate even more questions/proposals about the 'inconsistency'.

So I agree with Phillip on for loop targets and leave them consistent with 
explicit assignment targets, which they will otherwise continue to act 
like.

| But 'with' is easy
| to coerce into the same category as 'import' and 'except' since it
| already uses 'as'...

Yes, I already see it that way.  It would be nice if the rule for 'as' 
targets were as consistent as possible (ie, at least the same for except 
and with)

Terry Jan Reedy




From pje at telecommunity.com  Thu Jan 11 22:05:57 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 11 Jan 2007 16:05:57 -0500
Subject: [Python-3000] self-contained exceptions
In-Reply-To: <eo66vb$gbk$1@sea.gmane.org>
References: <ca471dc20701090941x17110b56y193110bad6c6fd53@mail.gmail.com>
	<43aa6ff70701091817w7ef85d55l7ec4ed35640fb84e@mail.gmail.com>
	<20070109213607.8D6D.JCARLSON@uci.edu>
	<5.1.1.6.0.20070110124448.0299ac90@sparrow.telecommunity.com>
	<45A64BBA.10603@gmail.com>
	<5.1.1.6.0.20070111111705.02ea6678@sparrow.telecommunity.com>
	<5.1.1.6.0.20070111114501.0426e338@sparrow.telecommunity.com>
	<ca471dc20701110947l75f38833lfd37b210859c5219@mail.gmail.com>
Message-ID: <5.1.1.6.0.20070111160406.02f1cb08@sparrow.telecommunity.com>

At 03:33 PM 1/11/2007 -0500, Terry Reedy wrote:
>Yes, I already see it that way.  It would be nice if the rule for 'as'
>targets were as consistent as possible (ie, at least the same for except
>and with)

No can do - with needs tuples, except isn't getting them.

If we did allow tuples in both cases, the "except" ones wouldn't be cleared 
at the end of the block, but the "with" ones would...  unless we decided to 
make them all go away.  There might be sane reasons for doing that with 
"with", but I'm not sure those reasons apply to "except".

(sigh) Good language design is *hard*.


From collinw at gmail.com  Thu Jan 11 22:42:29 2007
From: collinw at gmail.com (Collin Winter)
Date: Thu, 11 Jan 2007 15:42:29 -0600
Subject: [Python-3000] self-contained exceptions
In-Reply-To: <5.1.1.6.0.20070111160406.02f1cb08@sparrow.telecommunity.com>
References: <ca471dc20701090941x17110b56y193110bad6c6fd53@mail.gmail.com>
	<43aa6ff70701091817w7ef85d55l7ec4ed35640fb84e@mail.gmail.com>
	<20070109213607.8D6D.JCARLSON@uci.edu>
	<5.1.1.6.0.20070110124448.0299ac90@sparrow.telecommunity.com>
	<45A64BBA.10603@gmail.com>
	<5.1.1.6.0.20070111111705.02ea6678@sparrow.telecommunity.com>
	<5.1.1.6.0.20070111114501.0426e338@sparrow.telecommunity.com>
	<ca471dc20701110947l75f38833lfd37b210859c5219@mail.gmail.com>
	<eo66vb$gbk$1@sea.gmane.org>
	<5.1.1.6.0.20070111160406.02f1cb08@sparrow.telecommunity.com>
Message-ID: <43aa6ff70701111342t60fab865k8752c90144973e20@mail.gmail.com>

On 1/11/07, Phillip J. Eby <pje at telecommunity.com> wrote:
> At 03:33 PM 1/11/2007 -0500, Terry Reedy wrote:
> >Yes, I already see it that way.  It would be nice if the rule for 'as'
> >targets were as consistent as possible (ie, at least the same for except
> >and with)
>
> No can do - with needs tuples, except isn't getting them.
>
> If we did allow tuples in both cases, the "except" ones wouldn't be cleared
> at the end of the block, but the "with" ones would...  unless we decided to
> make them all go away.  There might be sane reasons for doing that with
> "with", but I'm not sure those reasons apply to "except".

We can allow tuples for 'except' at the grammar level, same as 'with';
it's just that anyone trying it would get the usual "TypeError: unpack
non-sequence" since the exceptions-as-sequence behaviour is going
away.

Collin Winter

From rrr at ronadam.com  Thu Jan 11 22:59:49 2007
From: rrr at ronadam.com (Ron Adam)
Date: Thu, 11 Jan 2007 15:59:49 -0600
Subject: [Python-3000] Code working in both 2.x and 3.X
In-Reply-To: <fb6fbf560701110812p71b12567y6baae509c9747681@mail.gmail.com>
References: <fb6fbf560701110812p71b12567y6baae509c9747681@mail.gmail.com>
Message-ID: <eo6c90$5ea$1@sea.gmane.org>

Jim Jewett wrote:
> Raymond Hettinger wrote:
> 
>> Also, I'm wondering if the desire for 2.6 warnings is based on the notion that
>> it will be possible to get large tools to work under both Py2.x and Py3.x.
> 
> I had certainly assumed it would be possible.
> 
> In fact, I had assumed that the 2->3 translator would have a mode
> which left the code running in both.
> 
> This might not be the cleanest possible code for either line (parens
> in 2.x, some extra iter calls or missed shortcuts, etc), but it should
> certainly exist.  If it doesn't, then people who do want to support
> both lines will themselves have to work exclusively in 2.x and
> "compile" to 3.x.
> 
>> With all the module renaming/packaging,
[etc... clipped to shorten]

There seems to be a fair amount of anxiety starting to appear about the 
difficulty of trying to support both 2.x and 3.x at the same time.  It's 
understandable, but I think it may be the earlier than expected 3.0 release date 
that is adding to that.

Would it be reasonable to define X.0.X versions as official transition versions 
which need not have both back word compatibility, or overly strict future 
compatibility?

I thinking that the 3.0.X version be considered a try it out (alpha) release to 
generate plenty of feed back, and the 3.1.X version be the first version meant 
for actual development use.

    3.0.X <=> 3.alpha.X
    3.1.X <=> 3.first.X

So the release order may be 3.0.0, 2.6.0, 3.1.0. ...

(With minor bug fix releases in between, of course.)

My thinking is developers won't need to support the 3.0.X version *ever*.  And 
even 3.1.X need not be "too strongly" constrained to be back word compatible 
with version 3.0.X, as it may still have quite a few changes.  Version 3.1.0 
would then be the first version that future, 3.(1+n), versions will need to 
maintain back words compatibility with.

This gives a lot more time for developers to try it, give feed back, and work 
out how to do things like move stuff from 2.X to 3.X, etc...  It also may give 
more freedom to make changes in version 3.0 with less anxiety to third party 
developers.

Version 3.0  ->  a (relatively) clean start with big changes.

Version 2.6  ->  may take into account things in version 3.0.

Version 3.1  ->  builds on 3.0 and may take into account version 2.6


Isn't this the general (informal) context already being considered?


Just a few thoughts,
   Ron



From pje at telecommunity.com  Thu Jan 11 23:07:39 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 11 Jan 2007 17:07:39 -0500
Subject: [Python-3000] self-contained exceptions
In-Reply-To: <43aa6ff70701111342t60fab865k8752c90144973e20@mail.gmail.co
 m>
References: <5.1.1.6.0.20070111160406.02f1cb08@sparrow.telecommunity.com>
	<ca471dc20701090941x17110b56y193110bad6c6fd53@mail.gmail.com>
	<43aa6ff70701091817w7ef85d55l7ec4ed35640fb84e@mail.gmail.com>
	<20070109213607.8D6D.JCARLSON@uci.edu>
	<5.1.1.6.0.20070110124448.0299ac90@sparrow.telecommunity.com>
	<45A64BBA.10603@gmail.com>
	<5.1.1.6.0.20070111111705.02ea6678@sparrow.telecommunity.com>
	<5.1.1.6.0.20070111114501.0426e338@sparrow.telecommunity.com>
	<ca471dc20701110947l75f38833lfd37b210859c5219@mail.gmail.com>
	<eo66vb$gbk$1@sea.gmane.org>
	<5.1.1.6.0.20070111160406.02f1cb08@sparrow.telecommunity.com>
Message-ID: <5.1.1.6.0.20070111170435.02f72850@sparrow.telecommunity.com>

At 03:42 PM 1/11/2007 -0600, Collin Winter wrote:
>On 1/11/07, Phillip J. Eby <pje at telecommunity.com> wrote:
>>At 03:33 PM 1/11/2007 -0500, Terry Reedy wrote:
>> >Yes, I already see it that way.  It would be nice if the rule for 'as'
>> >targets were as consistent as possible (ie, at least the same for except
>> >and with)
>>
>>No can do - with needs tuples, except isn't getting them.
>>
>>If we did allow tuples in both cases, the "except" ones wouldn't be cleared
>>at the end of the block, but the "with" ones would...  unless we decided to
>>make them all go away.  There might be sane reasons for doing that with
>>"with", but I'm not sure those reasons apply to "except".
>
>We can allow tuples for 'except' at the grammar level, same as 'with';
>it's just that anyone trying it would get the usual "TypeError: unpack
>non-sequence" since the exceptions-as-sequence behaviour is going
>away.

Well, people can easily put it back in if they want; all that's required is 
an __iter__ method, after all.

I'm talking about the consequences of allowing it at the grammar level, 
such as the need to explain why "except foo as (a,b)" does different things 
to the names than "except foo as ab".

Unless, of course, we're going to say that "except foo as (a,b)" makes 'a' 
and 'b' disappear outside the block.


From guido at python.org  Thu Jan 11 23:52:35 2007
From: guido at python.org (Guido van Rossum)
Date: Thu, 11 Jan 2007 14:52:35 -0800
Subject: [Python-3000] self-contained exceptions
In-Reply-To: <5.1.1.6.0.20070111170435.02f72850@sparrow.telecommunity.com>
References: <ca471dc20701090941x17110b56y193110bad6c6fd53@mail.gmail.com>
	<20070109213607.8D6D.JCARLSON@uci.edu>
	<5.1.1.6.0.20070110124448.0299ac90@sparrow.telecommunity.com>
	<45A64BBA.10603@gmail.com>
	<5.1.1.6.0.20070111111705.02ea6678@sparrow.telecommunity.com>
	<5.1.1.6.0.20070111114501.0426e338@sparrow.telecommunity.com>
	<ca471dc20701110947l75f38833lfd37b210859c5219@mail.gmail.com>
	<eo66vb$gbk$1@sea.gmane.org>
	<5.1.1.6.0.20070111160406.02f1cb08@sparrow.telecommunity.com>
	<5.1.1.6.0.20070111170435.02f72850@sparrow.telecommunity.com>
Message-ID: <ca471dc20701111452v6a402d69lea767944448355a5@mail.gmail.com>

On 1/11/07, Phillip J. Eby <pje at telecommunity.com> wrote:
> At 03:42 PM 1/11/2007 -0600, Collin Winter wrote:
> >On 1/11/07, Phillip J. Eby <pje at telecommunity.com> wrote:
> >>At 03:33 PM 1/11/2007 -0500, Terry Reedy wrote:
> >> >Yes, I already see it that way.  It would be nice if the rule for 'as'
> >> >targets were as consistent as possible (ie, at least the same for except
> >> >and with)
> >>
> >>No can do - with needs tuples, except isn't getting them.
> >>
> >>If we did allow tuples in both cases, the "except" ones wouldn't be cleared
> >>at the end of the block, but the "with" ones would...  unless we decided to
> >>make them all go away.  There might be sane reasons for doing that with
> >>"with", but I'm not sure those reasons apply to "except".
> >
> >We can allow tuples for 'except' at the grammar level, same as 'with';
> >it's just that anyone trying it would get the usual "TypeError: unpack
> >non-sequence" since the exceptions-as-sequence behaviour is going
> >away.
>
> Well, people can easily put it back in if they want; all that's required is
> an __iter__ method, after all.

Ow, you're right. But let's not encourage that.

> I'm talking about the consequences of allowing it at the grammar level,
> such as the need to explain why "except foo as (a,b)" does different things
> to the names than "except foo as ab".
>
> Unless, of course, we're going to say that "except foo as (a,b)" makes 'a'
> and 'b' disappear outside the block.

I like what we have at the grammar level now (in the p3yk branch):
only a single name is allowed. If you want to unpack it into something
else, you'll have to do the tuple-unpack inside the except clause.
Makes sense to me...

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Thu Jan 11 23:54:09 2007
From: guido at python.org (Guido van Rossum)
Date: Thu, 11 Jan 2007 14:54:09 -0800
Subject: [Python-3000] Code working in both 2.x and 3.X
In-Reply-To: <eo6c90$5ea$1@sea.gmane.org>
References: <fb6fbf560701110812p71b12567y6baae509c9747681@mail.gmail.com>
	<eo6c90$5ea$1@sea.gmane.org>
Message-ID: <ca471dc20701111454m2954a817tb68adafed161a631@mail.gmail.com>

On 1/11/07, Ron Adam <rrr at ronadam.com> wrote:
> Jim Jewett wrote:
> > Raymond Hettinger wrote:
> >
> >> Also, I'm wondering if the desire for 2.6 warnings is based on the notion that
> >> it will be possible to get large tools to work under both Py2.x and Py3.x.
> >
> > I had certainly assumed it would be possible.
> >
> > In fact, I had assumed that the 2->3 translator would have a mode
> > which left the code running in both.
> >
> > This might not be the cleanest possible code for either line (parens
> > in 2.x, some extra iter calls or missed shortcuts, etc), but it should
> > certainly exist.  If it doesn't, then people who do want to support
> > both lines will themselves have to work exclusively in 2.x and
> > "compile" to 3.x.
> >
> >> With all the module renaming/packaging,
> [etc... clipped to shorten]
>
> There seems to be a fair amount of anxiety starting to appear about the
> difficulty of trying to support both 2.x and 3.x at the same time.  It's
> understandable, but I think it may be the earlier than expected 3.0 release date
> that is adding to that.
>
> Would it be reasonable to define X.0.X versions as official transition versions
> which need not have both back word compatibility, or overly strict future
> compatibility?
>
> I thinking that the 3.0.X version be considered a try it out (alpha) release to
> generate plenty of feed back, and the 3.1.X version be the first version meant
> for actual development use.
>
>     3.0.X <=> 3.alpha.X
>     3.1.X <=> 3.first.X
>
> So the release order may be 3.0.0, 2.6.0, 3.1.0. ...
>
> (With minor bug fix releases in between, of course.)
>
> My thinking is developers won't need to support the 3.0.X version *ever*.  And
> even 3.1.X need not be "too strongly" constrained to be back word compatible
> with version 3.0.X, as it may still have quite a few changes.  Version 3.1.0
> would then be the first version that future, 3.(1+n), versions will need to
> maintain back words compatibility with.
>
> This gives a lot more time for developers to try it, give feed back, and work
> out how to do things like move stuff from 2.X to 3.X, etc...  It also may give
> more freedom to make changes in version 3.0 with less anxiety to third party
> developers.
>
> Version 3.0  ->  a (relatively) clean start with big changes.
>
> Version 2.6  ->  may take into account things in version 3.0.
>
> Version 3.1  ->  builds on 3.0 and may take into account version 2.6

This last line is news to me, and I'm not in favor of it. My plans for
3.1 are different -- to take into account user feedback on 3.0.

> Isn't this the general (informal) context already being considered?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From rrr at ronadam.com  Fri Jan 12 01:00:56 2007
From: rrr at ronadam.com (Ron Adam)
Date: Thu, 11 Jan 2007 18:00:56 -0600
Subject: [Python-3000] Code working in both 2.x and 3.X
In-Reply-To: <ca471dc20701111454m2954a817tb68adafed161a631@mail.gmail.com>
References: <fb6fbf560701110812p71b12567y6baae509c9747681@mail.gmail.com>	<eo6c90$5ea$1@sea.gmane.org>
	<ca471dc20701111454m2954a817tb68adafed161a631@mail.gmail.com>
Message-ID: <eo6jc2$u69$1@sea.gmane.org>

Guido van Rossum wrote:
> On 1/11/07, Ron Adam <rrr at ronadam.com> wrote:


>> I thinking that the 3.0.X version be considered a try it out (alpha) release to
>> generate plenty of feed back, and the 3.1.X version be the first version meant
>> for actual development use.

I did list "plenty of feedback" as a purpose of 3.0. ie... "builds on 3.0." 
included that in my mind.


>> Version 3.0  ->  a (relatively) clean start with big changes.
>>
>> Version 2.6  ->  may take into account things in version 3.0.
>>
>> Version 3.1  ->  builds on 3.0 and may take into account version 2.6

> This last line is news to me, and I'm not in favor of it. My plans for
> 3.1 are different -- to take into account user feedback on 3.0.

Should have been a "if needed/where desired" on that last line.

Just thinking that some things in 2.6 may influence how some things in 3.1 are 
decided on.  That might refer to issues of conversion programs and or utilities, 
etc..

I think maybe the main idea was each version may take into account an earlier 
release, but not be overly concerned with a not yet released version.  The order 
of the releases will allow for things to be done without trying to predict the 
future.

(But in actuality there is no clear lines of separation, is there?)


On python-dev...

 > On 1/11/07, James Y Knight <foom at fuhm.net> wrote:
 >> > If the goal is really to have Py 3.0 be released later this year,
 >
 > 3.0 alpha is scheduled for this year.  3.0 final is not scheduled till
 > next year, and of course another level of tweaks will have to be made
 > after it's been in the Real World for a few months, so it won't fully
 > stabilize until 3.1 or 3.2.  That gives 1 1/2 - 2 years if you want to
 > target "production-level" 3.1.  Also, major parts of the 3.0 core have
 > not been written yet, so it remains to be seen how realistic the
 > schedule is (or how radical the changes will be if the schedule is
 > kept).


This works out to basically what I was describing in any case I think.

Cheers,
    Ron













From aahz at pythoncraft.com  Fri Jan 12 01:25:15 2007
From: aahz at pythoncraft.com (Aahz)
Date: Thu, 11 Jan 2007 16:25:15 -0800
Subject: [Python-3000] PEP 3108 and modules to be removed (current list)
In-Reply-To: <bbaeab100701091508nbab73a5t4ab19b96432914a1@mail.gmail.com>
References: <bbaeab100701051214h54c0e1f9w73c2e059fafbb04d@mail.gmail.com>
	<43aa6ff70701091455w616f6503s1ff64a729aab821a@mail.gmail.com>
	<bbaeab100701091508nbab73a5t4ab19b96432914a1@mail.gmail.com>
Message-ID: <20070112002515.GA1576@panix.com>

On Tue, Jan 09, 2007, Brett Cannon wrote:
> On 1/9/07, Collin Winter <collinw at gmail.com> wrote:
>> On 1/5/07, Brett Cannon <brett at python.org> wrote:
>>> Obsolete
>>> --------
>>>
>>> Becoming obsolete signifies that either another module in the stdlib
>>> or a widely distributed third-party library provides a better solution
>>> for what the module is meant for.
>>
>> How about the sets module? The set and frozenset types would seem to
>> have made a separate module obsolete.
>>
>> I've just refreshed SF patch #1500611, which removes all usages of the
>> sets module from the stdlib and test suite.
> 
> Well, I have no issue removing it, but I know some people really like
> having the Python version of the built-ins around (e.g., User*).  Plus
> I don't know if PyPy or IronPython use it for their set
> implementation.
> 
> What do other people think?

I'm generally a fan of providing Python versions of C modules.  It makes
Python more accesible to someone ramping up.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"Plus ca change, plus c'est la meme chose."

From larry at hastings.org  Fri Jan 12 05:34:56 2007
From: larry at hastings.org (Larry Hastings)
Date: Thu, 11 Jan 2007 20:34:56 -0800
Subject: [Python-3000] Lazy strings (was Re: Py3k release schedule
	worries)
In-Reply-To: <ca471dc20612301832s4aa0f1c4rac0709e5d9800a55@mail.gmail.com>
References: <20061223155403.GA13959@panix.com> <4596F51F.3010701@hastings.org>
	<ca471dc20612301832s4aa0f1c4rac0709e5d9800a55@mail.gmail.com>
Message-ID: <45A70FF0.1010300@hastings.org>


I've just posted my first pass of the full "lazy strings" patch, with 
both "lazy concatenation" and "lazy slices".  It's available on 
Sourceforge here:
    
https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1629305&group_id=5470
It applies cleanly (with "patch -p1") against the current revision of 
the Py3k branch, #53392.  The resulting tree builds as cleanly as an 
unpatched Py3k, on both Win32 and Linux, and the output from the 
regression tests is also unchanged.

As discussed on that page, the current version of the patch could cause 
crashes in low-memory conditions.  I welcome suggestions on how best to 
resolve this problem.  Apart from that fly in the ointment I'm pretty 
happy with how it all turned out.

In case it's of interest, I'm using a local revision control system to 
track my changes.  It makes working with a read-only Subversion 
repository much easier; in particular it makes generating patches and 
updating to new revisions of Python a snap.  I blogged about it here:
    http://www.momentaryfascinations.com/
And that's all I'm gonna say about that.

I look forward to your comments,


/larry/

From ncoghlan at gmail.com  Fri Jan 12 10:43:13 2007
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 12 Jan 2007 19:43:13 +1000
Subject: [Python-3000] self-contained exceptions
In-Reply-To: <5.1.1.6.0.20070111111705.02ea6678@sparrow.telecommunity.com>
References: <5.1.1.6.0.20070110124448.0299ac90@sparrow.telecommunity.com>
	<20070109213607.8D6D.JCARLSON@uci.edu>
	<ca471dc20701090941x17110b56y193110bad6c6fd53@mail.gmail.com>
	<43aa6ff70701091817w7ef85d55l7ec4ed35640fb84e@mail.gmail.com>
	<20070109213607.8D6D.JCARLSON@uci.edu>
	<5.1.1.6.0.20070110124448.0299ac90@sparrow.telecommunity.com>
	<5.1.1.6.0.20070111111705.02ea6678@sparrow.telecommunity.com>
Message-ID: <45A75831.6090800@gmail.com>

Phillip J. Eby wrote:
> At 12:37 AM 1/12/2007 +1000, Nick Coghlan wrote:
>> ...a utility which is also known as contextlib.nested().
>>
>> For some strange reason, I suspect you're familiar with that 
>> particular context manager ;)
> 
> Not that I recall, no, otherwise I'd have used the name.  I thought 
> there was such a thing in the PEP, but didn't know it had gotten in the 
> stdlib (which of course is an even better reason to keep tuple 
> assignment in the "with" statement).

I thought you added that module, but svn tells me it was actually Guido. 
   It appears I got mixed up between the initial checkins and your later 
edits to improve contextlib.contextmanager. Sorry for the noise :)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From guido at python.org  Fri Jan 12 16:16:32 2007
From: guido at python.org (Guido van Rossum)
Date: Fri, 12 Jan 2007 07:16:32 -0800
Subject: [Python-3000] Lazy strings (was Re: Py3k release schedule
	worries)
In-Reply-To: <45A70FF0.1010300@hastings.org>
References: <20061223155403.GA13959@panix.com> <4596F51F.3010701@hastings.org>
	<ca471dc20612301832s4aa0f1c4rac0709e5d9800a55@mail.gmail.com>
	<45A70FF0.1010300@hastings.org>
Message-ID: <ca471dc20701120716s7166f458le5eb632b1bf8345a@mail.gmail.com>

On 1/11/07, Larry Hastings <larry at hastings.org> wrote:
>
> I've just posted my first pass of the full "lazy strings" patch, with
> both "lazy concatenation" and "lazy slices".  It's available on
> Sourceforge here:
>
> https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1629305&group_id=5470

I'll try to look at it ASAP!

> It applies cleanly (with "patch -p1") against the current revision of
> the Py3k branch, #53392.  The resulting tree builds as cleanly as an
> unpatched Py3k, on both Win32 and Linux, and the output from the
> regression tests is also unchanged.
>
> As discussed on that page, the current version of the patch could cause
> crashes in low-memory conditions.  I welcome suggestions on how best to
> resolve this problem.  Apart from that fly in the ointment I'm pretty
> happy with how it all turned out.

What kind of crashes? The right thing to do is to raise MemoryError.
Is there anything besides sheer will power that prevents that?

> In case it's of interest, I'm using a local revision control system to
> track my changes.  It makes working with a read-only Subversion
> repository much easier; in particular it makes generating patches and
> updating to new revisions of Python a snap.  I blogged about it here:
>     http://www.momentaryfascinations.com/
> And that's all I'm gonna say about that.

Yes, I've heard good things about Mercurial.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From jcarlson at uci.edu  Fri Jan 12 18:29:55 2007
From: jcarlson at uci.edu (Josiah Carlson)
Date: Fri, 12 Jan 2007 09:29:55 -0800
Subject: [Python-3000] Lazy strings (was Re: Py3k release schedule
	worries)
In-Reply-To: <ca471dc20701120716s7166f458le5eb632b1bf8345a@mail.gmail.com>
References: <45A70FF0.1010300@hastings.org>
	<ca471dc20701120716s7166f458le5eb632b1bf8345a@mail.gmail.com>
Message-ID: <20070112091513.8DB0.JCARLSON@uci.edu>


"Guido van Rossum" <guido at python.org> wrote:
> On 1/11/07, Larry Hastings <larry at hastings.org> wrote:
> > It applies cleanly (with "patch -p1") against the current revision of
> > the Py3k branch, #53392.  The resulting tree builds as cleanly as an
> > unpatched Py3k, on both Win32 and Linux, and the output from the
> > regression tests is also unchanged.
> >
> > As discussed on that page, the current version of the patch could cause
> > crashes in low-memory conditions.  I welcome suggestions on how best to
> > resolve this problem.  Apart from that fly in the ointment I'm pretty
> > happy with how it all turned out.
> 
> What kind of crashes? The right thing to do is to raise MemoryError.
> Is there anything besides sheer will power that prevents that?

It may require changing around 400 uses of the PyUnicode_AS_UNICODE
macro in the trunk to allow the return of a NULL (which the current
semantics don't allow).  Any 3rd party code using the macro would also
need to be changed.

 - Josiah


From larry at hastings.org  Fri Jan 12 18:48:25 2007
From: larry at hastings.org (Larry Hastings)
Date: Fri, 12 Jan 2007 09:48:25 -0800
Subject: [Python-3000] Lazy strings (was Re: Py3k release schedule
	worries)
In-Reply-To: <ca471dc20701120716s7166f458le5eb632b1bf8345a@mail.gmail.com>
References: <20061223155403.GA13959@panix.com> <4596F51F.3010701@hastings.org>	
	<ca471dc20612301832s4aa0f1c4rac0709e5d9800a55@mail.gmail.com>	
	<45A70FF0.1010300@hastings.org>
	<ca471dc20701120716s7166f458le5eb632b1bf8345a@mail.gmail.com>
Message-ID: <45A7C9E9.2050308@hastings.org>

Guido van Rossum wrote:
>> As discussed on that page, the current version of the patch could cause
>> crashes in low-memory conditions.  I welcome suggestions on how best to
>> resolve this problem.  Apart from that fly in the ointment I'm pretty
>> happy with how it all turned out.
> What kind of crashes? The right thing to do is to raise MemoryError.
> Is there anything besides sheer will power that prevents that?
Nothing *has* prevented that; the relevant code already calls 
PyErr_NoMemory().  The problem is that *its* callers currently won't 
notice, and continue on their merry way.

My patch adds a new wrinkle to the API: PyUnicode_AS_UNICODE() can now 
fail.  And currently when it fails it returns NULL.  (Why could it 
fail?  Under the covers, PyUnicode_AS_UNICODE() may attempt to allocate 
memory.)

Without the patch PyUnicode_AS_UNICODE() always works.  Since no caller 
ever expects it to fail, code looks like this:

    static
    int fixupper(PyUnicodeObject *self)
    {
        Py_ssize_t len = self->length;
        Py_UNICODE *s = PyUnicode_AS_UNICODE(self);
        int status = 0;

        while (len-- > 0) {
        register Py_UNICODE ch;

        ch = Py_UNICODE_TOUPPER(*s);
        ...

And there you are; when s is NULL, Python crashes.

In the patch comments I proposed four possible solutions for this 
problem, listed in order of least-likely to most-likely.  I just came up 
with a fifth one, and I'll include it here.

   1. Redefine the API such that PyUnicode_AS_UNICODE() is allowed to
      return NULL, and fix every place in the Python source tree that
      calls it to check for a NULL return.  Document this with strong
      language for external C module authors.
   2. Pre-allocate the str buffer used to render the lazy string
      objects.  Update this buffer whenever the size of the string
      changes.  That moves the failure to a better place for error
      reporting; once again PyUnicode_AS_UNICODE() can never fail.  But
      this approach also negates a healthy chunk of what made the patch
      faster.
   3. Change the length to 0 and return a constant empty string. 
      Suggest that users of the Unicode API ask for the pointer *first*
      and the length *second*.
   4. Change the length to 0 and return a previously-allocated buffer of
      some hopefully-big-enough-size (4096 bytes? 8192 bytes?), such
      that even if the caller iterates over the buffer, odds are good
      they'll stop before they hit the end.  Again, suggest that users
      of the Unicode API ask for the pointer *first* and the length
      *second*.
   5. The patch is not accepted.

(You see what an optimist I am.)

I'm open to suggestions (and patches!) of other approaches to solve this 
problem.

Cheers,


/larry/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20070112/56f21a03/attachment.html 

From guido at python.org  Fri Jan 12 19:25:54 2007
From: guido at python.org (Guido van Rossum)
Date: Fri, 12 Jan 2007 10:25:54 -0800
Subject: [Python-3000] Lazy strings (was Re: Py3k release schedule
	worries)
In-Reply-To: <45A7C9E9.2050308@hastings.org>
References: <20061223155403.GA13959@panix.com> <4596F51F.3010701@hastings.org>
	<ca471dc20612301832s4aa0f1c4rac0709e5d9800a55@mail.gmail.com>
	<45A70FF0.1010300@hastings.org>
	<ca471dc20701120716s7166f458le5eb632b1bf8345a@mail.gmail.com>
	<45A7C9E9.2050308@hastings.org>
Message-ID: <ca471dc20701121025s65e04c62v6d76a58afcf2fe60@mail.gmail.com>

[Larry Hastings]
> As discussed on that page, the current version of the patch could cause
>  crashes in low-memory conditions.  I welcome suggestions on how best to
>  resolve this problem.  Apart from that fly in the ointment I'm pretty
>  happy with how it all turned out.

[Guido]
>  What kind of crashes? The right thing to do is to raise MemoryError.
>  Is there anything besides sheer will power that prevents that? Nothing
> *has* prevented that; the relevant code already calls PyErr_NoMemory().  The
> problem is that *its* callers currently won't notice, and continue on their
> merry way.

[Larry]
>  My patch adds a new wrinkle to the API: PyUnicode_AS_UNICODE() can now
> fail.  And currently when it fails it returns NULL.  (Why could it fail?
> Under the covers, PyUnicode_AS_UNICODE() may attempt to allocate memory.)
>
>  Without the patch PyUnicode_AS_UNICODE() always works.  Since no caller
> ever expects it to fail, code looks like this:
>
> static
>  int fixupper(PyUnicodeObject *self)
>  {
>      Py_ssize_t len = self->length;
>      Py_UNICODE *s = PyUnicode_AS_UNICODE(self);
>      int status = 0;
>
>      while (len-- > 0) {
>      register Py_UNICODE ch;
>
>      ch = Py_UNICODE_TOUPPER(*s);
>      ...
>  And there you are; when s is NULL, Python crashes.

Which is unacceptable. We might as well not have MemoryError and just
say "if you run out of memory, behavior of any program is undefined".
And I'll never go for that; it would take away a major advantage of
using Python. This would also directly affect security.

>  In the patch comments I proposed four possible solutions for this problem,
> listed in order of least-likely to most-likely.  I just came up with a fifth
> one, and I'll include it here.

Thanks -- I haven't had the time to look at the patch yet (but I will).

> 1. Redefine the API such that PyUnicode_AS_UNICODE() is allowed to return NULL,
> and fix every place in the Python source tree that calls it to check for a
> NULL return.  Document this with strong language for external C module
> authors.
> 2. Pre-allocate the str buffer used to render the lazy string objects.  Update
> this buffer whenever the size of the string changes.  That moves the failure
> to a better place for error reporting; once again PyUnicode_AS_UNICODE() can
> never fail.  But this approach also negates a healthy chunk of what made the
> patch faster.
> 3. Change the length to 0 and return a constant empty string.  Suggest that
> users of the Unicode API ask for the pointer *first* and the length
> *second*.
> 4. Change the length to 0 and return a previously-allocated buffer of some
> hopefully-big-enough-size (4096 bytes? 8192 bytes?), such that even if the
> caller iterates over the buffer, odds are good they'll stop before they hit
> the end.  Again, suggest that users of the Unicode API ask for the pointer
> *first* and the length *second*.
> 5. The patch is not accepted. (You see what an optimist I am.)
>
>  I'm open to suggestions (and patches!) of other approaches to solve this
> problem.

#1 would be my preference. We should probably rename the macro (and
change the case since it's no longer a macro) so as to *force* folks
to consider this.
#2 is a different way of spelling #5; it would defeat the purpose.
I don't understand what you mean by #3 and #4; change *which* length?
The phrasing of #3 using "hopefully-big-enough" and "odds" immediately
makes me think "buffer overflow attack" which is a non-starter.

Finally (unrelated to the memory problem) I'd like to see some
benchmarks to prove that this is really worth it.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From larry at hastings.org  Fri Jan 12 20:27:42 2007
From: larry at hastings.org (Larry Hastings)
Date: Fri, 12 Jan 2007 11:27:42 -0800
Subject: [Python-3000] Lazy strings (was Re: Py3k release schedule
	worries)
In-Reply-To: <ca471dc20701121025s65e04c62v6d76a58afcf2fe60@mail.gmail.com>
References: <20061223155403.GA13959@panix.com> <4596F51F.3010701@hastings.org>	
	<ca471dc20612301832s4aa0f1c4rac0709e5d9800a55@mail.gmail.com>	
	<45A70FF0.1010300@hastings.org>	
	<ca471dc20701120716s7166f458le5eb632b1bf8345a@mail.gmail.com>	
	<45A7C9E9.2050308@hastings.org>
	<ca471dc20701121025s65e04c62v6d76a58afcf2fe60@mail.gmail.com>
Message-ID: <45A7E12E.4060908@hastings.org>


> I don't understand what you mean by #3 and #4; change *which* length?
> The phrasing of #3 using "hopefully-big-enough" and "odds" immediately
> makes me think "buffer overflow attack" which is a non-starter.
Change the length of the string.

Example: our lazy string should be 4032 bytes long, but PyMem_NEW() 
returns NULL.  If I change the string's length to 0 and return a 
non-NULL string, then callers don't have to explicitly check for 
failure; all they need do to work correctly is ask for the length 
*after* asking for the pointer.  That way, they'll think the string is 
zero-length, they won't do any processing, and they'll exit normally.  
(The "hopefully-big-enough" buffer was only in case they'd asked for the 
length *first*; it was just to give the caller something to examine.  If 
it's longer then the string they tried to render they won't crash.)  
Truncating the string changes the meaning of the program, but I figured 
that's okay as we've already thrown a memory exception and the program 
has already failed.

Actually I think this approach could work pretty well if we were willing 
to change the API.  If PyUnicode_AS_UNICODE(self, p, len) gave you the 
Py_UNICODE * in p and the length in len, I could ensure that len was 0 
if rendering failed.  The API's contract would require that if you call 
PyUnicode_AS_UNICODE(), you must use the length it gives you 
(previously-queried lengths might be "stale"), but this would be easy 
for callers to get right.  I can produce a variant of the patch that 
works this way if you think it'll help.

> Finally (unrelated to the memory problem) I'd like to see some 
> benchmarks to prove that this is really worth it.
I'll try to post some today.  For 8-bit strings, concatenation was about 
as fast as the array-append-then-"".join() idiom, and the StringSlicing 
benchark in pystone was 65% faster.

Cheers,


/larry/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20070112/67b7abe7/attachment.html 

From guido at python.org  Fri Jan 12 21:31:24 2007
From: guido at python.org (Guido van Rossum)
Date: Fri, 12 Jan 2007 12:31:24 -0800
Subject: [Python-3000] Lazy strings (was Re: Py3k release schedule
	worries)
In-Reply-To: <45A7E12E.4060908@hastings.org>
References: <20061223155403.GA13959@panix.com> <4596F51F.3010701@hastings.org>
	<ca471dc20612301832s4aa0f1c4rac0709e5d9800a55@mail.gmail.com>
	<45A70FF0.1010300@hastings.org>
	<ca471dc20701120716s7166f458le5eb632b1bf8345a@mail.gmail.com>
	<45A7C9E9.2050308@hastings.org>
	<ca471dc20701121025s65e04c62v6d76a58afcf2fe60@mail.gmail.com>
	<45A7E12E.4060908@hastings.org>
Message-ID: <ca471dc20701121231l33593fc6le879d1c96abccb33@mail.gmail.com>

[Guido]
> > I don't understand what you mean by #3 and #4; change *which* length?
> > The phrasing of #3 using "hopefully-big-enough" and "odds" immediately
> > makes me think "buffer overflow attack" which is a non-starter.

On 1/12/07, Larry Hastings <larry at hastings.org> wrote:
> Change the length of the string.

But IIUC the string may already have been seen by other code, right?
This violates immutability, and that's not acceptable.

>  Example: our lazy string should be 4032 bytes long, but PyMem_NEW() returns
> NULL.  If I change the string's length to 0 and return a non-NULL string,
> then callers don't have to explicitly check for failure; all they need do to
> work correctly is ask for the length *after* asking for the pointer.  That
> way, they'll think the string is zero-length, they won't do any processing,
> and they'll exit normally.  (The "hopefully-big-enough" buffer was only in
> case they'd asked for the length *first*; it was just to give the caller
> something to examine.  If it's longer then the string they tried to render
> they won't crash.)  Truncating the string changes the meaning of the
> program, but I figured that's okay as we've already thrown a memory
> exception and the program has already failed.

That's not the right attitude towards memory exceptions. You're
supposed to be able to clean up in finally clauses and __del__
methods, and all that still requires that existing objects remain
intact.

You seem to be proposing to change the MemoryError exception into a
fatal error (which would mean *no* cleanup happens) but that seems a
real reduction in functionality.

>  Actually I think this approach could work pretty well if we were willing to
> change the API.

Changing the API is the only reasonable solution amongst all the
options I've seen.

> If PyUnicode_AS_UNICODE(self, p, len) gave you the
> Py_UNICODE * in p and the length in len, I could ensure that len was 0 if
> rendering failed.  The API's contract would require that if you call
> PyUnicode_AS_UNICODE(), you must use the length it gives you
> (previously-queried lengths might be "stale"), but this would be easy for
> callers to get right.  I can produce a variant of the patch that works this
> way if you think it'll help.

> > Finally (unrelated to the memory problem) I'd like to see some benchmarks to
> > prove that this is really worth it.

>  I'll try to post some today.  For 8-bit strings, concatenation was about as
> fast as the array-append-then-"".join() idiom, and the StringSlicing
> benchark in pystone was 65% faster.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From larry at hastings.org  Fri Jan 12 22:12:52 2007
From: larry at hastings.org (Larry Hastings)
Date: Fri, 12 Jan 2007 13:12:52 -0800
Subject: [Python-3000] Lazy strings (was Re: Py3k release schedule
	worries)
In-Reply-To: <ca471dc20701121231l33593fc6le879d1c96abccb33@mail.gmail.com>
References: <20061223155403.GA13959@panix.com> <4596F51F.3010701@hastings.org>	
	<ca471dc20612301832s4aa0f1c4rac0709e5d9800a55@mail.gmail.com>	
	<45A70FF0.1010300@hastings.org>	
	<ca471dc20701120716s7166f458le5eb632b1bf8345a@mail.gmail.com>	
	<45A7C9E9.2050308@hastings.org>	
	<ca471dc20701121025s65e04c62v6d76a58afcf2fe60@mail.gmail.com>	
	<45A7E12E.4060908@hastings.org>
	<ca471dc20701121231l33593fc6le879d1c96abccb33@mail.gmail.com>
Message-ID: <45A7F9D4.9090407@hastings.org>

  Guido van Rossum wrote:
> Changing the API is the only reasonable solution amongst all the 
> options I've seen.
I defer to you.  Perhaps another approach will surface; in the meantime 
I'll start on the API change.

You suggested changing the name as a reminder of the semantics change.  
Are you interested in a structural change too?  My personal aesthetic 
for APIs is to return success/failure separately from output 
parameters.  So I propose something this:
    #define PyUnicode_VALUE(self, p, len)
which would return nonzero on success and zero on failure.  If it fails, 
p and len would be unchanged.

Let me know what you'd like and I'll make it happen.

Cheers,


/larry/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20070112/4f0c540c/attachment.html 

From guido at python.org  Fri Jan 12 22:54:19 2007
From: guido at python.org (Guido van Rossum)
Date: Fri, 12 Jan 2007 13:54:19 -0800
Subject: [Python-3000] Lazy strings (was Re: Py3k release schedule
	worries)
In-Reply-To: <45A7F9D4.9090407@hastings.org>
References: <20061223155403.GA13959@panix.com> <4596F51F.3010701@hastings.org>
	<ca471dc20612301832s4aa0f1c4rac0709e5d9800a55@mail.gmail.com>
	<45A70FF0.1010300@hastings.org>
	<ca471dc20701120716s7166f458le5eb632b1bf8345a@mail.gmail.com>
	<45A7C9E9.2050308@hastings.org>
	<ca471dc20701121025s65e04c62v6d76a58afcf2fe60@mail.gmail.com>
	<45A7E12E.4060908@hastings.org>
	<ca471dc20701121231l33593fc6le879d1c96abccb33@mail.gmail.com>
	<45A7F9D4.9090407@hastings.org>
Message-ID: <ca471dc20701121354i37752101oc02c9f529cadde1@mail.gmail.com>

On 1/12/07, Larry Hastings <larry at hastings.org> wrote:
>  Guido van Rossum wrote:
> Changing the API is the only reasonable solution amongst all the options
> I've seen. I defer to you.  Perhaps another approach will surface; in the
> meantime I'll start on the API change.
>
>  You suggested changing the name as a reminder of the semantics change.  Are
> you interested in a structural change too?  My personal aesthetic for APIs
> is to return success/failure separately from output parameters.  So I
> propose something this:
>      #define PyUnicode_VALUE(self, p, len)
>  which would return nonzero on success and zero on failure.  If it fails, p
> and len would be unchanged.

I like having a functionm (not macro) that returns pointer and length
through output variables. Yould call it as PyUnicode_Value(self, &p,
&len).

I don't like having a separate 'success' return value; in that case,
you could just set p to NULL and len to -1. Or if you like shortcuts,
you could make p the return value and len an output parameter, or vice
versa.

Experiment, see what results in the cleanest code in typical usage.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From larry at hastings.org  Fri Jan 12 23:17:43 2007
From: larry at hastings.org (Larry Hastings)
Date: Fri, 12 Jan 2007 14:17:43 -0800
Subject: [Python-3000] Lazy strings (was Re: Py3k release schedule
	worries)
In-Reply-To: <ca471dc20701121354i37752101oc02c9f529cadde1@mail.gmail.com>
References: <20061223155403.GA13959@panix.com> <4596F51F.3010701@hastings.org>	
	<ca471dc20612301832s4aa0f1c4rac0709e5d9800a55@mail.gmail.com>	
	<45A70FF0.1010300@hastings.org>	
	<ca471dc20701120716s7166f458le5eb632b1bf8345a@mail.gmail.com>	
	<45A7C9E9.2050308@hastings.org>	
	<ca471dc20701121025s65e04c62v6d76a58afcf2fe60@mail.gmail.com>	
	<45A7E12E.4060908@hastings.org>	
	<ca471dc20701121231l33593fc6le879d1c96abccb33@mail.gmail.com>	
	<45A7F9D4.9090407@hastings.org>
	<ca471dc20701121354i37752101oc02c9f529cadde1@mail.gmail.com>
Message-ID: <45A80907.3080308@hastings.org>

Guido van Rossum wrote:
> I like having a function (not macro) that returns pointer and length 
> through output variables.
I proposed a macro because I planned some cheap inlining.  Casts and 
parentheses omitted for *cough* clarity:
    #define PyUnicode_VALUE(self, p, len) ((self->str != NULL) ? (p = 
self->str, len = self->length, 1) : PyUnicode_ComputeValue(self, &p, &len))

> I don't like having a separate 'success' return value; in that case, 
> you could just set p to NULL and len to -1. Or if you like shortcuts, 
> you could make p the return value and len an output parameter, or vice 
> versa.
>
> Experiment, see what results in the cleanest code in typical usage.
Ah, but you see, I think this looks cleanest:

    if (!PyUnicode_VALUE(self, p, len))
        /* failure, p and len are unchanged */
        return;
    /* success, p and len are now valid, continue processing */

But I suspected you might prefer something else, which is why I asked.

Lacking more concrete guidance, I'll do it that way and let you tell me 
to change it.  :)

Cheers,


/larry/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20070112/0ff231c4/attachment.htm 

From jimjjewett at gmail.com  Fri Jan 12 23:18:41 2007
From: jimjjewett at gmail.com (Jim Jewett)
Date: Fri, 12 Jan 2007 17:18:41 -0500
Subject: [Python-3000] Lazy strings (was Re: Py3k release schedule
	worries)
In-Reply-To: <ca471dc20701121231l33593fc6le879d1c96abccb33@mail.gmail.com>
References: <20061223155403.GA13959@panix.com> <4596F51F.3010701@hastings.org>
	<ca471dc20612301832s4aa0f1c4rac0709e5d9800a55@mail.gmail.com>
	<45A70FF0.1010300@hastings.org>
	<ca471dc20701120716s7166f458le5eb632b1bf8345a@mail.gmail.com>
	<45A7C9E9.2050308@hastings.org>
	<ca471dc20701121025s65e04c62v6d76a58afcf2fe60@mail.gmail.com>
	<45A7E12E.4060908@hastings.org>
	<ca471dc20701121231l33593fc6le879d1c96abccb33@mail.gmail.com>
Message-ID: <fb6fbf560701121418k2c8c8fffq21ff1ede5f83bba8@mail.gmail.com>

On 1/12/07, Guido van Rossum <guido at python.org> wrote:
> [Guido]
> > > I don't understand what you mean by #3 and #4; change *which* length?
> > > The phrasing of #3 using "hopefully-big-enough" and "odds" immediately
> > > makes me think "buffer overflow attack" which is a non-starter.

I think 3 and 4 are just different ways of salvaging even buggy code.
If you're ready to change the name and insist that they check before
compiling, then those options aren't needed.

> On 1/12/07, Larry Hastings <larry at hastings.org> wrote:
> > Change the length of the string.

> But IIUC the string may already have been seen by other code, right?
> This violates immutability, and that's not acceptable.

No.

IIRC, the string was rendered as soon as any code viewed it.  Since it
isn't rendered yet, nothing has viewed it previously.

> That's not the right attitude towards memory exceptions. You're
> supposed to be able to clean up in finally clauses and __del__
> methods, and all that still requires that existing objects remain
> intact.

But it is true that this may prevent rendering an object that
something *thought* was already created, unless Larry probes deeper
into the twisty passages of fallbacks.

-jJ

From larry at hastings.org  Fri Jan 12 23:41:49 2007
From: larry at hastings.org (Larry Hastings)
Date: Fri, 12 Jan 2007 14:41:49 -0800
Subject: [Python-3000] Lazy strings (was Re: Py3k release schedule
	worries)
In-Reply-To: <fb6fbf560701121418k2c8c8fffq21ff1ede5f83bba8@mail.gmail.com>
References: <20061223155403.GA13959@panix.com> <4596F51F.3010701@hastings.org>	
	<ca471dc20612301832s4aa0f1c4rac0709e5d9800a55@mail.gmail.com>	
	<45A70FF0.1010300@hastings.org>	
	<ca471dc20701120716s7166f458le5eb632b1bf8345a@mail.gmail.com>	
	<45A7C9E9.2050308@hastings.org>	
	<ca471dc20701121025s65e04c62v6d76a58afcf2fe60@mail.gmail.com>	
	<45A7E12E.4060908@hastings.org>	
	<ca471dc20701121231l33593fc6le879d1c96abccb33@mail.gmail.com>
	<fb6fbf560701121418k2c8c8fffq21ff1ede5f83bba8@mail.gmail.com>
Message-ID: <45A80EAD.1050809@hastings.org>

Jim Jewett wrote:
> On 1/12/07, Guido van Rossum <guido at python.org> wrote:
>> But IIUC the string may already have been seen by other code, right?
>> This violates immutability, and that's not acceptable.
> No.
>
> IIRC, the string was rendered as soon as any code viewed it.  Since it 
> isn't rendered yet, nothing has viewed it previously.
"Yes," because someone (the caller) is viewing it *now*.  The caller 
might compute something based on the invalid (unrendered) string and 
think it had been successful.  If those results outlive the memory 
exception and get used for further processing, the program has just 
silently diverged from correct behavior.

Of the five I've proposed so far, Guido's right, only changing the API 
will preserve the speedup *and* result in correct behavior.  But I 
maintain some dim hope that a sixth exists, waiting only to be suggested 
by some shining intellect.

Cheers,


/larry/

From larry at hastings.org  Sat Jan 13 00:59:08 2007
From: larry at hastings.org (Larry Hastings)
Date: Fri, 12 Jan 2007 15:59:08 -0800
Subject: [Python-3000] Lazy strings (was Re: Py3k release schedule
	worries)
In-Reply-To: <ca471dc20701121025s65e04c62v6d76a58afcf2fe60@mail.gmail.com>
References: <20061223155403.GA13959@panix.com> <4596F51F.3010701@hastings.org>	
	<ca471dc20612301832s4aa0f1c4rac0709e5d9800a55@mail.gmail.com>	
	<45A70FF0.1010300@hastings.org>	
	<ca471dc20701120716s7166f458le5eb632b1bf8345a@mail.gmail.com>	
	<45A7C9E9.2050308@hastings.org>
	<ca471dc20701121025s65e04c62v6d76a58afcf2fe60@mail.gmail.com>
Message-ID: <45A820CC.80108@hastings.org>

Guido van Rossum wrote:
> Finally (unrelated to the memory problem) I'd like to see some 
> benchmarks to prove that this is really worth it.
Here's a first cut at some benchmarks.  I gently hacked the pybench in 
Tools so it'd run, and compared the full "lazy strings" patch to an 
unpatched tree.  In the following output, "this" is unpatched and 
"other" has the lazy patch.  The envelope please:

-------------------------------------------------------------------------------
PYBENCH 2.0
-------------------------------------------------------------------------------
* using Python 3.0x
* disabled garbage collection
* system check interval set to maximum: 2147483647
* using timer: time.clock

Running 10 round(s) of the suite at warp factor 10:

Test                             minimum run-time        average  run-time
                                 this    other   diff    this    other   
diff
-------------------------------------------------------------------------------
                 ConcatUnicode:   185ms    46ms +298.6%   206ms    48ms 
+332.7%
       CreateUnicodeWithConcat:   129ms    67ms  +93.1%   132ms    71ms  
+86.5%
                UnicodeSlicing:   156ms    75ms +108.0%   161ms    77ms 
+108.9%
-------------------------------------------------------------------------------
Totals:                          8350ms  8148ms   +2.5%  8586ms  
8416ms   +2.0%


I'll post a zip file containing the full results and the pybench data 
files to the patch page.

As I suspected, this is a bigger win than it was with 8-bit strings, as 
8-bit strings have gotten a lot more TLC over the years.  Once we 
propagate the accumulated tweaks from stringobject.c to unicodeobject.c 
the improvement will be a little less dramatic.  Then again, some of 
those improvements help lazy evaluation too, most notably the 
concatenation speed hack in Python/ceval.c string_concatenation()  (see 
line 4179, comment "In the common case").

Keep in mind that "lazy slices" means more than just [:] notation; I 
converted things like str.split() and str.strip() and str.partition() to 
generate lazy slices too, and nearly all of unicodeobject.c will process 
lazy slices directly without rendering.  So the speed improvements (and 
corresponding change in memory usage) affects more than you might 
suspect at first glance.

Cheers,


/larry/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20070112/3c117f6e/attachment.html 

From jcarlson at uci.edu  Sat Jan 13 07:27:08 2007
From: jcarlson at uci.edu (Josiah Carlson)
Date: Fri, 12 Jan 2007 22:27:08 -0800
Subject: [Python-3000] Lazy strings (was Re: Py3k release schedule
	worries)
In-Reply-To: <45A80EAD.1050809@hastings.org>
References: <fb6fbf560701121418k2c8c8fffq21ff1ede5f83bba8@mail.gmail.com>
	<45A80EAD.1050809@hastings.org>
Message-ID: <20070112220621.8DBA.JCARLSON@uci.edu>


Larry Hastings <larry at hastings.org> wrote:
> Of the five I've proposed so far, Guido's right, only changing the API 
> will preserve the speedup *and* result in correct behavior.  But I 
> maintain some dim hope that a sixth exists, waiting only to be suggested 
> by some shining intellect.

There is a 6th option, which I offered before you wrote your patch,
which I offered again in the last day or so, but which you have refused
because you believe there should be one obvious way to do it.  A wraper
type.

I agree that there should be only one way, however, your proposed change
isn't just about improving speed, it's about changing the semantics of
the underlying type in a way that makes the underlying type difficult to
understand and maintain.  Nevermind the 400+ other changes necessary.

Even better, a wrapper type for unicode makes it *simple* to have a
wrapper type for bytes, buffers, arrays, etc., which offer the same
methods as unicode (which are quite useful for certain file format and
socket protocol parsing operations), which could be used to improve
*many* types, not just unicode.

But hey, don't listen to me, I'm just the crazy guy who has been
advocating the simplification of the implementation of str, unicode,
bytes, array, and mmaps through the use of a wrapper type for 4 and 1/2
months.


 - Josiah


From adam at atlas.st  Sat Jan 13 08:23:05 2007
From: adam at atlas.st (Adam Atlas)
Date: Sat, 13 Jan 2007 02:23:05 -0500
Subject: [Python-3000] Possible alternative lambda syntax?
Message-ID: <12AB1E59-DDA9-42F4-B7A9-F653553EED7E@atlas.st>

I've seen <http://mail.python.org/pipermail/python-dev/2006-February/ 
060415.html> and the various discussions about this, so I hope I'm  
not beating an utterly dead horse here, so if I am, please ignore  
this... but I've been thinking about this issue and I had a couple of  
ideas that I wanted to air.

The first is inspired a little bit by Python 2.4's generator  
expressions. Imagine this as the syntax: (a, b: a+b). I find that to  
be quite simple and Pythonic. I don't see it having been proposed by  
anyone on the wiki's AlternateLambdaSyntax page, but it seems pretty  
intuitive. Certainly better than some of the odd proposals on that  
page like (a + b from args(a, b)).

The second is an interesting idea for a syntax abstraction... imagine  
if the 'def' keyword and the syntax for creating a function were  
decoupled. def x y would assign the value y to the name x in the  
local namespace, and, if y has a __name__ property, assign x to it.  
The value b could technically be anything, not just a function.  
Meanwhile, the syntax for constructing a function object would be  
such that plain function definitions would look the same as they do  
now, but it would also be usable elsewhere. The syntax would be  
(arglist): statements. Where there's no ambiguity (such as after a  
'def' or in parentheses), it could be multi-line as usual.  
Advantages: Lambdas and normal functions would be one in the same  
(internally); what we now call a lambda would be defined like ((a,  
b): return a+b), which would be taken just like any other function  
but without its __name__ defined. Not as concise for lambdas as my  
first idea, but it's more consistent all around, and you can still  
easily see what's going on. It could also be used to pass whole  
multiline functions anonymously as arguments, e.g.:

     doSomething((a, b):
         print a, b
         return a+b
     )

That way, there's no need for adding braces or anything unthinkable  
like that. When it's somewhere more complex, like between other  
arguments to a function, or somewhere where its arglist could be  
misinterpreted as a tuple, just enclose the whole construction in  
parentheses to remove any ambiguity. That way the parsing could  
remain simple.

Any thoughts on either of these?

From larry at hastings.org  Sat Jan 13 10:16:39 2007
From: larry at hastings.org (Larry Hastings)
Date: Sat, 13 Jan 2007 01:16:39 -0800
Subject: [Python-3000] Lazy strings (was Re: Py3k release schedule
	worries)
In-Reply-To: <ca471dc20701121354i37752101oc02c9f529cadde1@mail.gmail.com>
References: <20061223155403.GA13959@panix.com> <4596F51F.3010701@hastings.org>	
	<ca471dc20612301832s4aa0f1c4rac0709e5d9800a55@mail.gmail.com>	
	<45A70FF0.1010300@hastings.org>	
	<ca471dc20701120716s7166f458le5eb632b1bf8345a@mail.gmail.com>	
	<45A7C9E9.2050308@hastings.org>	
	<ca471dc20701121025s65e04c62v6d76a58afcf2fe60@mail.gmail.com>	
	<45A7E12E.4060908@hastings.org>	
	<ca471dc20701121231l33593fc6le879d1c96abccb33@mail.gmail.com>	
	<45A7F9D4.9090407@hastings.org>
	<ca471dc20701121354i37752101oc02c9f529cadde1@mail.gmail.com>
Message-ID: <45A8A377.70907@hastings.org>

Guido van Rossum wrote:
> Experiment, see what results in the cleanest code in typical usage.
I experimented, and surprise! it's the API we already have.  Returning 
both "p" and "len" was only helpful for the ill-fated "options #3 and 
#4"; once that requirement dropped away, the best approach became that 
old C chestnut, the inline assignment:

    if ((p = PyUnicode_AS_UNICODE(self)) == NULL)
       return fail_or_whatever();

So I guess I consider the patch ready for initial review as-is.

Not ready for /acceptance,/ of course; there's plenty more work to do, 
fixing up the callees and finding a better name for the above macro.  
But I'll hold off on that for now following the principle of "lazy 
improvement".

Cheers,


/larry/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20070113/2be48523/attachment.html 

From tomerfiliba at gmail.com  Sat Jan 13 16:07:04 2007
From: tomerfiliba at gmail.com (tomer filiba)
Date: Sat, 13 Jan 2007 17:07:04 +0200
Subject: [Python-3000] Possible alternative lambda syntax?
Message-ID: <1d85506f0701130707we7394a7o62c3cfa43882c317@mail.gmail.com>

[Adam Atlas]
> Imagine this as the syntax: (a, b: a+b)

i tend to like this feature. i'm either +0 or +1, i can't decide at the moment.
i always found lambda to be cumbersome and verbose. but i'm strictly -1
on having meaningful whitespace inside parenthesis, as your other
idea suggested.

when i try to think of it in practice (as i use lambdas in construct),
it looks like too many parenthesis too me:

Struct("foo",
    UBInt8("length"),
    Field("value", (ctx: ctx.length))
)

maybe omit the parenthesis at all?
Field("value", ctx: ctx.length)

you would only need parenthesis when you have multiple
arguments, i.e.

Field("foo", (a, b): a + b)

but then again, it's not too consistent either :-/

i guess the lambda grammar really is as good as it gets.


-tomer

From g.brandl at gmx.net  Sat Jan 13 16:12:52 2007
From: g.brandl at gmx.net (Georg Brandl)
Date: Sat, 13 Jan 2007 15:12:52 +0000
Subject: [Python-3000] Possible alternative lambda syntax?
In-Reply-To: <1d85506f0701130707we7394a7o62c3cfa43882c317@mail.gmail.com>
References: <1d85506f0701130707we7394a7o62c3cfa43882c317@mail.gmail.com>
Message-ID: <eoastk$7hj$1@sea.gmane.org>

tomer filiba schrieb:
> [Adam Atlas]
>> Imagine this as the syntax: (a, b: a+b)
> 
> i tend to like this feature.

Could you please move lambda syntax discussions to python-ideas
and not let them swash back ever again?

Georg


From tony at PageDNA.com  Sat Jan 13 18:24:46 2007
From: tony at PageDNA.com (Tony Lownds)
Date: Sat, 13 Jan 2007 09:24:46 -0800
Subject: [Python-3000] Compiling 2.x code under 3k
Message-ID: <F59F799D-5B72-4F48-9D22-786EA94B3968@PageDNA.com>

While compiling psyco under Python 3000, I ran into some issues that  
might be of interest.

At the C level:

statichere & staticforward caused the oddest error messages.

The removal of Py_TPFLAGS_HAVE_XXX touched the most places.

I think it be possible to redefine PyType_HasFeature and all of the  
old flags
in a compatibility header file, something like :

#define PyType_HasFeature(t,f)  ((f) && ( ((t)->tp_flags & (f) ) != 0))

nb_divide and nb_nonzero changes popped up a lot too.

At the python level:

Forced absolute imports was the only syntax issue not handled by 2to3.

This wrinkle of absolute imports surprised me:

 >>> from .sibling import *
   File "<stdin>", line 1
SyntaxError: 'import *' not allowed with 'from .'

This is a Python 3000 behavior:

 >>> hash(sys._getframe())
Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
TypeError: unhashable type: 'frame'

Potentially a bunch of types will become unhashable since this
code was removed from PyObject_Hash:

         if (tp->tp_compare == NULL && RICHCOMPARE(tp) == NULL) {
                 return _Py_HashPointer(v); /* Use address as hash  
value */
         }

I was wondering whether those two behaviors could be changed.

Thanks
-Tony


From brett at python.org  Sat Jan 13 20:10:02 2007
From: brett at python.org (Brett Cannon)
Date: Sat, 13 Jan 2007 11:10:02 -0800
Subject: [Python-3000] possible new packages (PEP 3108)
In-Reply-To: <-8400800251376027017@unknownmsgid>
References: <bbaeab100701091427ob6cc8c2jd45feb9ee22b9889@mail.gmail.com>
	<284183686114447699@unknownmsgid>
	<bbaeab100701091934v4c99b32bn11b0be402218efca@mail.gmail.com>
	<-4021698297383678739@unknownmsgid>
	<fb6fbf560701100950n160aeaf3w9abe69dc6b1ff26c@mail.gmail.com>
	<3334693148356440096@unknownmsgid>
	<bbaeab100701101143q175436c6wd215f8b626868a97@mail.gmail.com>
	<-8400800251376027017@unknownmsgid>
Message-ID: <bbaeab100701131110udc47bdbv57d9e72220360f2@mail.gmail.com>

On 1/10/07, Bill Janssen <janssen at parc.com> wrote:
> > Otherwise I am removing the Open Issue and I will
> > leave it up to someone else to write up a PEP and pitch it to the list
> > and Guido.
>
> That would be my recommendation.
>

Lo and behold, that's what happened!  =)

-Brett

From brett at python.org  Sat Jan 13 20:27:50 2007
From: brett at python.org (Brett Cannon)
Date: Sat, 13 Jan 2007 11:27:50 -0800
Subject: [Python-3000] PEP 3108 is ready for pronouncement
Message-ID: <bbaeab100701131127g1889ec9cj151e9d6c7476437c@mail.gmail.com>

With the package reorganization having been killed from lack of
interest I now consider PEP 3108 done and ready for Guido to tear it
apart when he has the chance and inclination.  =)

-Brett

From adam at atlas.st  Sat Jan 13 22:48:49 2007
From: adam at atlas.st (Adam Atlas)
Date: Sat, 13 Jan 2007 16:48:49 -0500
Subject: [Python-3000] Possible alternative lambda syntax?
In-Reply-To: <3cdcefb80701130905k7f4c5842t2e4eef2c36d8760c@mail.gmail.com>
References: <12AB1E59-DDA9-42F4-B7A9-F653553EED7E@atlas.st>
	<3cdcefb80701130905k7f4c5842t2e4eef2c36d8760c@mail.gmail.com>
Message-ID: <DADDCBAD-74C6-4148-A7CF-82A842681767@atlas.st>


On 13 Jan 2007, at 12.05, Greg Falcon wrote:
> On 1/13/07, Adam Atlas <adam at atlas.st> wrote:
>> [lambda ideas snipped]
>>
>> Any thoughts on either of these?
>
> Replying privately instead of on the mailing list, because I'm not
> trying to call you out in public or anything like that.
>
> But Guido wrote a great essay that covers what you're suggesting:
> http://www.artima.com/weblogs/viewpost.jsp?thread=147358
>
> Also you should take a look at PEP 3099 before suggesting features for
> Python 3000.
>
> Greg F

(Replying on the mailing list because being called out in public is  
fine with me. Especially by the guy who made "Irrational Exuberance"  
and "Dash". :P)

Okay, I definitely understand Guido's point about switching to paying  
attention to indentation within an expression. I guess that could get  
very ugly. My suggestion for (a, b: a+b) syntax still stands; it  
seems unambiguous enough that we can simply drop the "lambda" keyword  
and the parentheses around the arglist, bringing a syntax that's not  
fundamentally very different from the current one, but quicker to  
write and visually more elegant and simple.

As for allowing statements in lambdas, how about this -- the part  
after the colon can simply be a series of statements (separated by ;,  
not multiline). If so, the function is constructed from those  
statements, and it's expected to have a 'return' instead of just  
having the last statement be an expression. If, rather, the part  
after the colon is one expression, then it acts like it does now.  
It's minimally obtrusive; it doesn't really use any syntax that  
doesn't already exist, it just allows it in a different context.

(Again, apologies if I'm beating a dead horse, or a Dead Parrot as  
the case may be, but these ideas don't seem too far-out to me; I  
could even try implementing this as a patch for the current Py3K  
codebase myself, probably.)

From guido at python.org  Sun Jan 14 00:12:36 2007
From: guido at python.org (Guido van Rossum)
Date: Sat, 13 Jan 2007 15:12:36 -0800
Subject: [Python-3000] Possible alternative lambda syntax?
In-Reply-To: <DADDCBAD-74C6-4148-A7CF-82A842681767@atlas.st>
References: <12AB1E59-DDA9-42F4-B7A9-F653553EED7E@atlas.st>
	<3cdcefb80701130905k7f4c5842t2e4eef2c36d8760c@mail.gmail.com>
	<DADDCBAD-74C6-4148-A7CF-82A842681767@atlas.st>
Message-ID: <ca471dc20701131512x23cc8a5p55f9e86f3fa74f62@mail.gmail.com>

Yes, you're beating a dead horse. Everybody's been hinting at that but
you don't seem to want to hear it. So, again, please stop this
discussion while you have an ounce of credibility left.

--Guido

On 1/13/07, Adam Atlas <adam at atlas.st> wrote:
>
> On 13 Jan 2007, at 12.05, Greg Falcon wrote:
> > On 1/13/07, Adam Atlas <adam at atlas.st> wrote:
> >> [lambda ideas snipped]
> >>
> >> Any thoughts on either of these?
> >
> > Replying privately instead of on the mailing list, because I'm not
> > trying to call you out in public or anything like that.
> >
> > But Guido wrote a great essay that covers what you're suggesting:
> > http://www.artima.com/weblogs/viewpost.jsp?thread=147358
> >
> > Also you should take a look at PEP 3099 before suggesting features for
> > Python 3000.
> >
> > Greg F
>
> (Replying on the mailing list because being called out in public is
> fine with me. Especially by the guy who made "Irrational Exuberance"
> and "Dash". :P)
>
> Okay, I definitely understand Guido's point about switching to paying
> attention to indentation within an expression. I guess that could get
> very ugly. My suggestion for (a, b: a+b) syntax still stands; it
> seems unambiguous enough that we can simply drop the "lambda" keyword
> and the parentheses around the arglist, bringing a syntax that's not
> fundamentally very different from the current one, but quicker to
> write and visually more elegant and simple.
>
> As for allowing statements in lambdas, how about this -- the part
> after the colon can simply be a series of statements (separated by ;,
> not multiline). If so, the function is constructed from those
> statements, and it's expected to have a 'return' instead of just
> having the last statement be an expression. If, rather, the part
> after the colon is one expression, then it acts like it does now.
> It's minimally obtrusive; it doesn't really use any syntax that
> doesn't already exist, it just allows it in a different context.
>
> (Again, apologies if I'm beating a dead horse, or a Dead Parrot as
> the case may be, but these ideas don't seem too far-out to me; I
> could even try implementing this as a patch for the current Py3K
> codebase myself, probably.)
> _______________________________________________
> Python-3000 mailing list
> Python-3000 at python.org
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From tomerfiliba at gmail.com  Sun Jan 14 13:28:23 2007
From: tomerfiliba at gmail.com (tomer filiba)
Date: Sun, 14 Jan 2007 14:28:23 +0200
Subject: [Python-3000] multi-dispatch again
Message-ID: <1d85506f0701140428q2ac7cf7eyd22c7b1ef72ed7e8@mail.gmail.com>

i just thought of a so-to-speak counter-example for ABCs... it's not really
a counter-example, but i believe it shows a deficiency in the concept.

theoretically speaking, objects are a made of type and state. ABCs,
isinstance()
and interfaces at general only check the type part. for example:

@dispatch
def log_to_file(text: str, device: file):
    file.write(text)

this will constrain the *type* of the device, but not its *state*.
practically speaking, i can pass a closed file, or a file open for
reading-only,
and it would pass silently.

basing multi-dispatch on types is of course a leap forward, but if we
already plan to take this leap, why not make it general enough to support
more complex use-cases?

this way we could rewrite the snippet above as

@dispatch
def log_to_file(text: str, device: open_file):
    file.write(text)

where open_file isn't a type, but rather a "checker" that may also examine
the
state. by default, type objects would check for inheritance (via a special
method),
but checkers could extend this behavior.

for efficiency purposes, we can have two decorators:
@type_dispatch - dispatches based on type only
@full_dispatch - dispatches based on type and state

bottom line -- we can't just look at the type of the object for dispatching,

overlooking its state. the state is meaningful, and we'd want the function
not to be called at all if the state of the object is wrong.


-tomer
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20070114/296537df/attachment.htm 

From guido at python.org  Sun Jan 14 17:10:51 2007
From: guido at python.org (Guido van Rossum)
Date: Sun, 14 Jan 2007 08:10:51 -0800
Subject: [Python-3000] multi-dispatch again
In-Reply-To: <1d85506f0701140428q2ac7cf7eyd22c7b1ef72ed7e8@mail.gmail.com>
References: <1d85506f0701140428q2ac7cf7eyd22c7b1ef72ed7e8@mail.gmail.com>
Message-ID: <ca471dc20701140810h66c7d18dl3a2aa1ee84eb3383@mail.gmail.com>

Please move this to python-ideas; it is insufficiently thought through
to be relevant to py3k development.

On 1/14/07, tomer filiba <tomerfiliba at gmail.com> wrote:
> i just thought of a so-to-speak counter-example for ABCs... it's not really
> a counter-example, but i believe it shows a deficiency in the concept.
>
> theoretically speaking, objects are a made of type and state. ABCs,
> isinstance()
> and interfaces at general only check the type part. for example:
>
> @dispatch
> def log_to_file(text: str, device: file):
>     file.write(text)
>
> this will constrain the *type* of the device, but not its *state*.
> practically speaking, i can pass a closed file, or a file open for
> reading-only,
> and it would pass silently.
>
> basing multi-dispatch on types is of course a leap forward, but if we
> already plan to take this leap, why not make it general enough to support
> more complex use-cases?
>
> this way we could rewrite the snippet above as
>
> @dispatch
>  def log_to_file(text: str, device: open_file):
>      file.write(text)
>
> where open_file isn't a type, but rather a "checker" that may also examine
> the
> state. by default, type objects would check for inheritance (via a special
> method),
> but checkers could extend this behavior.
>
> for efficiency purposes, we can have two decorators:
> @type_dispatch - dispatches based on type only
> @full_dispatch - dispatches based on type and state
>
> bottom line -- we can't just look at the type of the object for dispatching,
> overlooking its state. the state is meaningful, and we'd want the function
> not to be called at all if the state of the object is wrong.
>
>
> -tomer
> _______________________________________________
> Python-3000 mailing list
> Python-3000 at python.org
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe:
> http://mail.python.org/mailman/options/python-3000/guido%40python.org
>
>
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From armin.ronacher at active-4.com  Mon Jan 15 18:17:10 2007
From: armin.ronacher at active-4.com (Armin Ronacher)
Date: Mon, 15 Jan 2007 17:17:10 +0000 (UTC)
Subject: [Python-3000] New Import Behavior
Message-ID: <loom.20070115T181152-236@post.gmane.org>

Hossa,

I like the python import system but it's limited at some point. Imagine you have
a large application with a plugin system and you want plugins to appear below
`applicationname.plugins` (Me and the pocoo team encountered that issue some
time ago when working on the pocoo component architecture).

Imagine the following package structure::

   yourapp/
     plugins/
       plugin1/
         __init__.py
         foo.py
       plugin2.py

And additional this one:

   thirdparty/
     yourapp/
       plugins/
         plugin3.py

`thirdparty` is in the pythonpath, the folder where in `yourapp` is in too.
If now someone tries to import `yourapp.plugins.plugin3` this would not work
because python stops the import process somewhere in `yourapp/plugins/` not
trying the `thirdparty` folder at all. For plugin systems this is very bad. Java
solves that better.

That limitation currently causes many packages polluting the root namespace in
site-packages that just act as plugins for other libraries or programs (trac
uses eggs as plugins, turbogears does etc)

The solution would be that python continues testing on the pythonpath if one
module in a package does not exist.

Regards,
Armin


From barry at python.org  Mon Jan 15 18:24:01 2007
From: barry at python.org (Barry Warsaw)
Date: Mon, 15 Jan 2007 12:24:01 -0500
Subject: [Python-3000] New Import Behavior
In-Reply-To: <loom.20070115T181152-236@post.gmane.org>
References: <loom.20070115T181152-236@post.gmane.org>
Message-ID: <C21C74F3-4BCD-48E0-B838-B39C38B0BA95@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Jan 15, 2007, at 12:17 PM, Armin Ronacher wrote:

> `thirdparty` is in the pythonpath, the folder where in `yourapp` is  
> in too.
> If now someone tries to import `yourapp.plugins.plugin3` this would  
> not work
> because python stops the import process somewhere in `yourapp/ 
> plugins/` not
> trying the `thirdparty` folder at all. For plugin systems this is  
> very bad. Java
> solves that better.

Use __path__

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRau4t3EjvBPtnXfVAQIlYgP/faKFPIql7ylZQ6bIr9WMGxVq9OVgVGRF
T7baC6Op8NrjiWzqNH6DDMu0CitQ1Clx3j9IHoT0UQeLt8xxHP2+lzPeBb1ZrqHP
XkFRHhzFMKD7sSPF3shNRcubG0U9beUSQzH8X/4osrpLqVsGV5UpwuKLtKVQhk2F
p4rszO/g6ro=
=U/L+
-----END PGP SIGNATURE-----

From pje at telecommunity.com  Mon Jan 15 18:52:55 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 15 Jan 2007 12:52:55 -0500
Subject: [Python-3000] New Import Behavior
In-Reply-To: <loom.20070115T181152-236@post.gmane.org>
Message-ID: <5.1.1.6.0.20070115124943.04347378@sparrow.telecommunity.com>

At 05:17 PM 1/15/2007 +0000, Armin Ronacher wrote:
>Hossa,
>
>I like the python import system but it's limited at some point. Imagine 
>you have
>a large application with a plugin system and you want plugins to appear below
>`applicationname.plugins` (Me and the pocoo team encountered that issue some
>time ago when working on the pocoo component architecture).
>
>Imagine the following package structure::
>
>    yourapp/
>      plugins/
>        plugin1/
>          __init__.py
>          foo.py
>        plugin2.py
>
>And additional this one:
>
>    thirdparty/
>      yourapp/
>        plugins/
>          plugin3.py

There are two existing solutions for this problem: the pkgutil module 
(available since Python 2.3, and updated in 2.5 to support zipped packages 
as well), and setuptools.  Setuptools in fact has *multiple* solutions for 
this problem, and only one of them requires you to set up such elaborate 
structures in the first place.

If you will kindly re-raise the issue on the distutils-sig, I will 
elaborate further there, as this is off-topic for the Python 3000 list.


From exarkun at divmod.com  Mon Jan 15 18:56:42 2007
From: exarkun at divmod.com (Jean-Paul Calderone)
Date: Mon, 15 Jan 2007 12:56:42 -0500
Subject: [Python-3000] New Import Behavior
In-Reply-To: <C21C74F3-4BCD-48E0-B838-B39C38B0BA95@python.org>
Message-ID: <20070115175642.11447.867035588.divmod.quotient.17616@ohm>

On Mon, 15 Jan 2007 12:24:01 -0500, Barry Warsaw <barry at python.org> wrote:
>-----BEGIN PGP SIGNED MESSAGE-----
>Hash: SHA1
>
>On Jan 15, 2007, at 12:17 PM, Armin Ronacher wrote:
>
>> `thirdparty` is in the pythonpath, the folder where in `yourapp` is
>> in too.
>> If now someone tries to import `yourapp.plugins.plugin3` this would
>> not work
>> because python stops the import process somewhere in `yourapp/
>> plugins/` not
>> trying the `thirdparty` folder at all. For plugin systems this is
>> very bad. Java
>> solves that better.
>
>Use __path__

For example:

  http://twistedmatrix.com/trac/browser/trunk/twisted/plugins/__init__.py

Jean-Paul

From armin.ronacher at active-4.com  Mon Jan 15 21:35:21 2007
From: armin.ronacher at active-4.com (Armin Ronacher)
Date: Mon, 15 Jan 2007 20:35:21 +0000 (UTC)
Subject: [Python-3000] New Import Behavior
References: <loom.20070115T181152-236@post.gmane.org>
	<C21C74F3-4BCD-48E0-B838-B39C38B0BA95@python.org>
Message-ID: <loom.20070115T212844-907@post.gmane.org>

Hi,

Barry Warsaw <barry <at> python.org> writes:

> Use __path__

I know that __path__ is a solution. But it's not such a good solution. Say you
have written a pretty large framework with the modules `fw.foo`, `fw.bar` and
many more. Now you want to distribute just `fw.foo` with another module. If the
python import way would be changed this would be much simpler.

I know that setuptools fixes a lot of those issues but on the way to python3000
some improvements in the way modules are imported could be possible imho.

Regards,
Armin


From jcarlson at uci.edu  Tue Jan 16 09:23:09 2007
From: jcarlson at uci.edu (Josiah Carlson)
Date: Tue, 16 Jan 2007 00:23:09 -0800
Subject: [Python-3000] New Import Behavior
In-Reply-To: <loom.20070115T181152-236@post.gmane.org>
References: <loom.20070115T181152-236@post.gmane.org>
Message-ID: <20070116002036.8DCC.JCARLSON@uci.edu>


Armin Ronacher <armin.ronacher at active-4.com> wrote:
> I like the python import system but it's limited at some point. Imagine you have
> a large application with a plugin system and you want plugins to appear below
> `applicationname.plugins` (Me and the pocoo team encountered that issue some
> time ago when working on the pocoo component architecture).
[snip]

There have already been discussions on package placement, import
semantics, etc., discussed here.  From setuptools .egg, to sqlite
databases with versioning, etc.  Read that discussion before continuing
with your suggestions here, you aren't saying anything new.

 - Josiah


From guido at python.org  Tue Jan 16 19:19:29 2007
From: guido at python.org (Guido van Rossum)
Date: Tue, 16 Jan 2007 10:19:29 -0800
Subject: [Python-3000] [Python-Dev] dict.items as attributes [Was: The
	bytes type]
In-Reply-To: <00c601c73998$0a2f1b20$f100a8c0@ryoko>
References: <20070112202227.17094.1740638192.divmod.xquotient.1876@joule.divmod.com>
	<6e9196d20701121513i41ca78c6gc348bd52145ba8d4@mail.gmail.com>
	<9e804ac0701150502i2e65430dtbe766f71eddfdc2b@mail.gmail.com>
	<FAB32448-D96D-4DA6-AC7E-3BE2A7341120@fuhm.net>
	<5.1.1.6.0.20070116114140.04e8b8f0@sparrow.telecommunity.com>
	<00c601c73998$0a2f1b20$f100a8c0@ryoko>
Message-ID: <ca471dc20701161019g5658d346gd295e0b65724dbf0@mail.gmail.com>

[redirecting to the py3k list where this belongs]

On 1/16/07, Tim Delaney <tcdelaney at optusnet.com.au> wrote:
> Phillip J. Eby wrote:
>
> > To be honest, the items() and keys() thing personally baffles me.  If
> > they're supposed to be *views* on the underlying dictionary, wouldn't
> > it
> > make more sense for them to be *attributes* instead of methods?  I.e.
> > dict.items and dict.keys.  Then, we could provide that feature in
> > 2.6, and
> > drop the availability of the callable forms in 3.0.
> >
> > Then you could write code like:
> >
> >     for k,v in somedict.items:
> >         ...
> >
> > And have it work in 2.6 and 3.0.  Meanwhile, .items() would still
> > return a
> > list in 2.6 (but be warnable about with a -3 switch), but go away
> > entirely
> > in 3.0.
>
> I think this comes down to whether or not the views returned have any
> independent state. There's something that tells me that attributes (even
> properties) should not return different objects with independent state -
> working on two views obtained from the same dictionary property should
> either work identically to working on one view bound to two names, or they
> should not be obtained from a property.
>
> But unless I'm mistaken, everything done to a view would pass through to the
> dict, or result in another object that has independent state (e.g. iter())
> so the effect of working on two views of a dict *would* be identical to
> working on two names to the same view. The only case I can think of for
> which we might want to hold state in the view is for detecting concurrent
> modification - I know that iterators should throw exceptions in this case,
> but I can't remember what (if anything) was decided for views.

Indeed, the views won't have independent state (apart from the pointer
to the dict). But independent state is not the only reason to make
something a method instead of an attribute.

I find it hard to explain, but my gut feeling is that using an
attribute instead of a method would be a mistake here. In the past
I've paid deerly for ignoring my gut feelings (about language design
as well as about other things) so I'd rather wait until a rational
explanation comes to me instead of changing my mind now. I'll gives a
few half-reasons that might help:

- We had the same argument (attribute or method) over hash(), and
decided against change (see PEP 3099).

- In terms of API design, choosing methods is actually *more*
compatible with Python 2.x than choosing attributes.

- I looked at the pros and cons of caching the view objects in the
dict as opposed to creating a new one on each use, and decided that
the extra slots needed in every dict to hold three different views
weren't worth the minor savings that could be had by the caching;
caching could even be a net loss since each view would end up living
as long as the underlying dict once created. Any more complicated
approach (weak refs, a separately malloc'ed appendage) would cause
more complexity than it's worth, since the views are super-simple
objects (in terms of their state) anyway.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From tcdelaney at optusnet.com.au  Tue Jan 16 21:45:09 2007
From: tcdelaney at optusnet.com.au (Tim Delaney)
Date: Wed, 17 Jan 2007 07:45:09 +1100
Subject: [Python-3000] [Python-Dev] dict.items as attributes [Was: The
	bytes type]
References: <20070112202227.17094.1740638192.divmod.xquotient.1876@joule.divmod.com>
	<6e9196d20701121513i41ca78c6gc348bd52145ba8d4@mail.gmail.com>
	<9e804ac0701150502i2e65430dtbe766f71eddfdc2b@mail.gmail.com>
	<FAB32448-D96D-4DA6-AC7E-3BE2A7341120@fuhm.net>
	<5.1.1.6.0.20070116114140.04e8b8f0@sparrow.telecommunity.com>
	<00c601c73998$0a2f1b20$f100a8c0@ryoko>
	<ca471dc20701161019g5658d346gd295e0b65724dbf0@mail.gmail.com>
Message-ID: <00d001c739af$36c15b50$f100a8c0@ryoko>

Guido van Rossum wrote:

> I find it hard to explain, but my gut feeling is that using an
> attribute instead of a method would be a mistake here. In the past
> I've paid deerly for ignoring my gut feelings (about language design
> as well as about other things) so I'd rather wait until a rational
> explanation comes to me instead of changing my mind now. I'll gives a
> few half-reasons that might help:

My gut tells me the same thing. I was trying to work out why, and the 
"independent state" bit seemed to be it - except then I worked out that 
there wouldn't be any.

But the idea of a property returning an object as complex as a view still 
feels wrong to me.

Tim Delaney 


From jimjjewett at gmail.com  Tue Jan 16 23:04:53 2007
From: jimjjewett at gmail.com (Jim Jewett)
Date: Tue, 16 Jan 2007 17:04:53 -0500
Subject: [Python-3000] [Python-Dev] dict.items as attributes [Was: The
	bytes type]
In-Reply-To: <00d001c739af$36c15b50$f100a8c0@ryoko>
References: <20070112202227.17094.1740638192.divmod.xquotient.1876@joule.divmod.com>
	<6e9196d20701121513i41ca78c6gc348bd52145ba8d4@mail.gmail.com>
	<9e804ac0701150502i2e65430dtbe766f71eddfdc2b@mail.gmail.com>
	<FAB32448-D96D-4DA6-AC7E-3BE2A7341120@fuhm.net>
	<5.1.1.6.0.20070116114140.04e8b8f0@sparrow.telecommunity.com>
	<00c601c73998$0a2f1b20$f100a8c0@ryoko>
	<ca471dc20701161019g5658d346gd295e0b65724dbf0@mail.gmail.com>
	<00d001c739af$36c15b50$f100a8c0@ryoko>
Message-ID: <fb6fbf560701161404v1986fffcv7eca0e0772850bd1@mail.gmail.com>

Other than dict.items (and .keys and .values) returning a non-list,
are there any other cases where the Py3K idiom can't already be used
in (or at least backported to) Py 2.x?

Is there a chance that this special case could be handled by an import
hook in py 2.6?

For example 2.6 could have an additional LOAD_ATTR_PY3 opcode that
just handles the special cases before delegating to the normal
LOAD_ATTR.

It would be generated instead of LOAD_ATTR whenever

    >>> from __future__ import py3

was in effect.

Supporting 2.5 (and 2.2) would be a bit harder, but I think the new
exception syntax would already require a py3->py2 translator, and
these three names still seem like a fairly simple transform.  So long
as the "real" source is kept in py3 format, autogenerating python 2
code shouldn't be much worse than autogenerating pyc files.

-jJ

From barry at python.org  Tue Jan 16 23:47:18 2007
From: barry at python.org (Barry Warsaw)
Date: Tue, 16 Jan 2007 17:47:18 -0500
Subject: [Python-3000] [Python-Dev] dict.items as attributes [Was: The
	bytes type]
In-Reply-To: <fb6fbf560701161404v1986fffcv7eca0e0772850bd1@mail.gmail.com>
References: <20070112202227.17094.1740638192.divmod.xquotient.1876@joule.divmod.com>
	<6e9196d20701121513i41ca78c6gc348bd52145ba8d4@mail.gmail.com>
	<9e804ac0701150502i2e65430dtbe766f71eddfdc2b@mail.gmail.com>
	<FAB32448-D96D-4DA6-AC7E-3BE2A7341120@fuhm.net>
	<5.1.1.6.0.20070116114140.04e8b8f0@sparrow.telecommunity.com>
	<00c601c73998$0a2f1b20$f100a8c0@ryoko>
	<ca471dc20701161019g5658d346gd295e0b65724dbf0@mail.gmail.com>
	<00d001c739af$36c15b50$f100a8c0@ryoko>
	<fb6fbf560701161404v1986fffcv7eca0e0772850bd1@mail.gmail.com>
Message-ID: <F3974591-324D-41D6-8C33-DECBDCB7CAFF@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Jan 16, 2007, at 5:04 PM, Jim Jewett wrote:

> Other than dict.items (and .keys and .values) returning a non-list,
> are there any other cases where the Py3K idiom can't already be used
> in (or at least backported to) Py 2.x?

I know Guido is against attribute syntax for dict.items and friends,  
and I agree with him for reasons I can't quite put my finger on.   
But, would it be possible to support both the py3k way and the Python  
2 way if you accepted attribute syntax for returning the view  
thingie?  Couldn't that view thingie have an __call__ that returned  
the backward compatible list object?

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRa1V9nEjvBPtnXfVAQIODQQAlCI4iykl93zKJeIW/3QuuOm4jCRrhffV
BOkuu+mM4N/1Rqc2TCBHUIi995TkYo/6Wuzb7LrXweG5kLm2cdWTNm+aSgZq35HL
XxPVP9qwCElKbZjl84udx/dbLdl9AfVasT2iVUiEVZN3Kq5j+UbyK+oaQdVjBc+5
j51iDW+vhcc=
=Qq7c
-----END PGP SIGNATURE-----

From janssen at parc.com  Wed Jan 17 00:28:46 2007
From: janssen at parc.com (Bill Janssen)
Date: Tue, 16 Jan 2007 15:28:46 PST
Subject: [Python-3000] multi-dispatch again
In-Reply-To: <1d85506f0701140428q2ac7cf7eyd22c7b1ef72ed7e8@mail.gmail.com> 
References: <1d85506f0701140428q2ac7cf7eyd22c7b1ef72ed7e8@mail.gmail.com>
Message-ID: <07Jan16.152847pst."58648"@synergy1.parc.xerox.com>

Of course, the purpose of using ABCs is to allow easy inspection of
the capabilities of an object, and for some objects, the state can be
an important part.  But don't be misled to confuse ABCs with simply
type-based function dispatch.  They're useful for other things as
well.

I think that stateful value types should probably have, as you say,
mechanisms for inspecting that state.  For instance, the current
"file" type has a "closed" attribute, doesn't it?

Bill

From tomerfiliba at gmail.com  Wed Jan 17 10:25:43 2007
From: tomerfiliba at gmail.com (tomer filiba)
Date: Wed, 17 Jan 2007 11:25:43 +0200
Subject: [Python-3000] multi-dispatch again
In-Reply-To: <-5902762711968070404@unknownmsgid>
References: <1d85506f0701140428q2ac7cf7eyd22c7b1ef72ed7e8@mail.gmail.com>
	<-5902762711968070404@unknownmsgid>
Message-ID: <1d85506f0701170125t18a2ea6bhed7e5bfeef01ba0f@mail.gmail.com>

file objects nowadays have ".closed" and ".mode" attributes, but the point
is --
the dispatch mechanism should be able to put constraints on not only on the
*type*, but on the *state* as well -- so once we have multi-dispatch, we
wouldn't need to "manually" check the state of the arguments.

we want to be able to express that in the dispatch itself. imagine this:

def factorial(n):
    if n < 0:
        raise ValueError
    if n < 2:
        return 1
    return n*factorial(n-1)

where i want to write it as

@dispatched
def factorial(n : (lambda n: 0 <= n < 2)):
    return n

@dispatched
def factorial(n : (lambda n: n >= 2)):
    return n * factorial(n-1)

also note that no case is defined for (n < 0), which makes it an exception
automatically. this makes it very much like Haskell's pattern matching.

you can see some more info at http://en.wikipedia.org/wiki/ERights
(look for "guard")


-tomer

On 1/17/07, Bill Janssen <janssen at parc.com> wrote:
>
> Of course, the purpose of using ABCs is to allow easy inspection of
> the capabilities of an object, and for some objects, the state can be
> an important part.  But don't be misled to confuse ABCs with simply
> type-based function dispatch.  They're useful for other things as
> well.
>
> I think that stateful value types should probably have, as you say,
> mechanisms for inspecting that state.  For instance, the current
> "file" type has a "closed" attribute, doesn't it?
>
> Bill
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20070117/02c36fce/attachment.html 

From thomas at python.org  Wed Jan 17 13:19:06 2007
From: thomas at python.org (Thomas Wouters)
Date: Wed, 17 Jan 2007 04:19:06 -0800
Subject: [Python-3000] [Python-Dev] dict.items as attributes [Was: The
	bytes type]
In-Reply-To: <fb6fbf560701161404v1986fffcv7eca0e0772850bd1@mail.gmail.com>
References: <20070112202227.17094.1740638192.divmod.xquotient.1876@joule.divmod.com>
	<6e9196d20701121513i41ca78c6gc348bd52145ba8d4@mail.gmail.com>
	<9e804ac0701150502i2e65430dtbe766f71eddfdc2b@mail.gmail.com>
	<FAB32448-D96D-4DA6-AC7E-3BE2A7341120@fuhm.net>
	<5.1.1.6.0.20070116114140.04e8b8f0@sparrow.telecommunity.com>
	<00c601c73998$0a2f1b20$f100a8c0@ryoko>
	<ca471dc20701161019g5658d346gd295e0b65724dbf0@mail.gmail.com>
	<00d001c739af$36c15b50$f100a8c0@ryoko>
	<fb6fbf560701161404v1986fffcv7eca0e0772850bd1@mail.gmail.com>
Message-ID: <9e804ac0701170419v1d802b5aqd49456231a75f1cc@mail.gmail.com>

On 1/16/07, Jim Jewett <jimjjewett at gmail.com> wrote:
>
> Other than dict.items (and .keys and .values) returning a non-list,
> are there any other cases where the Py3K idiom can't already be used
> in (or at least backported to) Py 2.x?


Well, there is that bit where strings are all unicode, including all string
literals and attribute names and all that. And the bit where open() will be
different, although how different hasn't been decided yet. (Or rather, it
hasn't been written down in code yet, and to me that means it hasn't been
decided yet :)

What we need to do now is do the work on 3.0, and *then* (and a bit
'during', too) think about backward compatibility. As far as I can see, all
the existing changes in the p3yk branch are upgrade-pathable: the int/long
unification has long been done on the Python-visible side, 'except as' will
make it into 2.6, print/exec as functions can be imported from the future,
sys.intern/compile/etc can be made available in 2.6, classic slices are
already on the way out, et cetera ad nauseam. The aim for 2.6 should be to
have all the new features that 3.0 has, as well as full backward
compatibility (albeit potentially with a few extra warnings, as any release
might, but only for code that can be written sanely for 2.5-and-earlier as
well.) The biggest hurdle will be the unicode/string unification and the I/O
system (much bigger than the dict keys/values/items change), but since those
haven't been implemented yet, it's rather pointless to talk about how to
handle them in 2.6.

Is there a chance that this special case could be handled by an import
> hook in py 2.6?


I expect you don't mean an actual import hook, but just a future import?
Then sure, there is a chance. We'll have to see.

-- 
Thomas Wouters <thomas at python.org>

Hi! I'm a .signature virus! copy me into your .signature file to help me
spread!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20070117/94cc8fbb/attachment.htm 

From veloso at verylowsodium.com  Wed Jan 17 14:38:03 2007
From: veloso at verylowsodium.com (Greg Falcon)
Date: Wed, 17 Jan 2007 08:38:03 -0500
Subject: [Python-3000] [Python-Dev] dict.items as attributes [Was: The
	bytes type]
In-Reply-To: <F3974591-324D-41D6-8C33-DECBDCB7CAFF@python.org>
References: <20070112202227.17094.1740638192.divmod.xquotient.1876@joule.divmod.com>
	<6e9196d20701121513i41ca78c6gc348bd52145ba8d4@mail.gmail.com>
	<9e804ac0701150502i2e65430dtbe766f71eddfdc2b@mail.gmail.com>
	<FAB32448-D96D-4DA6-AC7E-3BE2A7341120@fuhm.net>
	<5.1.1.6.0.20070116114140.04e8b8f0@sparrow.telecommunity.com>
	<00c601c73998$0a2f1b20$f100a8c0@ryoko>
	<ca471dc20701161019g5658d346gd295e0b65724dbf0@mail.gmail.com>
	<00d001c739af$36c15b50$f100a8c0@ryoko>
	<fb6fbf560701161404v1986fffcv7eca0e0772850bd1@mail.gmail.com>
	<F3974591-324D-41D6-8C33-DECBDCB7CAFF@python.org>
Message-ID: <3cdcefb80701170538q2a67d3ebt1a75f42970d60b20@mail.gmail.com>

On 1/16/07, Barry Warsaw <barry at python.org> wrote:
> I know Guido is against attribute syntax for dict.items and friends,
> and I agree with him for reasons I can't quite put my finger on.

I agree that making dict.items and friends into attributes feels
wrong.  I suspect it may be useful to put a finger on why.  The
discussion about hash() linked in PEP 3099 led to a set of guidelines
for when properties are inappropriate, and notably, none of these
guidelines apply here.

Two explanations given here so far don't seem to explain my aversion.
It shouldn't matter that views are complex objects; I have a gut
feeling, anyway, that datatype shouldn't be a factor.  Nor does it
seem to matter whether the implementation pre-allocates these views or
creates them on-demand, since views are stateless, and since either
approach has a cheap fixed cost.

After a bit of mulling it over, the best objection I can come up with
is this: views aren't "attributes" or "properties" of a dict object in
the plain-English sense of the term.  This might simply be a flawed
appeal to an overloaded word, but I think it captures my gut feeling
well.  A dictionary object contains keys and values, and those
*contents* can be said to be attributes of a dictionary.  Views are
just wrapper objects that expose those contents in interesting ways.

Along those lines: very recently at work I added a method to a class
to provide an iterator over its items in a non-standard order.
Although I often use 'property' as a decorator, it didn't even occur
to me to make that method an attribute.  It still doesn't feel as
though I should, even though it would be cheap and easy to return a
stateless object with an appropriate __iter__ method.

A second objection: Although this is a clever way of adding
forwards-compatibility to 2.6, the side effect that views are callable
is odd.  It's especially unintuitive that v() is synonymous with
list(v).  The existing stateless, sequence-proxying objects (like
xrange) don't behave that way, and it would be strange if they did.

Part of me suspects that if Python builtin types or library types made
bigger use of attributes, this proposal wouldn't seem so weird.  There
are many opportunities in the library to replace zero-arg, cheap,
side-effect-free methods with read-only attributes, but that hasn't
happened.  This sort of thing still isn't really idiomatic, and just
because Python 3 is a major version bump doesn't mean we *should*
change the idioms.

Greg F

From guido at python.org  Wed Jan 17 16:26:08 2007
From: guido at python.org (Guido van Rossum)
Date: Wed, 17 Jan 2007 07:26:08 -0800
Subject: [Python-3000] multi-dispatch again
In-Reply-To: <1d85506f0701170125t18a2ea6bhed7e5bfeef01ba0f@mail.gmail.com>
References: <1d85506f0701140428q2ac7cf7eyd22c7b1ef72ed7e8@mail.gmail.com>
	<-5902762711968070404@unknownmsgid>
	<1d85506f0701170125t18a2ea6bhed7e5bfeef01ba0f@mail.gmail.com>
Message-ID: <ca471dc20701170726o466e5afbm88b9077e17a70808@mail.gmail.com>

Please move to the python-ideas list (2nd plea!)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From barry at python.org  Wed Jan 17 17:43:48 2007
From: barry at python.org (Barry Warsaw)
Date: Wed, 17 Jan 2007 11:43:48 -0500
Subject: [Python-3000] [Python-Dev] dict.items as attributes [Was: The
	bytes type]
In-Reply-To: <3cdcefb80701170538q2a67d3ebt1a75f42970d60b20@mail.gmail.com>
References: <20070112202227.17094.1740638192.divmod.xquotient.1876@joule.divmod.com>
	<6e9196d20701121513i41ca78c6gc348bd52145ba8d4@mail.gmail.com>
	<9e804ac0701150502i2e65430dtbe766f71eddfdc2b@mail.gmail.com>
	<FAB32448-D96D-4DA6-AC7E-3BE2A7341120@fuhm.net>
	<5.1.1.6.0.20070116114140.04e8b8f0@sparrow.telecommunity.com>
	<00c601c73998$0a2f1b20$f100a8c0@ryoko>
	<ca471dc20701161019g5658d346gd295e0b65724dbf0@mail.gmail.com>
	<00d001c739af$36c15b50$f100a8c0@ryoko>
	<fb6fbf560701161404v1986fffcv7eca0e0772850bd1@mail.gmail.com>
	<F3974591-324D-41D6-8C33-DECBDCB7CAFF@python.org>
	<3cdcefb80701170538q2a67d3ebt1a75f42970d60b20@mail.gmail.com>
Message-ID: <5E5F03C3-717F-4321-8C53-D472C7D07B69@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Jan 17, 2007, at 8:38 AM, Greg Falcon wrote:

> After a bit of mulling it over, the best objection I can come up with
> is this: views aren't "attributes" or "properties" of a dict object in
> the plain-English sense of the term.  This might simply be a flawed
> appeal to an overloaded word, but I think it captures my gut feeling
> well.  A dictionary object contains keys and values, and those
> *contents* can be said to be attributes of a dictionary.  Views are
> just wrapper objects that expose those contents in interesting ways.

> A second objection: Although this is a clever way of adding
> forwards-compatibility to 2.6, the side effect that views are callable
> is odd.  It's especially unintuitive that v() is synonymous with
> list(v).  The existing stateless, sequence-proxying objects (like
> xrange) don't behave that way, and it would be strange if they did.
>
> Part of me suspects that if Python builtin types or library types made
> bigger use of attributes, this proposal wouldn't seem so weird.  There
> are many opportunities in the library to replace zero-arg, cheap,
> side-effect-free methods with read-only attributes, but that hasn't
> happened.  This sort of thing still isn't really idiomatic, and just
> because Python 3 is a major version bump doesn't mean we *should*
> change the idioms.

You make some good points, though I tend to think that the fact that  
these have always been methods may be the largest factor in why  
making them attributes looks weird.

As for the idioms that we tend to use, I've definitely come full  
circle on that.  The API for our product started out using a  
functional (everything methods) interface both for getting and  
setting.  But it just seemed so verbose and frankly unreadable.  Then  
we turned all zero-arg, side-effect-free accessor methods into  
attributes and it make everything so much cleaner and more readable.   
Most of these attributes were read-only so it made clearer sense to  
do this, but there are a few that were read/write and some of those  
were made into attributes.  For others, we kept the setter methods  
mostly because the setters often took additional arguments.  It still  
came down to a case-by-case basis but with the emphasis toward  
readability, attributes seemed more natural.

I do think that as we update the stdlib for py3k, we should think  
about whether setters/getters can be turned into attributes.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRa5SRXEjvBPtnXfVAQKxkgQAlTFO+HARcC73BlcvEfB3z2qB2NuIpYv/
dPHwiLVWkuTC9fjft/2rbEeV5jfOYVaQCrlq9cPGSeQ/H/GkD1JeI1AoBY/U4EZY
yiPW6Opac5FEWCJtSjs5p4Av6Uoq3076lyq6MFqa2ogRSGq8/lw9WVuCjXNIcT8N
w0TWfji15qg=
=rlBy
-----END PGP SIGNATURE-----

From jimjjewett at gmail.com  Wed Jan 17 18:05:36 2007
From: jimjjewett at gmail.com (Jim Jewett)
Date: Wed, 17 Jan 2007 12:05:36 -0500
Subject: [Python-3000] [Python-Dev] dict.items as attributes [Was: The
	bytes type]
In-Reply-To: <9e804ac0701170419v1d802b5aqd49456231a75f1cc@mail.gmail.com>
References: <20070112202227.17094.1740638192.divmod.xquotient.1876@joule.divmod.com>
	<6e9196d20701121513i41ca78c6gc348bd52145ba8d4@mail.gmail.com>
	<9e804ac0701150502i2e65430dtbe766f71eddfdc2b@mail.gmail.com>
	<FAB32448-D96D-4DA6-AC7E-3BE2A7341120@fuhm.net>
	<5.1.1.6.0.20070116114140.04e8b8f0@sparrow.telecommunity.com>
	<00c601c73998$0a2f1b20$f100a8c0@ryoko>
	<ca471dc20701161019g5658d346gd295e0b65724dbf0@mail.gmail.com>
	<00d001c739af$36c15b50$f100a8c0@ryoko>
	<fb6fbf560701161404v1986fffcv7eca0e0772850bd1@mail.gmail.com>
	<9e804ac0701170419v1d802b5aqd49456231a75f1cc@mail.gmail.com>
Message-ID: <fb6fbf560701170905l6cbfe29qf92fd87352ed3290@mail.gmail.com>

On 1/17/07, Thomas Wouters <thomas at python.org> wrote:
> On 1/16/07, Jim Jewett <jimjjewett at gmail.com> wrote:
> > Other than dict.items (and .keys and .values) returning a non-list,
> > are there any other cases where the Py3K idiom can't already be used
> > in (or at least backported to) Py 2.x?

> The aim for 2.6 should be to have all the new features that 3.0 has,
> as well as full backward compatibility ...

And I'm asking if there are any real barriers to this.  Some people
have suggested that they'll have to maintain separate code bases.  So
far, the closest I've seen[1] to something that can't use
common-subset is wanting an iterator over a mapping.

> Well, there is that bit where strings are all unicode, including all string
> literals and attribute names and all that.

(But note that attribute names will still be limited to ASCII.)

I don't see any problems for python code there, unless you're going
out of your way to care which stringlike type you get.

I do see that C extensions could be trickier, though I haven't yet see
anything that couldn't be handled with some relatively simple
conditionally-defined macros.

> And the bit where open() will be different,

This I couldn't find, unless it was the security abilities.  These
wouldn't bother conservative code, and could probably be backported to
2.6, or at least 2.7.

> The biggest hurdle will be the unicode/string unification and the I/O
> system (much bigger than the dict keys/values/items change), but since those
> haven't been implemented yet, it's rather pointless to talk about how to
> handle them in 2.6.

Fair enough.


[1]  I did miss some introspection names, but that can be handled in
python, and anyone using, rather than reinventing, inspect should be
fine.

-jJ

From ncoghlan at gmail.com  Thu Jan 18 13:02:49 2007
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 18 Jan 2007 22:02:49 +1000
Subject: [Python-3000] [Python-Dev] dict.items as attributes [Was: The
 bytes type]
In-Reply-To: <fb6fbf560701170905l6cbfe29qf92fd87352ed3290@mail.gmail.com>
References: <20070112202227.17094.1740638192.divmod.xquotient.1876@joule.divmod.com>	<6e9196d20701121513i41ca78c6gc348bd52145ba8d4@mail.gmail.com>	<9e804ac0701150502i2e65430dtbe766f71eddfdc2b@mail.gmail.com>	<FAB32448-D96D-4DA6-AC7E-3BE2A7341120@fuhm.net>	<5.1.1.6.0.20070116114140.04e8b8f0@sparrow.telecommunity.com>	<00c601c73998$0a2f1b20$f100a8c0@ryoko>	<ca471dc20701161019g5658d346gd295e0b65724dbf0@mail.gmail.com>	<00d001c739af$36c15b50$f100a8c0@ryoko>	<fb6fbf560701161404v1986fffcv7eca0e0772850bd1@mail.gmail.com>	<9e804ac0701170419v1d802b5aqd49456231a75f1cc@mail.gmail.com>
	<fb6fbf560701170905l6cbfe29qf92fd87352ed3290@mail.gmail.com>
Message-ID: <45AF61E9.9090600@gmail.com>

Jim Jewett wrote:
> On 1/17/07, Thomas Wouters <thomas at python.org> wrote:
>> On 1/16/07, Jim Jewett <jimjjewett at gmail.com> wrote:
>>> Other than dict.items (and .keys and .values) returning a non-list,
>>> are there any other cases where the Py3K idiom can't already be used
>>> in (or at least backported to) Py 2.x?
> 
>> The aim for 2.6 should be to have all the new features that 3.0 has,
>> as well as full backward compatibility ...
> 
> And I'm asking if there are any real barriers to this.  Some people
> have suggested that they'll have to maintain separate code bases.  So
> far, the closest I've seen[1] to something that can't use
> common-subset is wanting an iterator over a mapping.

To help with the transition code, could we have a 'py3migration' module 
that looked something like:

   if sys.version_info >= (3, 0, 0):
       from _py2compat import *
   else:
       from _py3compat import *

By providing functions like dkeys(), ditems(), dvalues() and maybe a few 
others in that module, this could allow code that needs to work in both 
environments to be written, without causing significant pain to pure 
Python 2 code or to pure Python 3 code.

OK, so it would mean there's still some backward compatibility cruft in 
Py3k, but one module seems a small price to pay for a single obvious way 
to handle semantic changes in the builtin APIs.

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From collinw at gmail.com  Sun Jan 21 06:46:53 2007
From: collinw at gmail.com (Collin Winter)
Date: Sat, 20 Jan 2007 23:46:53 -0600
Subject: [Python-3000] Exceptions internals and removing sys.exc_*
Message-ID: <43aa6ff70701202146q3cd34739u3a41c0b78f2a23b9@mail.gmail.com>

Guido has mentioned [1] that since exceptions will be growing a
__traceback__ attribute in Python 3, it should be possible to remove
sys.exc_info(). In addition, PEP 3100 mentions the removal of
sys.exc_{type,value,traceback}. With that in mind, is there a reason
to keep the exc_{type,value,traceback} fields on PyThreadState
structs? Is there a reason why these couldn't be consolidated into a
single PyThreadState->exception field in Python 3?

Thanks,
Collin Winter

[1] - http://mail.python.org/pipermail/python-3000/2007-January/005385.html

From tony at PageDNA.com  Sun Jan 21 19:26:36 2007
From: tony at PageDNA.com (Tony Lownds)
Date: Sun, 21 Jan 2007 10:26:36 -0800
Subject: [Python-3000] Exceptions internals and removing sys.exc_*
In-Reply-To: <43aa6ff70701202146q3cd34739u3a41c0b78f2a23b9@mail.gmail.com>
References: <43aa6ff70701202146q3cd34739u3a41c0b78f2a23b9@mail.gmail.com>
Message-ID: <A68BA56D-27F3-4F10-BA26-43057383F1A4@PageDNA.com>


On Jan 20, 2007, at 9:46 PM, Collin Winter wrote:

> Guido has mentioned [1] that since exceptions will be growing a
> __traceback__ attribute in Python 3, it should be possible to remove
> sys.exc_info().

I hope that doesn't happen. sys.exc_info() is the only way to write code
using exception values that will run on both 2.5 and 3.0.

except Exception:
   e = sys.exc_info()[1]

-Tony



From brett at python.org  Sun Jan 21 19:57:56 2007
From: brett at python.org (Brett Cannon)
Date: Sun, 21 Jan 2007 10:57:56 -0800
Subject: [Python-3000] Exceptions internals and removing sys.exc_*
In-Reply-To: <A68BA56D-27F3-4F10-BA26-43057383F1A4@PageDNA.com>
References: <43aa6ff70701202146q3cd34739u3a41c0b78f2a23b9@mail.gmail.com>
	<A68BA56D-27F3-4F10-BA26-43057383F1A4@PageDNA.com>
Message-ID: <bbaeab100701211057m5c9b9cem30d7379be98138e1@mail.gmail.com>

On 1/21/07, Tony Lownds <tony at pagedna.com> wrote:
>
> On Jan 20, 2007, at 9:46 PM, Collin Winter wrote:
>
> > Guido has mentioned [1] that since exceptions will be growing a
> > __traceback__ attribute in Python 3, it should be possible to remove
> > sys.exc_info().
>
> I hope that doesn't happen. sys.exc_info() is the only way to write code
> using exception values that will run on both 2.5 and 3.0.
>
> except Exception:
>    e = sys.exc_info()[1]
>

As has been stated on python-dev, 3.0 will not be hindered by
backwards-compatibility.  2.6 is going to be the version that has both
2.x and 3.0 features to allow for transitioning.

-Brett

From brett at python.org  Sun Jan 21 19:59:45 2007
From: brett at python.org (Brett Cannon)
Date: Sun, 21 Jan 2007 10:59:45 -0800
Subject: [Python-3000] Exceptions internals and removing sys.exc_*
In-Reply-To: <43aa6ff70701202146q3cd34739u3a41c0b78f2a23b9@mail.gmail.com>
References: <43aa6ff70701202146q3cd34739u3a41c0b78f2a23b9@mail.gmail.com>
Message-ID: <bbaeab100701211059p6a6af542va55bdffac08deb39@mail.gmail.com>

On 1/20/07, Collin Winter <collinw at gmail.com> wrote:
> Guido has mentioned [1] that since exceptions will be growing a
> __traceback__ attribute in Python 3, it should be possible to remove
> sys.exc_info(). In addition, PEP 3100 mentions the removal of
> sys.exc_{type,value,traceback}. With that in mind, is there a reason
> to keep the exc_{type,value,traceback} fields on PyThreadState
> structs? Is there a reason why these couldn't be consolidated into a
> single PyThreadState->exception field in Python 3?
>

I don't see why this can't be done.  All three should be available
directly off the exception.  We should provide accessor methods,
though, for that information when given an exception.

-Brett

From tony at pagedna.com  Sun Jan 21 20:50:42 2007
From: tony at pagedna.com (Tony Lownds)
Date: Sun, 21 Jan 2007 11:50:42 -0800
Subject: [Python-3000] Exceptions internals and removing sys.exc_*
In-Reply-To: <bbaeab100701211057m5c9b9cem30d7379be98138e1@mail.gmail.com>
References: <43aa6ff70701202146q3cd34739u3a41c0b78f2a23b9@mail.gmail.com>
	<A68BA56D-27F3-4F10-BA26-43057383F1A4@PageDNA.com>
	<bbaeab100701211057m5c9b9cem30d7379be98138e1@mail.gmail.com>
Message-ID: <1AC1C3B8-7364-4151-9FED-DC03FA867521@pagedna.com>


On Jan 21, 2007, at 10:57 AM, Brett Cannon wrote:

> On 1/21/07, Tony Lownds <tony at pagedna.com> wrote:
>>
>> On Jan 20, 2007, at 9:46 PM, Collin Winter wrote:
>>
>> > Guido has mentioned [1] that since exceptions will be growing a
>> > __traceback__ attribute in Python 3, it should be possible to  
>> remove
>> > sys.exc_info().
>>
>> I hope that doesn't happen. sys.exc_info() is the only way to  
>> write code
>> using exception values that will run on both 2.5 and 3.0.
>>
>> except Exception:
>>    e = sys.exc_info()[1]
>>
>
> As has been stated on python-dev, 3.0 will not be hindered by
> backwards-compatibility.  2.6 is going to be the version that has both
> 2.x and 3.0 features to allow for transitioning.
>

Not allowing 3.0 to be hindered is reasonable, as long as its not a  
blind absolute.
ISTM that backwards compatability ought to be weighed, with a strong  
bias towards
freedom to change in 3.0. Check out this post:

http://mail.python.org/pipermail/python-dev/2007-January/070665.html

What's the cost of keeping sys.exc_info() on 3.0? If that cost is a  
hindrance on 3.0, then
fine, sys.exc_info() and the ability for module authors to use a  
single codebase on 2.x and
3.0 can be given up.

If sys.exc_info() is not a hindrance to keep, why would we want to  
increase incompatibility?

Maybe I'm misreading that post. I'm not trying to cherry pick posts  
or play word games; I just
feel that the loss of the possibility of a single codebase on <=2.5  
and 3.0 should not be given
up lightly.

-Tony



From rasky at develer.com  Mon Jan 22 02:12:25 2007
From: rasky at develer.com (Giovanni Bajo)
Date: Mon, 22 Jan 2007 02:12:25 +0100
Subject: [Python-3000] Exceptions internals and removing sys.exc_*
In-Reply-To: <1AC1C3B8-7364-4151-9FED-DC03FA867521@pagedna.com>
References: <43aa6ff70701202146q3cd34739u3a41c0b78f2a23b9@mail.gmail.com>	<A68BA56D-27F3-4F10-BA26-43057383F1A4@PageDNA.com>	<bbaeab100701211057m5c9b9cem30d7379be98138e1@mail.gmail.com>
	<1AC1C3B8-7364-4151-9FED-DC03FA867521@pagedna.com>
Message-ID: <ep131q$22a$1@sea.gmane.org>

On 21/01/2007 20.50, Tony Lownds wrote:

> What's the cost of keeping sys.exc_info() on 3.0? If that cost is a  
> hindrance on 3.0, then
> fine, sys.exc_info() and the ability for module authors to use a  
> single codebase on 2.x and
> 3.0 can be given up.

It's been stated many times that making it possible to have a single codebase 
for both 2.x and 3.0 is *not* a goal of the Python 3.0 project.

What it is more likely to happen is that it will be possible to maintain a 
codebase which runs under 2.x and can be automatically converted to a 
3.0-compatible codebase. This will be possible through some (all?) of the 
following utilities (nothing is set in stone at this point):

- A "-3" command line option in Python 2.x which turns on DeprecationWarnings 
on any construct that will go away in 3.x
- A "from __future__ import py3k" to activate special Py3k constructs (like 
new keywords) in 2.x.
- The addition of Py3k "features" to Python 2.x, in the cases where it does 
not hinder 2.x maintenance too much (eg: new names for modules/packages in 
stdlib).
- The 2to3 refactoring tool: a program to automatically convert Python 2.x 
source code to Python 3.x source code (an initial version is already available 
in the sandbox).
-- 
Giovanni Bajo


From rrr at ronadam.com  Mon Jan 22 03:33:48 2007
From: rrr at ronadam.com (Ron Adam)
Date: Sun, 21 Jan 2007 20:33:48 -0600
Subject: [Python-3000] Exceptions internals and removing sys.exc_*
In-Reply-To: <1AC1C3B8-7364-4151-9FED-DC03FA867521@pagedna.com>
References: <43aa6ff70701202146q3cd34739u3a41c0b78f2a23b9@mail.gmail.com>	<A68BA56D-27F3-4F10-BA26-43057383F1A4@PageDNA.com>	<bbaeab100701211057m5c9b9cem30d7379be98138e1@mail.gmail.com>
	<1AC1C3B8-7364-4151-9FED-DC03FA867521@pagedna.com>
Message-ID: <ep182v$cm7$1@sea.gmane.org>

Tony Lownds wrote:

> http://mail.python.org/pipermail/python-dev/2007-January/070665.html
> 
> What's the cost of keeping sys.exc_info() on 3.0? If that cost is a  
> hindrance on 3.0, then
> fine, sys.exc_info() and the ability for module authors to use a  
> single codebase on 2.x and
> 3.0 can be given up.

Well, This is how I see it. (but I believe it's been said here before a well)

It's not a present cost, but a future cost.  There have been lots of times where 
a suggestion has been put forth in 2.x to do x,y and z.  Only to be found, no 
that can't be done because it would break, u, v or w.  The more things there 
are, the more chance something new will break one of them.

There are a lot of things that haven't been done yet in computer science, and to 
give python 3.x the best chance of being a part of those future things, (by 
adding them to python), it's best that it is as clean as it can be at the start. 
  Of course 'clean as can be' is subjective, so it's not a line carved in stone. 
  And it's also not one blindly carved.

I expect that no matter how different 2.x is from 3.x, someone will find a way 
to make 2.x programs work on it with a minimum of trouble if not outright as is. 
    Writing an 2.x interpreter for 3.x is probably very doable, (especially with 
a  little 'C' magic thrown in).  And it may not even run slower.  One of the 
benefits of having a clean break, is it gives a sure target for such efforts.

Yes, I'm a optimist. ;-)
   Ron






From brett at python.org  Mon Jan 22 03:43:42 2007
From: brett at python.org (Brett Cannon)
Date: Sun, 21 Jan 2007 18:43:42 -0800
Subject: [Python-3000] Exceptions internals and removing sys.exc_*
In-Reply-To: <1AC1C3B8-7364-4151-9FED-DC03FA867521@pagedna.com>
References: <43aa6ff70701202146q3cd34739u3a41c0b78f2a23b9@mail.gmail.com>
	<A68BA56D-27F3-4F10-BA26-43057383F1A4@PageDNA.com>
	<bbaeab100701211057m5c9b9cem30d7379be98138e1@mail.gmail.com>
	<1AC1C3B8-7364-4151-9FED-DC03FA867521@pagedna.com>
Message-ID: <bbaeab100701211843o672c927bo6e55551a75e16c15@mail.gmail.com>

On 1/21/07, Tony Lownds <tony at pagedna.com> wrote:
>
> On Jan 21, 2007, at 10:57 AM, Brett Cannon wrote:
>
> > On 1/21/07, Tony Lownds <tony at pagedna.com> wrote:
> >>
> >> On Jan 20, 2007, at 9:46 PM, Collin Winter wrote:
> >>
> >> > Guido has mentioned [1] that since exceptions will be growing a
> >> > __traceback__ attribute in Python 3, it should be possible to
> >> remove
> >> > sys.exc_info().
> >>
> >> I hope that doesn't happen. sys.exc_info() is the only way to
> >> write code
> >> using exception values that will run on both 2.5 and 3.0.
> >>
> >> except Exception:
> >>    e = sys.exc_info()[1]
> >>
> >
> > As has been stated on python-dev, 3.0 will not be hindered by
> > backwards-compatibility.  2.6 is going to be the version that has both
> > 2.x and 3.0 features to allow for transitioning.
> >
>
> Not allowing 3.0 to be hindered is reasonable, as long as its not a
> blind absolute.

That's fine, as long as the change is for the better.  Moving the
traceback to exceptions is an improvement.  That also means that
sys.exc_info is redundant and thus does not need to be kept around.

> ISTM that backwards compatability ought to be weighed, with a strong
> bias towards
> freedom to change in 3.0.

It is.  There is a reason why we didn't rename a bunch of exceptions
to make their names better.  It was stated that the change from the
2.x to the suggested 3.0 way was minimal and thus not worth it.  But
the gains of moving the traceback to exceptions has been deemed worth
it.

> Check out this post:
>
> http://mail.python.org/pipermail/python-dev/2007-January/070665.html
>
> What's the cost of keeping sys.exc_info() on 3.0?

The same cost of keeping any redundant, not-as-nice API for anything
else we improve.  There is maintenance costs along with educating
people on two different APIs for the same thing.  TOOWTDI exists for a
reason.

> If that cost is a
> hindrance on 3.0, then
> fine, sys.exc_info() and the ability for module authors to use a
> single codebase on 2.x and
> 3.0 can be given up.
>
> If sys.exc_info() is not a hindrance to keep, why would we want to
> increase incompatibility?
>

3.0 is our chance to clean things up, not just do some small
improvement to the language.  If we are going to take the time to
improve things it should be done completely and done well.

> Maybe I'm misreading that post. I'm not trying to cherry pick posts
> or play word games; I just
> feel that the loss of the possibility of a single codebase on <=2.5
> and 3.0 should not be given
> up lightly.

It's not being done lightly.  But when something as obvious as moving
the traceback to exceptions presents itself and it means that another
way becomes antiquated we should take the chance to ditch the old way
so we don't have this weird situation of supporting both and old way
we discourage and a new way that we are pushing.

I understand the worry, Tony, but this has been argued on python-dev
and this worry has been made abundantly clear.  But the decision has
been made that 2.6 will be the first version that can run both 2.5
code and a good chunk of 3.0 code and that is explicitly so that 3.0
does not have to worry about 2.5 compatibility.

Anyway, this discussion of transitioning has/is gone on python-dev and
I am getting burned out on it, so I am just going to stay out of the
rest of this thread (nothing personal, Tony).

-Brett

From mhammond at skippinet.com.au  Mon Jan 22 07:56:33 2007
From: mhammond at skippinet.com.au (Mark Hammond)
Date: Mon, 22 Jan 2007 17:56:33 +1100
Subject: [Python-3000] Exceptions internals and removing sys.exc_*
In-Reply-To: <43aa6ff70701202146q3cd34739u3a41c0b78f2a23b9@mail.gmail.com>
Message-ID: <046401c73df2$74036cf0$0200a8c0@enfoldsystems.local>

> Guido has mentioned [1] that since exceptions will be growing a
> __traceback__ attribute in Python 3, it should be possible to remove
> sys.exc_info().

sys.exc_info() is also useful for returning the exception itself, not only
the traceback.  The traceback and logger modules both take advantage of it
to avoid the pain of passing exception objects around.  eg:

  try:
   ...
  except Foo:
    logger.exception("something bad happened")

is better than what may otherwise be necessary:

  except Foo, f:
    logger.exception("something bad happened", exc_info=f)

It seems a reasonable use case to have code that knows it is dealing with
exceptions, but not always be directly in the exception handler.

Cheers,

Mark


From pje at telecommunity.com  Mon Jan 22 16:25:25 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 22 Jan 2007 10:25:25 -0500
Subject: [Python-3000] Exceptions internals and removing sys.exc_*
In-Reply-To: <bbaeab100701211843o672c927bo6e55551a75e16c15@mail.gmail.co
 m>
References: <1AC1C3B8-7364-4151-9FED-DC03FA867521@pagedna.com>
	<43aa6ff70701202146q3cd34739u3a41c0b78f2a23b9@mail.gmail.com>
	<A68BA56D-27F3-4F10-BA26-43057383F1A4@PageDNA.com>
	<bbaeab100701211057m5c9b9cem30d7379be98138e1@mail.gmail.com>
	<1AC1C3B8-7364-4151-9FED-DC03FA867521@pagedna.com>
Message-ID: <5.1.1.6.0.20070122101919.06555fc8@sparrow.telecommunity.com>

At 06:43 PM 1/21/2007 -0800, Brett Cannon wrote:
>But when something as obvious as moving
>the traceback to exceptions presents itself and it means that another
>way becomes antiquated we should take the chance to ditch the old way
>so we don't have this weird situation of supporting both and old way
>we discourage and a new way that we are pushing.

How will this affect the __exit__ signature for context managers, which is 
defined in sys.exc_info() terms?  Also, WSGI defines certain 
exception-handling parameters in terms of sys.exc_info().

In fact, many APIs define things in terms of sys.exc_info() tuples.  That 
doesn't mean they have to come from sys.exc_info(), of course, but these 
APIs are defined that way based on a standard feature of the langauge.  So, 
documenting them may be more awkward.

Will we also be dropping three-argument "raise"?


From jimjjewett at gmail.com  Mon Jan 22 16:44:17 2007
From: jimjjewett at gmail.com (Jim Jewett)
Date: Mon, 22 Jan 2007 10:44:17 -0500
Subject: [Python-3000] Exceptions internals and removing sys.exc_*
In-Reply-To: <5.1.1.6.0.20070122101919.06555fc8@sparrow.telecommunity.com>
References: <43aa6ff70701202146q3cd34739u3a41c0b78f2a23b9@mail.gmail.com>
	<A68BA56D-27F3-4F10-BA26-43057383F1A4@PageDNA.com>
	<bbaeab100701211057m5c9b9cem30d7379be98138e1@mail.gmail.com>
	<1AC1C3B8-7364-4151-9FED-DC03FA867521@pagedna.com>
	<5.1.1.6.0.20070122101919.06555fc8@sparrow.telecommunity.com>
Message-ID: <fb6fbf560701220744m1640f732v92f797169294a419@mail.gmail.com>

On 1/22/07, Phillip J. Eby <pje at telecommunity.com> wrote:
> At 06:43 PM 1/21/2007 -0800, Brett Cannon wrote:
> >But when something as obvious as moving
> >the traceback to exceptions ... should take the chance to ditch the old way

> How will this affect the __exit__ signature for context managers, which is
> defined in sys.exc_info() terms?  Also, WSGI defines certain
> exception-handling parameters in terms of sys.exc_info().

They can refer to python 2.x, which will still have it.
They can be explicit, and mention a 3-tuple of exception (type, value,
traceback).

They should grow an alternate interface that just takes an exception instance.

> Will we also be dropping three-argument "raise"?

Is there any reason not to drop it?

Even in py2.x, the only reasons I see not to use the single-argument form

    >>> raise Exception("args")

are that (1)  you want to support string exceptions, or (2)  you want
to change the traceback, which could now be done with the traceback
(context/cause) attributes.

-jJ

From pje at telecommunity.com  Mon Jan 22 17:11:03 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 22 Jan 2007 11:11:03 -0500
Subject: [Python-3000] Exceptions internals and removing sys.exc_*
In-Reply-To: <fb6fbf560701220744m1640f732v92f797169294a419@mail.gmail.co
 m>
References: <5.1.1.6.0.20070122101919.06555fc8@sparrow.telecommunity.com>
	<43aa6ff70701202146q3cd34739u3a41c0b78f2a23b9@mail.gmail.com>
	<A68BA56D-27F3-4F10-BA26-43057383F1A4@PageDNA.com>
	<bbaeab100701211057m5c9b9cem30d7379be98138e1@mail.gmail.com>
	<1AC1C3B8-7364-4151-9FED-DC03FA867521@pagedna.com>
	<5.1.1.6.0.20070122101919.06555fc8@sparrow.telecommunity.com>
Message-ID: <5.1.1.6.0.20070122105624.0656d7f0@sparrow.telecommunity.com>

At 10:44 AM 1/22/2007 -0500, Jim Jewett wrote:
>On 1/22/07, Phillip J. Eby <pje at telecommunity.com> wrote:
>>At 06:43 PM 1/21/2007 -0800, Brett Cannon wrote:
>> >But when something as obvious as moving
>> >the traceback to exceptions ... should take the chance to ditch the old way
>
>>How will this affect the __exit__ signature for context managers, which is
>>defined in sys.exc_info() terms?  Also, WSGI defines certain
>>exception-handling parameters in terms of sys.exc_info().
>
>They can refer to python 2.x, which will still have it.
>They can be explicit, and mention a 3-tuple of exception (type, value,
>traceback).
>
>They should grow an alternate interface that just takes an exception instance.

Changing __exit__ would be a language change, and presumably requires a new 
or updated PEP, as would a change to WSGI.  Likewise the changes to the 
logging API.


>>Will we also be dropping three-argument "raise"?
>
>Is there any reason not to drop it?

How would you perform a three-argument raise in Python 2.6?  We can't add 
the traceback to exception objects in 2.6 without creating dangerous 
traceback reference cycles for 2.x code, at least not without some other 
sort of as-yet-undetermined magic, like temporarily removing and re-adding 
the traceback attribute to exceptions caught in old-style except: clauses.

I guess my point is that if we're going down this road, we need to figure 
out how we're going to add comparable features to 2.6.

I suppose, we could have old-style "except t, e:" clauses set the exception 
object's traceback attribute to None, so that old code would never see 
it.  Likewise, using sys.exc_info() or any other old-style exception access 
could do the same thing.

Somebody needs to come up with an idea for how to handle 
__exit__().  Currently, it requires all three arguments, but it sounds like 
you suggest it change to just one in Py3K.  I guess that 2.6 code could set 
default values of None for the value and traceback arguments, and 2.6 would 
still pass those values in.  The body of the __exit__ method would have to 
check on those, to avoid passing them to APIs that don't take them in 3.0.

(By the way, we *still* need a way to get the current active exception, 
regardless of how the API is spelled.  See the contextlib module's 
generator decorator for one example of why.)


From brett at python.org  Mon Jan 22 19:14:33 2007
From: brett at python.org (Brett Cannon)
Date: Mon, 22 Jan 2007 10:14:33 -0800
Subject: [Python-3000] Exceptions internals and removing sys.exc_*
In-Reply-To: <fb6fbf560701220744m1640f732v92f797169294a419@mail.gmail.com>
References: <43aa6ff70701202146q3cd34739u3a41c0b78f2a23b9@mail.gmail.com>
	<A68BA56D-27F3-4F10-BA26-43057383F1A4@PageDNA.com>
	<bbaeab100701211057m5c9b9cem30d7379be98138e1@mail.gmail.com>
	<1AC1C3B8-7364-4151-9FED-DC03FA867521@pagedna.com>
	<5.1.1.6.0.20070122101919.06555fc8@sparrow.telecommunity.com>
	<fb6fbf560701220744m1640f732v92f797169294a419@mail.gmail.com>
Message-ID: <bbaeab100701221014s49786021r8c099ace01c0230@mail.gmail.com>

On 1/22/07, Jim Jewett <jimjjewett at gmail.com> wrote:
> On 1/22/07, Phillip J. Eby <pje at telecommunity.com> wrote:
> > At 06:43 PM 1/21/2007 -0800, Brett Cannon wrote:
> > >But when something as obvious as moving
> > >the traceback to exceptions ... should take the chance to ditch the old way
>
> > How will this affect the __exit__ signature for context managers, which is
> > defined in sys.exc_info() terms?  Also, WSGI defines certain
> > exception-handling parameters in terms of sys.exc_info().
>
> They can refer to python 2.x, which will still have it.
> They can be explicit, and mention a 3-tuple of exception (type, value,
> traceback).
>
> They should grow an alternate interface that just takes an exception instance.
>

The other way would be to add a method to exceptions that return the
three objects.

> > Will we also be dropping three-argument "raise"?
>
> Is there any reason not to drop it?
>
> Even in py2.x, the only reasons I see not to use the single-argument form
>
>     >>> raise Exception("args")
>
> are that (1)  you want to support string exceptions, or (2)  you want
> to change the traceback, which could now be done with the traceback
> (context/cause) attributes.
>

Especially since as of right now the syntactic support for
three-argument 'raise' statements is slated to go away and
BaseException currently does not accept a traceback argument.

-Brett

From collinw at gmail.com  Mon Jan 22 22:10:30 2007
From: collinw at gmail.com (Collin Winter)
Date: Mon, 22 Jan 2007 15:10:30 -0600
Subject: [Python-3000] Exceptions internals and removing sys.exc_*
In-Reply-To: <5.1.1.6.0.20070122101919.06555fc8@sparrow.telecommunity.com>
References: <43aa6ff70701202146q3cd34739u3a41c0b78f2a23b9@mail.gmail.com>
	<A68BA56D-27F3-4F10-BA26-43057383F1A4@PageDNA.com>
	<bbaeab100701211057m5c9b9cem30d7379be98138e1@mail.gmail.com>
	<1AC1C3B8-7364-4151-9FED-DC03FA867521@pagedna.com>
	<5.1.1.6.0.20070122101919.06555fc8@sparrow.telecommunity.com>
Message-ID: <43aa6ff70701221310y5b3d4899sa230a62238ffddc5@mail.gmail.com>

On 1/22/07, Phillip J. Eby <pje at telecommunity.com> wrote:
> At 06:43 PM 1/21/2007 -0800, Brett Cannon wrote:
> >But when something as obvious as moving
> >the traceback to exceptions presents itself and it means that another
> >way becomes antiquated we should take the chance to ditch the old way
> >so we don't have this weird situation of supporting both and old way
> >we discourage and a new way that we are pushing.
>
> How will this affect the __exit__ signature for context managers, which is
> defined in sys.exc_info() terms?  Also, WSGI defines certain
> exception-handling parameters in terms of sys.exc_info().
>
> In fact, many APIs define things in terms of sys.exc_info() tuples.  That
> doesn't mean they have to come from sys.exc_info(), of course, but these
> APIs are defined that way based on a standard feature of the langauge.  So,
> documenting them may be more awkward.

Is there a reason why these API could not be defined in terms of a
single exception object? In Python 3, all the information from a
sys.exc_info() tuple  available from an exception instance: (type(e),
e, e.__traceback__)

As for documentation issues, I would think changing from a
sys.exc_info() tuple to a single exception object would improve the
language. Let's try a passage from PEP 333:

"""
The exc_info argument, if supplied, must be a Python sys.exc_info()
tuple. This argument should be supplied by the application only if
start_response is being called by an error handler. If exc_info is
supplied, and no HTTP headers have been output yet, start_response
should replace the currently-stored HTTP response headers with the
newly-supplied ones, thus allowing the application to "change its
mind" about the output when an error has occurred.

However, if exc_info is provided, and the HTTP headers have already
been sent, start_response must raise an error, and should raise the
exc_info tuple. That is:

raise exc_info[0],exc_info[1],exc_info[2]
"""

changes to

"""
The exc argument, if supplied, must be a Python exception instance.
This argument should be supplied by the application only if
start_response is being called by an error handler. If exc is
supplied, and no HTTP headers have been output yet, start_response
should replace the currently-stored HTTP response headers with the
newly-supplied ones, thus allowing the application to "change its
mind" about the output when an error has occurred.

However, if exc is provided, and the HTTP headers have already been
sent, start_response must raise an error, and should raise the exc
object. That is:

raise exc
"""

> Will we also be dropping three-argument "raise"?

Yes. I have a new version of 2to3's raise fixer that is able to handle
both the two- and three-argument forms of "raise".

Thanks,
Collin Winter

From tomerfiliba at gmail.com  Mon Jan 22 22:54:51 2007
From: tomerfiliba at gmail.com (tomer filiba)
Date: Mon, 22 Jan 2007 23:54:51 +0200
Subject: [Python-3000] simplifying the exception mechanism
Message-ID: <1d85506f0701221354u7247ebch918f5194377445f4@mail.gmail.com>

with all the talk about enhancing exceptions and removing unnecessary
functions from the codebase, i think we can also drop
PyErr_NormalizeException and friends.

after all, since string exceptions are out, BDFL has mentioned
he prefered "raise Foo()" instead of "raise Foo" [1], and since the
traceback will be stored inside the exception *instance* --
there's no more need for normalization.

we can either drop the "raise TYPE" version for "raise TYPE()",
or have the raise opcode instantiate TYPE internally [2].

personally i like omitting the parenthesis when there's no
arguments to be passed, i.e. "raise EOFError",
but i could live with typing two extra characters.


-tomer

[1] pep 317 and
http://mail.python.org/pipermail/python-dev/2005-August/055665.html

[2] on a side note, java et al use "throw new FooException();".
i know this doesn't mean much, but still, exceptions are
ultimately instances.

From collinw at gmail.com  Mon Jan 22 23:26:28 2007
From: collinw at gmail.com (Collin Winter)
Date: Mon, 22 Jan 2007 16:26:28 -0600
Subject: [Python-3000] simplifying the exception mechanism
In-Reply-To: <1d85506f0701221354u7247ebch918f5194377445f4@mail.gmail.com>
References: <1d85506f0701221354u7247ebch918f5194377445f4@mail.gmail.com>
Message-ID: <43aa6ff70701221426w2f12413dtc2516fa2e52be2c6@mail.gmail.com>

On 1/22/07, tomer filiba <tomerfiliba at gmail.com> wrote:
> with all the talk about enhancing exceptions and removing unnecessary
> functions from the codebase, i think we can also drop
> PyErr_NormalizeException and friends.

One argument in favor of keeping PyErr_NormalizeException() (as I've
learned from trying to rip it out) is the speed advantage it brings.
It's less expensive to sock away the exception class and arguments,
only instancing the exception when you really need it. This is
important in certain core types, where light(er)-weight exception
raising comes in handy.

Collin Winter

From collinw at gmail.com  Mon Jan 22 23:45:01 2007
From: collinw at gmail.com (Collin Winter)
Date: Mon, 22 Jan 2007 16:45:01 -0600
Subject: [Python-3000] Pre-peps on raise and except changes (was: Warning
	for 2.6 and greater)
Message-ID: <43aa6ff70701221445s5edb4b2do8c8cffdebc759c7@mail.gmail.com>

On 1/12/07, Steven Bethard <steven.bethard at gmail.com> wrote:
> On 1/12/07, Barry Warsaw <barry at python.org> wrote:
> > It's worth spending time thinking about how we can help ease the
> > transition for each.  There may be more thing too.  I wonder if it
> > doesn't make sense for all 3xxx PEPs to include a discussion section
> > on porting.
>
> Yeah, when this was talked about last time, I wrote PEP 3002 which
> requests exactly this:
>
>     http://www.python.org/dev/peps/pep-3002/
>
> It basically proposes that all backwards-incompatible changes be
> discussed in a PEP somewhere, and that code like Anthony's be added to
> 2.X to ease the transition.

Attached are drafts of the PEP 3002-compliant PEPs concerning the
changes to raise and except statements that have been discussed on
python-3000.

Any suggestions/corrections/additions would be much appreciated.

Thanks,
Collin Winter
-------------- next part --------------
PEP: 310?
Title: Except Statements in Python 3000
Version: $Revision$
Last-Modified: $Date$
Author: Collin Winter <collinw at gmail.com>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 16-Jan-2006
Python-Version: 3.0
Post-History:


Abstract
========

This PEP introduces a changes to ``except`` statements in Python 3.0
intended to help reduce ambiguities in Python's grammar, simplify
exception classes and simplify garbage collection for exceptions.


Rationale
=========

``except`` clauses in Python 2.x present a syntactic ambiguity where
the parser cannot differentiate whether ::

    except <expression>, <expression>:
    
should be interpreted as ::

    except <type>, <type>:
    
or ::

    except <type>, <name>:

Python 2 opts for the latter semantic, at the cost of requiring the
former to be parenthesized, like so ::

    except (<type>, <type>):

In addition, Python 3 will introduce a further incompatibility. As
specified in PEP 352 [#pep352]_, the ability to treat exceptions as
tuples will be removed, meaning this code will no longer work ::

    except os.error, (errno, errstr):
    
Because the automatic unpacking will no longer be possible, it is
desirable to remove the ability to use tuples as ``except`` targets.
Both of these issues will be resolved by changing Python's grammar.

Lastly, as specified in PEP 344 [#pep344]_, Python 3 exceptions will
possess a ``__traceback__`` attribute. The Open Issues section of that
PEP includes a paragraph on garbage collection difficulties caused by
this attribute, such as a ``exception -> traceback -> stack frame
-> exception'' reference cycle, keeping all locals in scope until the
next GC run. This PEP intends to resolve this issue by adding a
cleanup semantic to ``except`` clauses of the form ``except E as N``.


Grammar Changes
===============
    
In Python 3, the grammar for ``except`` statements will change
from [#grammar]_ ::

    except_clause: 'except' [test [',' test]]
    
to ::

    except_clause: 'except' [test ['as' NAME]]

The use of ``as`` in place of the comma token means that ::

    except AttributeError, os.error:
    
can be clearly understood as a tuple of exception classes. This new
syntax was first proposed by Greg Ewing [#firstproposal]_ and
endorsed ([#firstproposal]_, [#renaming]_) by the BDFL.

Further, the restriction of the token following ``as`` from ``test``
to ``NAME`` means that only valid identifiers can be used as
``except`` targets.


Semantic Changes
================

In order to resolve the garbage collection issue related to PEP 344,
``except`` statements will generate additional bytecode to delete the
target, thus eliminating the reference cycle. The source-to-source
translation, as suggested by Phillip J. Eby [#except-translation]_ is
::

    try:
        ...
    except E as N:
        ...
    ...
    
is translated to ::

    try:
        ...
    except E as N:
        ...
        N = None
        del N
    ...

An implementation has already been checked into the p3yk branch
[#translation-checkin]_.


Compatibility Issues
====================

Nearly all ``except`` clauses will need to be changed. ``except``
clauses with identifier targets will be converted from ::

    except E, N:
    
to ::
    
    except E as N:

``except`` clauses with non-tuple, non-identifier targets
(e.g., ``a.b.c[d]``) will need to be converted from ::

    except E, T:
    
to ::

    except E as t:
        T = t
        
Both of these cases can be handled by Guido van Rossum's ``2to3``
utility [#2to3]_ using the ``except`` fixer [#exceptfixer]_.
        
``except`` clauses with tuple targets will need to be converted
manually, on a case-by-case basis. These changes will usually need
to be accompanied by changes to the exception classes themselves.
While these changes generally cannot be automated, the ``2to3``
utility is able to point out cases where the target of an ``except``
clause is a tuple, simplifying conversion.

Situations where it is necessary to keep an exception instance around
past the end of the ``except`` suite can be easily translated like so
::

    try:
        ...
    except E as N:
        ...
    ...
    
is translated to ::

    try:
        ...
    except E as N:
        n = N
        ...
    ...
    
This way, when ``N`` is deleted at the end of the block, ``n`` will
persist and can be used as normal.


References
==========

.. [#pep352]
   http://www.python.org/dev/peps/pep-0352/
   
.. [#pep344]
   http://www.python.org/dev/peps/pep-0344/

.. [#firstproposal]
   http://mail.python.org/pipermail/python-dev/2006-March/062449.html

.. [#renaming]
   http://mail.python.org/pipermail/python-dev/2006-March/062640.html
   
.. [#grammar]
   http://www.python.org/doc/current/ref/try.html
   
.. [#except-translation]
   http://mail.python.org/pipermail/python-3000/2007-January/005395.html
   
.. [#translation-checkin]
   http://svn.python.org/view?rev=53342&view=rev
   
.. [#2to3]
   http://svn.python.org/view/sandbox/trunk/2to3/
   
.. [#exceptfixer]
   http://svn.python.org/view/sandbox/trunk/2to3/fixes/fix_except.py


Copyright
=========

This document has been placed in the public domain.



..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 70
   coding: utf-8
   End:
-------------- next part --------------
PEP: 310?
Title: Raising Exceptions in Python 3000
Version: $Revision$
Last-Modified: $Date$
Author: Collin Winter <collinw at gmail.com>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 19-Jan-2006
Python-Version: 3.0
Post-History:


Abstract
========

This PEP introduces a changes to exception raising mechanisms in
Python 3.0 intended to reduce both line noise and the size of the
language.


Rationale
=========

One of Python's guiding maxims is "There should be one -- and
preferably only one -- obvious way to do it" [#zen]_. Python 2.x's
``raise`` statement violates this principle, permitting multiple
ways of expressing the same thought. For example, these statements
are equivalent: ::
    
    raise E, V
    
    raise E(V)
    
There is a third form of the ``raise`` statement, allowing arbitrary
tracebacks to be attached to an exception [#grammar]_: ::

    raise E, V, T
    
where T is a traceback. Note that attaching the traceback in this
manner *requires* use of the ``raise E, V`` form. As specified in
PEP 344 [#pep344]_, exception objects in Python 3.x will possess
a ``__traceback__`` attribute, admitting this translation of the
three-expression ``raise`` statement: ::

    raise E, V, T
    
is translated to ::

    e = E(V)
    e.__traceback__ = T
    raise e
    
The ease of this translation allows a further simplification of the
language by removing the need for the ``raise E, V, T`` form.

There is a further, more tangible benefit to be obtained through this
consildation, as noted by A.M. Kuchling [#amk-line-noise]_. ::

    PEP 8 doesn't express any preference between the 
    two forms of raise statements:
    raise ValueError, 'blah'
    raise ValueError("blah")

    I like the second form better, because if the exception arguments
    are long or include string formatting, you don't need to use line
    continuation characters because of the containing parens.
    
The BDFL has concurred [#guido-declaration]_ and endorsed the
consolidation of the several ``raise`` forms.
    

Grammar Changes
===============
    
In Python 3, the grammar for ``raise`` statements will change
from [#grammar]_ ::

    raise_stmt: 'raise' [test [',' test [',' test]]]
    
to ::

    raise_stmt: 'raise' [test]

    
Other Changes
=============
    
Because of its relation to exception raising, the signature for the
``throw()`` method on generator objects will change, dropping the
optional second and third parameters. The signature thus changes 
from [#throw-sig]_ ::

    generator.throw(E, [T, [V]])
    
to ::

    generator.throw(E)
    

Compatibility Issues
====================

All two- and three-expression ``raise`` statements will require
modification, as will all two- and three-expression ``throw()`` calls
on generators. Fortunately, the translation from Python 2.x to
Python 3.x in this case is simple and can be handled mechanically
by Guido van Rossum's 2to3 utility [#2to3]_ using the ``raise`` and
``throw`` fixers ([#raise-fixer]_, [#throw-fixer]_).

The following translations will be performed:

1. Zero- and one-expression ``raise`` statements will be left
   intact.
   
2. Two-expression ``raise`` statements will be converted from ::
        
        raise E, V
        
   to ::
   
        raise E(V)
        
   Two-expression ``throw()`` calls will be converted from ::
        
        generator.throw(E, V)
        
   to ::
   
        generator.throw(E(V))
        
3. Three-expression ``raise`` statements will be converted from ::

        raise E, V, T
        
   to ::
   
        e = E(V)
        e.__traceback__ = T
        raise e
        
   Three-expression ``throw()`` calls will be converted from ::
   
        generator.throw(E, V, T)
        
   to ::
   
        e = E(V)
        e.__traceback__ = T
        generator.throw(e)


References
==========

.. [#zen]
   http://www.python.org/dev/peps/pep-0020/
   
.. [#grammar]
   http://www.python.org/doc/current/ref/raise.html
   
.. [#throw-sig]
   http://www.python.org/dev/peps/pep-0342/
   
.. [#pep344]
   http://www.python.org/dev/peps/pep-0344/
   
.. [#amk-line-noise]
   http://mail.python.org/pipermail/python-dev/2005-August/055187.html

.. [#guido-declaration]
   http://mail.python.org/pipermail/python-dev/2005-August/055190.html
   
.. [#2to3]
   http://svn.python.org/view/sandbox/trunk/2to3/
   
.. [#raise-fixer]
   http://svn.python.org/view/sandbox/trunk/2to3/fixes/fix_raise.py
   
.. [#throw-fixer]
   http://svn.python.org/view/sandbox/trunk/2to3/fixes/fix_throw.py


Copyright
=========

This document has been placed in the public domain.



..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 70
   coding: utf-8
   End:

From brett at python.org  Mon Jan 22 23:56:19 2007
From: brett at python.org (Brett Cannon)
Date: Mon, 22 Jan 2007 14:56:19 -0800
Subject: [Python-3000] simplifying the exception mechanism
In-Reply-To: <43aa6ff70701221426w2f12413dtc2516fa2e52be2c6@mail.gmail.com>
References: <1d85506f0701221354u7247ebch918f5194377445f4@mail.gmail.com>
	<43aa6ff70701221426w2f12413dtc2516fa2e52be2c6@mail.gmail.com>
Message-ID: <bbaeab100701221456l4f64a20t41d7a34f451ee7ff@mail.gmail.com>

On 1/22/07, Collin Winter <collinw at gmail.com> wrote:
> On 1/22/07, tomer filiba <tomerfiliba at gmail.com> wrote:
> > with all the talk about enhancing exceptions and removing unnecessary
> > functions from the codebase, i think we can also drop
> > PyErr_NormalizeException and friends.
>
> One argument in favor of keeping PyErr_NormalizeException() (as I've
> learned from trying to rip it out) is the speed advantage it brings.
> It's less expensive to sock away the exception class and arguments,
> only instancing the exception when you really need it. This is
> important in certain core types, where light(er)-weight exception
> raising comes in handy.
>

I'm with Collin on this one.  The speed argument is one of the reasons
PyErr_NormalizeException even exists.  We want exceptions in the
simple, general case to be as fast as possible.

-Brett

From brett at python.org  Tue Jan 23 00:08:44 2007
From: brett at python.org (Brett Cannon)
Date: Mon, 22 Jan 2007 15:08:44 -0800
Subject: [Python-3000] Pre-peps on raise and except changes (was:
	Warning for 2.6 and greater)
In-Reply-To: <43aa6ff70701221445s5edb4b2do8c8cffdebc759c7@mail.gmail.com>
References: <43aa6ff70701221445s5edb4b2do8c8cffdebc759c7@mail.gmail.com>
Message-ID: <bbaeab100701221508r4a90df17p9d17eee821838988@mail.gmail.com>

On 1/22/07, Collin Winter <collinw at gmail.com> wrote:
> On 1/12/07, Steven Bethard <steven.bethard at gmail.com> wrote:
> > On 1/12/07, Barry Warsaw <barry at python.org> wrote:
> > > It's worth spending time thinking about how we can help ease the
> > > transition for each.  There may be more thing too.  I wonder if it
> > > doesn't make sense for all 3xxx PEPs to include a discussion section
> > > on porting.
> >
> > Yeah, when this was talked about last time, I wrote PEP 3002 which
> > requests exactly this:
> >
> >     http://www.python.org/dev/peps/pep-3002/
> >
> > It basically proposes that all backwards-incompatible changes be
> > discussed in a PEP somewhere, and that code like Anthony's be added to
> > 2.X to ease the transition.
>
> Attached are drafts of the PEP 3002-compliant PEPs concerning the
> changes to raise and except statements that have been discussed on
> python-3000.
>
> Any suggestions/corrections/additions would be much appreciated.
>

For the exception PEP:

"In addition, Python 3 will introduce a further incompatibility."
You have not even mentioned a Py3K incompatibility yet so this
statement is awkward.

"This PEP intends to resolve this issue by adding a
cleanup semantic to ``except`` clauses of the form ``except E as N``".
This sentence is in reference to the cycle created with exceptions
holding a reference to the traceback.  Problem is the above gives
absolutely no indication how a syntactic change is going to help with
cyclic garbage collection.

---------

For the raise PEP:

"consildation"
It's "consolidation".

"The ease of this translation allows a further simplification of the
language by removing the need for the ``raise E, V, T`` form.".
At this point in the Rationale you have not explicitly stated that
``raise E(V)`` is going to be the only way to raise an exception.

You might want to also want to mention if you are or are not going to
remove sys.exc_info .  If you are then you need to state that and
mention that all places where three arguments for exceptions is
expected will need to be changed.  Might also not hurt to reference
any other exception information functions that are slated to go in PEP
3100.

--------

Thanks for doing this, Collin!

-Brett

From collinw at gmail.com  Tue Jan 23 00:27:16 2007
From: collinw at gmail.com (Collin Winter)
Date: Mon, 22 Jan 2007 17:27:16 -0600
Subject: [Python-3000] Pre-peps on raise and except changes (was:
	Warning for 2.6 and greater)
In-Reply-To: <bbaeab100701221508r4a90df17p9d17eee821838988@mail.gmail.com>
References: <43aa6ff70701221445s5edb4b2do8c8cffdebc759c7@mail.gmail.com>
	<bbaeab100701221508r4a90df17p9d17eee821838988@mail.gmail.com>
Message-ID: <43aa6ff70701221527o2616d6f0g363170bcf44bba36@mail.gmail.com>

On 1/22/07, Brett Cannon <brett at python.org> wrote:
> For the exception PEP:
>
> "In addition, Python 3 will introduce a further incompatibility."
> You have not even mentioned a Py3K incompatibility yet so this
> statement is awkward.

This sentence has been cut.

> "This PEP intends to resolve this issue by adding a
> cleanup semantic to ``except`` clauses of the form ``except E as N``".
> This sentence is in reference to the cycle created with exceptions
> holding a reference to the traceback.  Problem is the above gives
> absolutely no indication how a syntactic change is going to help with
> cyclic garbage collection.

I've cut the reference to the new syntax. The sentence now reads "This
PEP intends to resolve this issue by adding a cleanup semantic to
``except`` clauses in Python 3 where the target name is deleted at the
end of the ``except`` suite."

> For the raise PEP:
>
> "The ease of this translation allows a further simplification of the
> language by removing the need for the ``raise E, V, T`` form.".
> At this point in the Rationale you have not explicitly stated that
> ``raise E(V)`` is going to be the only way to raise an exception.

I've cut that sentence and added this in its place:

"""
Using these translations, we can reduce the ``raise`` statement from
four forms to two:

1. ``raise`` (with no arguments) is used to re-raise the active
   exception in an ``except`` suite.

2. ``raise E`` (with a single argument) is used to raise a new
   exception.
"""

> You might want to also want to mention if you are or are not going to
> remove sys.exc_info .  If you are then you need to state that and
> mention that all places where three arguments for exceptions is
> expected will need to be changed.  Might also not hurt to reference
> any other exception information functions that are slated to go in PEP
> 3100.

I'd say that based on the number of places that will have to change
(code, documentation), dropping sys.exc_info() and friends should
probably be its own PEP. We may even want to reconsider dropping
sys.exc_info() (though sys.exc_{type, value, traceback} should go)
given the difficulty of porting.

> Thanks for doing this, Collin!

Thanks for the feedback! The latest drafts are attached.

Collin Winter
-------------- next part --------------
PEP: 310?
Title: Except Statements in Python 3000
Version: $Revision$
Last-Modified: $Date$
Author: Collin Winter <collinw at gmail.com>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 16-Jan-2006
Python-Version: 3.0
Post-History:


Abstract
========

This PEP introduces changes to ``except`` statements in Python 3.0
intended to help reduce ambiguities in Python's grammar, simplify
exception classes and simplify garbage collection for exceptions.


Rationale
=========

``except`` clauses in Python 2.x present a syntactic ambiguity where
the parser cannot differentiate whether ::

    except <expression>, <expression>:
    
should be interpreted as ::

    except <type>, <type>:
    
or ::

    except <type>, <name>:

Python 2 opts for the latter semantic, at the cost of requiring the
former to be parenthesized, like so ::

    except (<type>, <type>):

As specified in PEP 352 [#pep352]_, the ability to treat exceptions as
tuples will be removed, meaning this code will no longer work ::

    except os.error, (errno, errstr):
    
Because the automatic unpacking will no longer be possible, it is
desirable to remove the ability to use tuples as ``except`` targets.
Both of these issues will be resolved by changing Python's grammar.

Lastly, as specified in PEP 344 [#pep344]_, Python 3 exceptions will
possess a ``__traceback__`` attribute. The Open Issues section of that
PEP includes a paragraph on garbage collection difficulties caused by
this attribute, such as a ``exception -> traceback -> stack frame
-> exception'' reference cycle, keeping all locals in scope until the
next GC run. This PEP intends to resolve this issue by adding a
cleanup semantic to ``except`` clauses in Python 3 where the target
name is deleted at the end of the ``except`` suite.


Grammar Changes
===============
    
In Python 3, the grammar for ``except`` statements will change
from [#grammar]_ ::

    except_clause: 'except' [test [',' test]]
    
to ::

    except_clause: 'except' [test ['as' NAME]]

The use of ``as`` in place of the comma token means that ::

    except AttributeError, os.error:
    
can be clearly understood as a tuple of exception classes. This new
syntax was first proposed by Greg Ewing [#firstproposal]_ and
endorsed ([#firstproposal]_, [#renaming]_) by the BDFL.

Further, the restriction of the token following ``as`` from ``test``
to ``NAME`` means that only valid identifiers can be used as
``except`` targets.


Semantic Changes
================

In order to resolve the garbage collection issue related to PEP 344,
``except`` statements will generate additional bytecode to delete the
target, thus eliminating the reference cycle. The source-to-source
translation, as suggested by Phillip J. Eby [#except-translation]_ is
::

    try:
        ...
    except E as N:
        ...
    ...
    
is translated to ::

    try:
        ...
    except E as N:
        ...
        N = None
        del N
    ...

An implementation has already been checked into the p3yk branch
[#translation-checkin]_.


Compatibility Issues
====================

Nearly all ``except`` clauses will need to be changed. ``except``
clauses with identifier targets will be converted from ::

    except E, N:
    
to ::
    
    except E as N:

``except`` clauses with non-tuple, non-identifier targets
(e.g., ``a.b.c[d]``) will need to be converted from ::

    except E, T:
    
to ::

    except E as t:
        T = t
        
Both of these cases can be handled by Guido van Rossum's ``2to3``
utility [#2to3]_ using the ``except`` fixer [#exceptfixer]_.
        
``except`` clauses with tuple targets will need to be converted
manually, on a case-by-case basis. These changes will usually need
to be accompanied by changes to the exception classes themselves.
While these changes generally cannot be automated, the ``2to3``
utility is able to point out cases where the target of an ``except``
clause is a tuple, simplifying conversion.

Situations where it is necessary to keep an exception instance around
past the end of the ``except`` suite can be easily translated like so
::

    try:
        ...
    except E as N:
        ...
    ...
    
is translated to ::

    try:
        ...
    except E as N:
        n = N
        ...
    ...
    
This way, when ``N`` is deleted at the end of the block, ``n`` will
persist and can be used as normal.


References
==========

.. [#pep352]
   http://www.python.org/dev/peps/pep-0352/
   
.. [#pep344]
   http://www.python.org/dev/peps/pep-0344/

.. [#firstproposal]
   http://mail.python.org/pipermail/python-dev/2006-March/062449.html

.. [#renaming]
   http://mail.python.org/pipermail/python-dev/2006-March/062640.html
   
.. [#grammar]
   http://www.python.org/doc/current/ref/try.html
   
.. [#except-translation]
   http://mail.python.org/pipermail/python-3000/2007-January/005395.html
   
.. [#translation-checkin]
   http://svn.python.org/view?rev=53342&view=rev
   
.. [#2to3]
   http://svn.python.org/view/sandbox/trunk/2to3/
   
.. [#exceptfixer]
   http://svn.python.org/view/sandbox/trunk/2to3/fixes/fix_except.py


Copyright
=========

This document has been placed in the public domain.



..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 70
   coding: utf-8
   End:
-------------- next part --------------
PEP: 310?
Title: Raising Exceptions in Python 3000
Version: $Revision$
Last-Modified: $Date$
Author: Collin Winter <collinw at gmail.com>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 19-Jan-2006
Python-Version: 3.0
Post-History:


Abstract
========

This PEP introduces a changes to exception raising mechanisms in
Python 3.0 intended to reduce both line noise and the size of the
language.


Rationale
=========

One of Python's guiding maxims is "There should be one -- and
preferably only one -- obvious way to do it" [#zen]_. Python 2.x's
``raise`` statement violates this principle, permitting multiple
ways of expressing the same thought. For example, these statements
are equivalent: ::
    
    raise E, V
    
    raise E(V)
    
There is a third form of the ``raise`` statement, allowing arbitrary
tracebacks to be attached to an exception [#grammar]_: ::

    raise E, V, T
    
where T is a traceback. Note that attaching the traceback in this
manner *requires* use of the ``raise E, V`` form. As specified in
PEP 344 [#pep344]_, exception objects in Python 3.x will possess
a ``__traceback__`` attribute, admitting this translation of the
three-expression ``raise`` statement: ::

    raise E, V, T
    
is translated to ::

    e = E(V)
    e.__traceback__ = T
    raise e
    
Using these translations, we can reduce the ``raise`` statement from
four forms to two:

1. ``raise`` (with no arguments) is used to re-raise the active
   exception in an ``except`` suite.
   
2. ``raise E`` (with a single argument) is used to raise a new
   exception.

There is a further, more tangible benefit to be obtained through this
consolidation, as noted by A.M. Kuchling [#amk-line-noise]_. ::

    PEP 8 doesn't express any preference between the 
    two forms of raise statements:
    raise ValueError, 'blah'
    raise ValueError("blah")

    I like the second form better, because if the exception arguments
    are long or include string formatting, you don't need to use line
    continuation characters because of the containing parens.
    
The BDFL has concurred [#guido-declaration]_ and endorsed the
consolidation of the several ``raise`` forms.
    

Grammar Changes
===============
    
In Python 3, the grammar for ``raise`` statements will change
from [#grammar]_ ::

    raise_stmt: 'raise' [test [',' test [',' test]]]
    
to ::

    raise_stmt: 'raise' [test]

    
Other Changes
=============
    
Because of its relation to exception raising, the signature for the
``throw()`` method on generator objects will change, dropping the
optional second and third parameters. The signature thus changes 
from [#throw-sig]_ ::

    generator.throw(E, [T, [V]])
    
to ::

    generator.throw(E)
    

Compatibility Issues
====================

All two- and three-expression ``raise`` statements will require
modification, as will all two- and three-expression ``throw()`` calls
on generators. Fortunately, the translation from Python 2.x to
Python 3.x in this case is simple and can be handled mechanically
by Guido van Rossum's 2to3 utility [#2to3]_ using the ``raise`` and
``throw`` fixers ([#raise-fixer]_, [#throw-fixer]_).

The following translations will be performed:

1. Zero- and one-expression ``raise`` statements will be left
   intact.
   
2. Two-expression ``raise`` statements will be converted from ::
        
        raise E, V
        
   to ::
   
        raise E(V)
        
   Two-expression ``throw()`` calls will be converted from ::
        
        generator.throw(E, V)
        
   to ::
   
        generator.throw(E(V))
        
3. Three-expression ``raise`` statements will be converted from ::

        raise E, V, T
        
   to ::
   
        e = E(V)
        e.__traceback__ = T
        raise e
        
   Three-expression ``throw()`` calls will be converted from ::
   
        generator.throw(E, V, T)
        
   to ::
   
        e = E(V)
        e.__traceback__ = T
        generator.throw(e)


References
==========

.. [#zen]
   http://www.python.org/dev/peps/pep-0020/
   
.. [#grammar]
   http://www.python.org/doc/current/ref/raise.html
   
.. [#throw-sig]
   http://www.python.org/dev/peps/pep-0342/
   
.. [#pep344]
   http://www.python.org/dev/peps/pep-0344/
   
.. [#amk-line-noise]
   http://mail.python.org/pipermail/python-dev/2005-August/055187.html

.. [#guido-declaration]
   http://mail.python.org/pipermail/python-dev/2005-August/055190.html
   
.. [#2to3]
   http://svn.python.org/view/sandbox/trunk/2to3/
   
.. [#raise-fixer]
   http://svn.python.org/view/sandbox/trunk/2to3/fixes/fix_raise.py
   
.. [#throw-fixer]
   http://svn.python.org/view/sandbox/trunk/2to3/fixes/fix_throw.py


Copyright
=========

This document has been placed in the public domain.



..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 70
   coding: utf-8
   End:

From pje at telecommunity.com  Tue Jan 23 03:44:18 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 22 Jan 2007 21:44:18 -0500
Subject: [Python-3000] Exceptions internals and removing sys.exc_*
In-Reply-To: <43aa6ff70701221310y5b3d4899sa230a62238ffddc5@mail.gmail.co
 m>
References: <5.1.1.6.0.20070122101919.06555fc8@sparrow.telecommunity.com>
	<43aa6ff70701202146q3cd34739u3a41c0b78f2a23b9@mail.gmail.com>
	<A68BA56D-27F3-4F10-BA26-43057383F1A4@PageDNA.com>
	<bbaeab100701211057m5c9b9cem30d7379be98138e1@mail.gmail.com>
	<1AC1C3B8-7364-4151-9FED-DC03FA867521@pagedna.com>
	<5.1.1.6.0.20070122101919.06555fc8@sparrow.telecommunity.com>
Message-ID: <5.1.1.6.0.20070122213225.04252ac8@sparrow.telecommunity.com>

At 03:10 PM 1/22/2007 -0600, Collin Winter wrote:
>Is there a reason why these API could not be defined in terms of a
>single exception object? In Python 3, all the information from a
>sys.exc_info() tuple  available from an exception instance: (type(e),
>e, e.__traceback__)
>
>As for documentation issues, I would think changing from a
>sys.exc_info() tuple to a single exception object would improve the
>language.
[example snipped]

Um, you're not taking 2.6 into consideration here.  Remember that there has 
to be a way to spell all these APIs in 2.6 that will still work in 3.0.  So 
the way the docs would actually have to be rewritten, would be to specify 
how to tell whether an exc_info argument is a tuple or an exception 
instance, and how the code would need to be handled.

You also ignored my point regarding three-argument "raise" in Python 2.6, 
which is that it's (presumably) a SyntaxError in Python 3.0, which means it 
doesn't matter if you do something like:

     if exc.traceback is None:
          raise type(exc), exc, sys.current_traceback()
     else:
          raise exc

The code still won't work in 3.0 because of the mere presence of a 
3-argument raise.

Presumably, the solution will require setting .traceback explicitly and 
using 1-argument raise, resulting in some rather hellish-looking changes 
that are probably not going to be able to be automated by the refactoring 
tool, since it only knows about sys.exc_info() and raise.

I suppose it could rewrite 'raise a, b, c' as 'b.traceback =c; raise b', 
but it still needs to rewrite sys.exc_info() to something, like 
'type(sys.current_exception()), sys.current_exception(), 
sys.current_traceback())'.

IMO, this seems sufficient reason to keep sys.exc_info() around, since 
there *has* to be some way to access the current exception and traceback, 
that exists in *both* 2.6 and 3.0.  Otherwise, there is no way to write 
code that operates on the current exception and/or traceback.

(And again, in 2.6 we are going to have to clear the .traceback of the 
exception when it's caught by an old-style "except", so we can't just have 
a sys.current_exception(); there has to be a way to get the current 
traceback, separately.)


From pje at telecommunity.com  Tue Jan 23 03:53:37 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 22 Jan 2007 21:53:37 -0500
Subject: [Python-3000] Pre-peps on raise and except changes (was:
 Warning for 2.6 and greater)
In-Reply-To: <43aa6ff70701221445s5edb4b2do8c8cffdebc759c7@mail.gmail.com
 >
Message-ID: <5.1.1.6.0.20070122214504.04425a80@sparrow.telecommunity.com>

At 04:45 PM 1/22/2007 -0600, Collin Winter wrote:
>On 1/12/07, Steven Bethard <steven.bethard at gmail.com> wrote:
>>On 1/12/07, Barry Warsaw <barry at python.org> wrote:
>> > It's worth spending time thinking about how we can help ease the
>> > transition for each.  There may be more thing too.  I wonder if it
>> > doesn't make sense for all 3xxx PEPs to include a discussion section
>> > on porting.
>>
>>Yeah, when this was talked about last time, I wrote PEP 3002 which
>>requests exactly this:
>>
>>     http://www.python.org/dev/peps/pep-3002/
>>
>>It basically proposes that all backwards-incompatible changes be
>>discussed in a PEP somewhere, and that code like Anthony's be added to
>>2.X to ease the transition.
>
>Attached are drafts of the PEP 3002-compliant PEPs concerning the
>changes to raise and except statements that have been discussed on
>python-3000.
>
>Any suggestions/corrections/additions would be much appreciated.

The correct translation of a try/except should be:

     try:
         ...
     except E as N:
         try:
             ...
         finally:
             N = None
             del N

i.e., you left out the crucial try/finally wrapper.  If you got this 
translation from the current implementation, then the implementation is 
wrong too.  :)

Also, the PEP should discuss the matter of how "except" should change in 
Python 2.6 to support this, including the current open issue of handling 
the .traceback attribute.

The ``throw()`` signature is incorrectly listed as:

     generator.throw(E, [T, [V]])

It is in fact:

     generator.throw(E, [V, [T]])

It should be noted that this will require special handling in Python 2.6; 
the implementation will need to detect whether the exception instance has a 
traceback, in order to convert to 2.x style exception processing.

The 2-expression "raise" statement translation is incorrect in the general 
case; it is possible for the second argument to be an instance of the first 
argument, in which case 'raise E, V' should become just 'raise V'.  This is 
not detectable by the refactoring tool, AFAIK.


From tjreedy at udel.edu  Tue Jan 23 04:49:48 2007
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 22 Jan 2007 22:49:48 -0500
Subject: [Python-3000] Exceptions internals and removing sys.exc_*
References: <5.1.1.6.0.20070122101919.06555fc8@sparrow.telecommunity.com><43aa6ff70701202146q3cd34739u3a41c0b78f2a23b9@mail.gmail.com><A68BA56D-27F3-4F10-BA26-43057383F1A4@PageDNA.com><bbaeab100701211057m5c9b9cem30d7379be98138e1@mail.gmail.com><1AC1C3B8-7364-4151-9FED-DC03FA867521@pagedna.com><5.1.1.6.0.20070122101919.06555fc8@sparrow.telecommunity.com>
	<43aa6ff70701221310y5b3d4899sa230a62238ffddc5@mail.gmail.co m>
	<5.1.1.6.0.20070122213225.04252ac8@sparrow.telecommunity.com>
Message-ID: <ep40kp$7ei$1@sea.gmane.org>


"Phillip J. Eby" <pje at telecommunity.com> wrote in message 
news:5.1.1.6.0.20070122213225.04252ac8 at sparrow.telecommunity.com...
| Um, you're not taking 2.6 into consideration here.  Remember that there 
has
| to be a way to spell all these APIs in 2.6 that will still work in 3.0.

| IMO, this seems sufficient reason to keep sys.exc_info() around, since
| there *has* to be some way to access the current exception and traceback,
| that exists in *both* 2.6 and 3.0.

>From what I understand of Guido's recent comments on the 2.6 list, 'has to 
be' is a bit strong.  Pehaps he will 'have to' clarify.

tjr




From tomerfiliba at gmail.com  Tue Jan 23 10:42:29 2007
From: tomerfiliba at gmail.com (tomer filiba)
Date: Tue, 23 Jan 2007 11:42:29 +0200
Subject: [Python-3000] simplifying the exception mechanism
In-Reply-To: <bbaeab100701221456l4f64a20t41d7a34f451ee7ff@mail.gmail.com>
References: <1d85506f0701221354u7247ebch918f5194377445f4@mail.gmail.com>
	<43aa6ff70701221426w2f12413dtc2516fa2e52be2c6@mail.gmail.com>
	<bbaeab100701221456l4f64a20t41d7a34f451ee7ff@mail.gmail.com>
Message-ID: <1d85506f0701230142p709ca844ude5a87c51e9f1798@mail.gmail.com>

well, if it helps improve speed, than that's fine. although
i don't see how really.

first of all, exceptions raised by python code must be
normalized as the code might need the exception object, i.e.

try:
    foo
except bar:
    ex = sys.exc_info()

so the only place it's meaningful is exceptions raised by
the core that are also caught by the core. the only place
where that happens is, afaict, in the getattr mechanism.

well, i guess it's important enough to keep that code fast.


-tomer

On 1/23/07, Brett Cannon <brett at python.org> wrote:
> On 1/22/07, Collin Winter <collinw at gmail.com> wrote:
> > On 1/22/07, tomer filiba <tomerfiliba at gmail.com> wrote:
> > > with all the talk about enhancing exceptions and removing unnecessary
> > > functions from the codebase, i think we can also drop
> > > PyErr_NormalizeException and friends.
> >
> > One argument in favor of keeping PyErr_NormalizeException() (as I've
> > learned from trying to rip it out) is the speed advantage it brings.
> > It's less expensive to sock away the exception class and arguments,
> > only instancing the exception when you really need it. This is
> > important in certain core types, where light(er)-weight exception
> > raising comes in handy.
> >
>
> I'm with Collin on this one.  The speed argument is one of the reasons
> PyErr_NormalizeException even exists.  We want exceptions in the
> simple, general case to be as fast as possible.
>
> -Brett
>

From ncoghlan at gmail.com  Tue Jan 23 13:02:14 2007
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 23 Jan 2007 22:02:14 +1000
Subject: [Python-3000] Pre-peps on raise and except changes
In-Reply-To: <5.1.1.6.0.20070122214504.04425a80@sparrow.telecommunity.com>
References: <5.1.1.6.0.20070122214504.04425a80@sparrow.telecommunity.com>
Message-ID: <45B5F946.8070109@gmail.com>

Phillip J. Eby wrote:
> The 2-expression "raise" statement translation is incorrect in the general 
> case; it is possible for the second argument to be an instance of the first 
> argument, in which case 'raise E, V' should become just 'raise V'.  This is 
> not detectable by the refactoring tool, AFAIK.

There is another issue which currently isn't entirely clear in this PEP 
or in PEP 352 - what happens to except statements which raise a subtype 
of Exception instead of an instance of it.

It's currently implicit that this will continue to be allowed (with 
subtypes being automatically instantiated with no arguments) - I think 
that behaviour should be explicitly stated as intentional in the new 
Py3k PEP.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From thomas at python.org  Tue Jan 23 16:07:55 2007
From: thomas at python.org (Thomas Wouters)
Date: Tue, 23 Jan 2007 16:07:55 +0100
Subject: [Python-3000] Exceptions internals and removing sys.exc_*
In-Reply-To: <046401c73df2$74036cf0$0200a8c0@enfoldsystems.local>
References: <43aa6ff70701202146q3cd34739u3a41c0b78f2a23b9@mail.gmail.com>
	<046401c73df2$74036cf0$0200a8c0@enfoldsystems.local>
Message-ID: <9e804ac0701230707s1a0d0c00le23da6add248a9f8@mail.gmail.com>

On 1/22/07, Mark Hammond <mhammond at skippinet.com.au> wrote:
>
> > Guido has mentioned [1] that since exceptions will be growing a
> > __traceback__ attribute in Python 3, it should be possible to remove
> > sys.exc_info().
>
> sys.exc_info() is also useful for returning the exception itself, not only
> the traceback.  The traceback and logger modules both take advantage of it
> to avoid the pain of passing exception objects around.  eg:
>
>   try:
>    ...
>   except Foo:
>     logger.exception("something bad happened")
>
> is better than what may otherwise be necessary:
>
>   except Foo, f:
>     logger.exception("something bad happened", exc_info=f)
>
> It seems a reasonable use case to have code that knows it is dealing with
> exceptions, but not always be directly in the exception handler.


I agree, and believe sys.exc_info() should just stay. There's also the case
of try/finally and fetching any raised exception there. (In fact, doesn't
the 'with' statement do something similar?)

-- 
Thomas Wouters <thomas at python.org>

Hi! I'm a .signature virus! copy me into your .signature file to help me
spread!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20070123/4381aa1f/attachment.html 

From brett at python.org  Tue Jan 23 17:44:42 2007
From: brett at python.org (Brett Cannon)
Date: Tue, 23 Jan 2007 08:44:42 -0800
Subject: [Python-3000] Pre-peps on raise and except changes
In-Reply-To: <45B5F946.8070109@gmail.com>
References: <5.1.1.6.0.20070122214504.04425a80@sparrow.telecommunity.com>
	<45B5F946.8070109@gmail.com>
Message-ID: <bbaeab100701230844mfea52abp330ce6bdbe0b67dc@mail.gmail.com>

On 1/23/07, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Phillip J. Eby wrote:
> > The 2-expression "raise" statement translation is incorrect in the general
> > case; it is possible for the second argument to be an instance of the first
> > argument, in which case 'raise E, V' should become just 'raise V'.  This is
> > not detectable by the refactoring tool, AFAIK.
>
> There is another issue which currently isn't entirely clear in this PEP
> or in PEP 352 - what happens to except statements which raise a subtype
> of Exception instead of an instance of it.
>

>From PEP 352's perspective, it should still work.  The PEP only made
exceptions new-style and ditched some methods.  It in no way addresses
changes in syntax.

-Brett

From brett at python.org  Tue Jan 23 17:47:38 2007
From: brett at python.org (Brett Cannon)
Date: Tue, 23 Jan 2007 08:47:38 -0800
Subject: [Python-3000] Exceptions internals and removing sys.exc_*
In-Reply-To: <9e804ac0701230707s1a0d0c00le23da6add248a9f8@mail.gmail.com>
References: <43aa6ff70701202146q3cd34739u3a41c0b78f2a23b9@mail.gmail.com>
	<046401c73df2$74036cf0$0200a8c0@enfoldsystems.local>
	<9e804ac0701230707s1a0d0c00le23da6add248a9f8@mail.gmail.com>
Message-ID: <bbaeab100701230847p2a9f4d64j5b49e64a51393f89@mail.gmail.com>

On 1/23/07, Thomas Wouters <thomas at python.org> wrote:
>
>
> On 1/22/07, Mark Hammond <mhammond at skippinet.com.au> wrote:
> > > Guido has mentioned [1] that since exceptions will be growing a
> > > __traceback__ attribute in Python 3, it should be possible to remove
> > > sys.exc_info().
> >
> > sys.exc_info() is also useful for returning the exception itself, not only
> > the traceback.  The traceback and logger modules both take advantage of it
> > to avoid the pain of passing exception objects around.  eg:
> >
> >   try:
> >    ...
> >   except Foo:
> >     logger.exception("something bad happened")
> >
> > is better than what may otherwise be necessary:
> >
> >   except Foo, f:
> >     logger.exception("something bad happened", exc_info=f)
> >
> > It seems a reasonable use case to have code that knows it is dealing with
> > exceptions, but not always be directly in the exception handler.
>
> I agree, and believe sys.exc_info() should just stay. There's also the case
> of try/finally and fetching any raised exception there. (In fact, doesn't
> the 'with' statement do something similar?)
>

We can at least discourage its use while leaving it in, with the
possibility of removing it in the future once people are used to
tracebacks being on exceptions.  How long that will be who knows
(maybe Py4K  =).

-Brett

From steven.bethard at gmail.com  Tue Jan 23 17:52:21 2007
From: steven.bethard at gmail.com (Steven Bethard)
Date: Tue, 23 Jan 2007 09:52:21 -0700
Subject: [Python-3000] Exceptions internals and removing sys.exc_*
In-Reply-To: <9e804ac0701230707s1a0d0c00le23da6add248a9f8@mail.gmail.com>
References: <43aa6ff70701202146q3cd34739u3a41c0b78f2a23b9@mail.gmail.com>
	<046401c73df2$74036cf0$0200a8c0@enfoldsystems.local>
	<9e804ac0701230707s1a0d0c00le23da6add248a9f8@mail.gmail.com>
Message-ID: <d11dcfba0701230852u2e862ad5o7163c3db81aab9ef@mail.gmail.com>

On 1/23/07, Thomas Wouters <thomas at python.org> wrote:
>
> On 1/22/07, Mark Hammond <mhammond at skippinet.com.au> wrote:
> > > Guido has mentioned [1] that since exceptions will be growing a
> > > __traceback__ attribute in Python 3, it should be possible to remove
> > > sys.exc_info().
> >
> > sys.exc_info() is also useful for returning the exception itself, not only
> > the traceback.  The traceback and logger modules both take advantage of it
> > to avoid the pain of passing exception objects around.  eg:
> >
> >   try:
> >    ...
> >   except Foo:
> >     logger.exception("something bad happened")
> >
> > is better than what may otherwise be necessary:
> >
> >   except Foo, f:
> >     logger.exception("something bad happened", exc_info=f)
> >
> > It seems a reasonable use case to have code that knows it is dealing with
> > exceptions, but not always be directly in the exception handler.
>
> I agree, and believe sys.exc_info() should just stay. There's also the case
> of try/finally and fetching any raised exception there. (In fact, doesn't
> the 'with' statement do something similar?)

Sorry, for those of us following along at home ;-) could someone
explain why we couldn't just introduce a ``sys.get_exception()`` or
something like that which would then make the exception (and therefore
the traceback) available?

STeVe
-- 
I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a
tiny blip on the distant coast of sanity.
        --- Bucky Katt, Get Fuzzy

From fdrake at acm.org  Tue Jan 23 18:00:15 2007
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Tue, 23 Jan 2007 12:00:15 -0500
Subject: [Python-3000] Exceptions internals and removing sys.exc_*
In-Reply-To: <d11dcfba0701230852u2e862ad5o7163c3db81aab9ef@mail.gmail.com>
References: <43aa6ff70701202146q3cd34739u3a41c0b78f2a23b9@mail.gmail.com>
	<9e804ac0701230707s1a0d0c00le23da6add248a9f8@mail.gmail.com>
	<d11dcfba0701230852u2e862ad5o7163c3db81aab9ef@mail.gmail.com>
Message-ID: <200701231200.15885.fdrake@acm.org>

On Tuesday 23 January 2007 11:52, Steven Bethard wrote:
 > Sorry, for those of us following along at home ;-) could someone
 > explain why we couldn't just introduce a ``sys.get_exception()`` or
 > something like that which would then make the exception (and therefore
 > the traceback) available?

Works for me!  ;-)


  -Fred

-- 
Fred L. Drake, Jr.   <fdrake at acm.org>

From thomas at python.org  Tue Jan 23 18:07:40 2007
From: thomas at python.org (Thomas Wouters)
Date: Tue, 23 Jan 2007 18:07:40 +0100
Subject: [Python-3000] Exceptions internals and removing sys.exc_*
In-Reply-To: <d11dcfba0701230852u2e862ad5o7163c3db81aab9ef@mail.gmail.com>
References: <43aa6ff70701202146q3cd34739u3a41c0b78f2a23b9@mail.gmail.com>
	<046401c73df2$74036cf0$0200a8c0@enfoldsystems.local>
	<9e804ac0701230707s1a0d0c00le23da6add248a9f8@mail.gmail.com>
	<d11dcfba0701230852u2e862ad5o7163c3db81aab9ef@mail.gmail.com>
Message-ID: <9e804ac0701230907p52c14de0t5290e734a195048@mail.gmail.com>

On 1/23/07, Steven Bethard <steven.bethard at gmail.com> wrote:
>
> On 1/23/07, Thomas Wouters <thomas at python.org> wrote:
> >
> > On 1/22/07, Mark Hammond <mhammond at skippinet.com.au> wrote:
> > > > Guido has mentioned [1] that since exceptions will be growing a
> > > > __traceback__ attribute in Python 3, it should be possible to remove
> > > > sys.exc_info().
> > >
> > > sys.exc_info() is also useful for returning the exception itself, not
> only
> > > the traceback.  The traceback and logger modules both take advantage
> of it
> > > to avoid the pain of passing exception objects around.  eg:
> > >
> > >   try:
> > >    ...
> > >   except Foo:
> > >     logger.exception("something bad happened")
> > >
> > > is better than what may otherwise be necessary:
> > >
> > >   except Foo, f:
> > >     logger.exception("something bad happened", exc_info=f)
> > >
> > > It seems a reasonable use case to have code that knows it is dealing
> with
> > > exceptions, but not always be directly in the exception handler.
> >
> > I agree, and believe sys.exc_info() should just stay. There's also the
> case
> > of try/finally and fetching any raised exception there. (In fact,
> doesn't
> > the 'with' statement do something similar?)
>
> Sorry, for those of us following along at home ;-) could someone
> explain why we couldn't just introduce a ``sys.get_exception()`` or
> something like that which would then make the exception (and therefore
> the traceback) available?


We could, of course, but what's the point? To rename the existing 'exc_info'
into 'get_exception' and some attribute-retrieval? I'm all for cleanup and
not looking back, but that's quite a lot of gratuitous breakage, to little
point. I don't consider it 'weird' or 'bad' or 'a wart' or 'a hysterical
raisin' that sys.exc_info() returns a tuple of three things.

-- 
Thomas Wouters <thomas at python.org>

Hi! I'm a .signature virus! copy me into your .signature file to help me
spread!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20070123/76df9383/attachment.htm 

From steven.bethard at gmail.com  Tue Jan 23 18:57:00 2007
From: steven.bethard at gmail.com (Steven Bethard)
Date: Tue, 23 Jan 2007 10:57:00 -0700
Subject: [Python-3000] Exceptions internals and removing sys.exc_*
In-Reply-To: <9e804ac0701230907p52c14de0t5290e734a195048@mail.gmail.com>
References: <43aa6ff70701202146q3cd34739u3a41c0b78f2a23b9@mail.gmail.com>
	<046401c73df2$74036cf0$0200a8c0@enfoldsystems.local>
	<9e804ac0701230707s1a0d0c00le23da6add248a9f8@mail.gmail.com>
	<d11dcfba0701230852u2e862ad5o7163c3db81aab9ef@mail.gmail.com>
	<9e804ac0701230907p52c14de0t5290e734a195048@mail.gmail.com>
Message-ID: <d11dcfba0701230957j75125e2eg61546495d51c5631@mail.gmail.com>

On 1/23/07, Thomas Wouters <thomas at python.org> wrote:
> On 1/23/07, Steven Bethard <steven.bethard at gmail.com> wrote:
> > Sorry, for those of us following along at home ;-) could someone
> > explain why we couldn't just introduce a ``sys.get_exception()`` or
> > something like that which would then make the exception (and therefore
> > the traceback) available?
>
> We could, of course, but what's the point? To rename the existing 'exc_info'
> into 'get_exception' and some attribute-retrieval? I'm all for cleanup and
> not looking back, but that's quite a lot of gratuitous breakage, to little
> point. I don't consider it 'weird' or 'bad' or 'a wart' or 'a hysterical
> raisin' that sys.exc_info() returns a tuple of three things.

If I understand it right, all three elements of the sys.exc_info()
tuple are derivable from only the middle element (at least in Python
3.0 where the middle element would always have to be an exception
instance[1])::

    def exc_info():
        exc = sys.get_exception()
        return exc.__class__, exc, exc.__traceback__

So I guess it just seems a little redundant to return a tuple of three
things when two of the three are easily derivable from the third.  I
was trying to think of any other API in Python which does something
like this, but I couldn't find one...


[1] http://docs.python.org/lib/module-sys.html
"The values returned are (type, value, traceback). Their meaning is:
... ``value`` gets the exception parameter (its associated value  or
the second argument to raise, which is always a class instance if the
exception type is a class object)"

STeVe
-- 
I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a
tiny blip on the distant coast of sanity.
        --- Bucky Katt, Get Fuzzy

From collinw at gmail.com  Tue Jan 23 23:07:13 2007
From: collinw at gmail.com (Collin Winter)
Date: Tue, 23 Jan 2007 16:07:13 -0600
Subject: [Python-3000] Exceptions internals and removing sys.exc_*
In-Reply-To: <d11dcfba0701230957j75125e2eg61546495d51c5631@mail.gmail.com>
References: <43aa6ff70701202146q3cd34739u3a41c0b78f2a23b9@mail.gmail.com>
	<046401c73df2$74036cf0$0200a8c0@enfoldsystems.local>
	<9e804ac0701230707s1a0d0c00le23da6add248a9f8@mail.gmail.com>
	<d11dcfba0701230852u2e862ad5o7163c3db81aab9ef@mail.gmail.com>
	<9e804ac0701230907p52c14de0t5290e734a195048@mail.gmail.com>
	<d11dcfba0701230957j75125e2eg61546495d51c5631@mail.gmail.com>
Message-ID: <43aa6ff70701231407mb6c77dfpb6a06dd7bc27e9a3@mail.gmail.com>

On 1/23/07, Steven Bethard <steven.bethard at gmail.com> wrote:
> On 1/23/07, Thomas Wouters <thomas at python.org> wrote:
> > On 1/23/07, Steven Bethard <steven.bethard at gmail.com> wrote:
> > > Sorry, for those of us following along at home ;-) could someone
> > > explain why we couldn't just introduce a ``sys.get_exception()`` or
> > > something like that which would then make the exception (and therefore
> > > the traceback) available?
> >
> > We could, of course, but what's the point? To rename the existing 'exc_info'
> > into 'get_exception' and some attribute-retrieval? I'm all for cleanup and
> > not looking back, but that's quite a lot of gratuitous breakage, to little
> > point. I don't consider it 'weird' or 'bad' or 'a wart' or 'a hysterical
> > raisin' that sys.exc_info() returns a tuple of three things.
>
> If I understand it right, all three elements of the sys.exc_info()
> tuple are derivable from only the middle element (at least in Python
> 3.0 where the middle element would always have to be an exception
> instance[1])::
>
>     def exc_info():
>         exc = sys.get_exception()
>         return exc.__class__, exc, exc.__traceback__
>
> So I guess it just seems a little redundant to return a tuple of three
> things when two of the three are easily derivable from the third.  I
> was trying to think of any other API in Python which does something
> like this, but I couldn't find one...

It strikes me as a redundant, too, but I'm -1 on dropping
sys.exc_info() entirely in 3.0 (at least not without a deprecation
period in 2.x). While a fixer for 2to3 to convert sys.exc_info() calls
into sys.get_exception() terms would be pretty trivial, that doesn't
cover converting all the functions that expect an exc_info() tuple as
arguments, nor does it cover all the necessary documentation changes.

Collin Winter

From collinw at gmail.com  Tue Jan 23 23:11:47 2007
From: collinw at gmail.com (Collin Winter)
Date: Tue, 23 Jan 2007 16:11:47 -0600
Subject: [Python-3000] Pre-peps on raise and except changes
In-Reply-To: <45B5F946.8070109@gmail.com>
References: <5.1.1.6.0.20070122214504.04425a80@sparrow.telecommunity.com>
	<45B5F946.8070109@gmail.com>
Message-ID: <43aa6ff70701231411y759f8b4jc829b7f1d425ec1f@mail.gmail.com>

On 1/23/07, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Phillip J. Eby wrote:
> > The 2-expression "raise" statement translation is incorrect in the general
> > case; it is possible for the second argument to be an instance of the first
> > argument, in which case 'raise E, V' should become just 'raise V'.  This is
> > not detectable by the refactoring tool, AFAIK.
>
> There is another issue which currently isn't entirely clear in this PEP
> or in PEP 352 - what happens to except statements which raise a subtype
> of Exception instead of an instance of it.
>
> It's currently implicit that this will continue to be allowed (with
> subtypes being automatically instantiated with no arguments) - I think
> that behaviour should be explicitly stated as intentional in the new
> Py3k PEP.

Does this language work for you?

"""
2. ``raise E`` (with a single argument) is used to raise a new
   exception. This form has two sub-variants: ``E`` may be either an
   instance of ``BaseException`` [#pep352]_ or a subclass of
   ``BaseException``. If ``E`` is a subclass, it will be called with
   no arguments to obtain an exception instance.

   To raise anything else is an error.
"""

Thanks,
Collin Winter

From brett at python.org  Tue Jan 23 23:34:07 2007
From: brett at python.org (Brett Cannon)
Date: Tue, 23 Jan 2007 14:34:07 -0800
Subject: [Python-3000] Pre-peps on raise and except changes
In-Reply-To: <43aa6ff70701231411y759f8b4jc829b7f1d425ec1f@mail.gmail.com>
References: <5.1.1.6.0.20070122214504.04425a80@sparrow.telecommunity.com>
	<45B5F946.8070109@gmail.com>
	<43aa6ff70701231411y759f8b4jc829b7f1d425ec1f@mail.gmail.com>
Message-ID: <bbaeab100701231434n1f49e704y32c0d19b3e992859@mail.gmail.com>

On 1/23/07, Collin Winter <collinw at gmail.com> wrote:
> On 1/23/07, Nick Coghlan <ncoghlan at gmail.com> wrote:
> > Phillip J. Eby wrote:
> > > The 2-expression "raise" statement translation is incorrect in the general
> > > case; it is possible for the second argument to be an instance of the first
> > > argument, in which case 'raise E, V' should become just 'raise V'.  This is
> > > not detectable by the refactoring tool, AFAIK.
> >
> > There is another issue which currently isn't entirely clear in this PEP
> > or in PEP 352 - what happens to except statements which raise a subtype
> > of Exception instead of an instance of it.
> >
> > It's currently implicit that this will continue to be allowed (with
> > subtypes being automatically instantiated with no arguments) - I think
> > that behaviour should be explicitly stated as intentional in the new
> > Py3k PEP.
>
> Does this language work for you?
>
> """
> 2. ``raise E`` (with a single argument) is used to raise a new
>    exception.

For some reason I thought you were initially talking about a single
argument to the exception, not to E.

>This form has two sub-variants: ``E`` may be either an
>    instance of ``BaseException`` [#pep352]_ or a subclass of
>    ``BaseException``. If ``E`` is a subclass, it will be called with
>    no arguments to obtain an exception instance.
>
>    To raise anything else is an error.
> """

I don't think that calling them a variant is right.  You can only
raise subclasses of BaseException (which implicitly implies
BaseException itself).  If you want to mention BaseException itself
that's fine, but there is no variant here; there is a single rule.

-Brett

From collinw at gmail.com  Wed Jan 24 00:03:18 2007
From: collinw at gmail.com (Collin Winter)
Date: Tue, 23 Jan 2007 17:03:18 -0600
Subject: [Python-3000] Pre-peps on raise and except changes
In-Reply-To: <bbaeab100701231434n1f49e704y32c0d19b3e992859@mail.gmail.com>
References: <5.1.1.6.0.20070122214504.04425a80@sparrow.telecommunity.com>
	<45B5F946.8070109@gmail.com>
	<43aa6ff70701231411y759f8b4jc829b7f1d425ec1f@mail.gmail.com>
	<bbaeab100701231434n1f49e704y32c0d19b3e992859@mail.gmail.com>
Message-ID: <43aa6ff70701231503v5bc08d69n9d53bbaa3373c5c@mail.gmail.com>

On 1/23/07, Brett Cannon <brett at python.org> wrote:
> On 1/23/07, Collin Winter <collinw at gmail.com> wrote:
> >This form has two sub-variants: ``E`` may be either an
> >    instance of ``BaseException`` [#pep352]_ or a subclass of
> >    ``BaseException``. If ``E`` is a subclass, it will be called with
> >    no arguments to obtain an exception instance.
> >
> >    To raise anything else is an error.
> > """
>
> I don't think that calling them a variant is right.  You can only
> raise subclasses of BaseException (which implicitly implies
> BaseException itself).  If you want to mention BaseException itself
> that's fine, but there is no variant here; there is a single rule.

You can raise both subclasses of BaseException and instances of
subclasses of BaseException. Are you intended to make the latter
illegal?

Collin Winter

From jimjjewett at gmail.com  Wed Jan 24 00:22:24 2007
From: jimjjewett at gmail.com (Jim Jewett)
Date: Tue, 23 Jan 2007 18:22:24 -0500
Subject: [Python-3000] Pre-peps on raise and except changes
In-Reply-To: <43aa6ff70701231411y759f8b4jc829b7f1d425ec1f@mail.gmail.com>
References: <5.1.1.6.0.20070122214504.04425a80@sparrow.telecommunity.com>
	<45B5F946.8070109@gmail.com>
	<43aa6ff70701231411y759f8b4jc829b7f1d425ec1f@mail.gmail.com>
Message-ID: <fb6fbf560701231522h3fea803fnca23ca8714ccf881@mail.gmail.com>

On 1/23/07, Collin Winter <collinw at gmail.com> wrote:
> Does this language work for you?

> """
> 2. ``raise E`` (with a single argument) is used to raise a new
>    exception. This form has two sub-variants: ``E`` may be either an
>    instance of ``BaseException`` [#pep352]_ or a subclass of
>    ``BaseException``. If ``E`` is a subclass, it will be called with
>    no arguments to obtain an exception instance.
>
>    To raise anything else is an error.
> """

It works for me.  In particular, it even clears up the pathological
case of an object which is both a subclass and an instance of
BaseException.

-jJ

From collinw at gmail.com  Wed Jan 24 00:23:43 2007
From: collinw at gmail.com (Collin Winter)
Date: Tue, 23 Jan 2007 17:23:43 -0600
Subject: [Python-3000] Pre-peps on raise and except changes (was:
	Warning for 2.6 and greater)
In-Reply-To: <5.1.1.6.0.20070122214504.04425a80@sparrow.telecommunity.com>
References: <5.1.1.6.0.20070122214504.04425a80@sparrow.telecommunity.com>
Message-ID: <43aa6ff70701231523x5af8e22dv587bd228d5aecc9e@mail.gmail.com>

On 1/22/07, Phillip J. Eby <pje at telecommunity.com> wrote:
> The correct translation of a try/except should be:
>
>      try:
>          ...
>      except E as N:
>          try:
>              ...
>          finally:
>              N = None
>              del N
>
> i.e., you left out the crucial try/finally wrapper.  If you got this
> translation from the current implementation, then the implementation is
> wrong too.  :)

Yikes! The implementation is right, it's my memory that's bad : )

> Also, the PEP should discuss the matter of how "except" should change in
> Python 2.6 to support this, including the current open issue of handling
> the .traceback attribute.

I believe it's been suggested that 2.x's "except" will support both
"," and "as" between the exception class and the target, but I don't
recall any explicit approval for that course of action.

> The 2-expression "raise" statement translation is incorrect in the general
> case; it is possible for the second argument to be an instance of the first
> argument, in which case 'raise E, V' should become just 'raise V'.  This is
> not detectable by the refactoring tool, AFAIK.

It's possible, but rare in practice. I've run across only nine such
usages in the entire standard distribution. They break down into two
camps:

As a synonym for a zero-argument raise:
    Demo/pysvr/pysvr.py
    Lib/re.py

Casting an exception from one type to another:
    Tools/webchecker/webchecker.py
    Lib/os.py
    Lib/pty.py
    Lib/distutils/bcppcompiler.py
    Lib/distutils/unixccompiler.py
    Lib/distutils/fancy_getopt.py
    Lib/distutils/emxccompiler.py

These latter usages could all be expressed as "raise E from V"
(starting as soon as that gets implemented). I've added language to
the "raise" PEP detailing how to convert these to "raise ... from" for
Python 3.

The latest version of both PEPs are attached.

Thanks for the feedback!

Collin Winter
-------------- next part --------------
PEP: 3109
Title: Catching Exceptions in Python 3000
Version: $Revision$
Last-Modified: $Date$
Author: Collin Winter <collinw at gmail.com>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 16-Jan-2006
Python-Version: 3.0
Post-History:


Abstract
========

This PEP introduces changes intended to help eliminate ambiguities
in Python's grammar, simplify exception classes, simplify garbage
collection for exceptions and reduce the size of the language in
Python 3.0.


Rationale
=========

1. ``except`` clauses in Python 2.x present a syntactic ambiguity
   where the parser cannot differentiate whether ::

       except <expression>, <expression>:
    
   should be interpreted as ::

       except <type>, <type>:
    
   or ::

       except <type>, <name>:

   Python 2 opts for the latter semantic, at the cost of requiring the
   former to be parenthesized, like so ::

    except (<type>, <type>):

2. As specified in PEP 352 [#pep352]_, the ability to treat exceptions
   as tuples will be removed, meaning this code will no longer work ::

       except os.error, (errno, errstr):
    
   Because the automatic unpacking will no longer be possible, it is
   desirable to remove the ability to use tuples as ``except`` targets.

3. As specified in PEP 344 [#pep344]_, exception instances in Python 3
   will possess a ``__traceback__`` attribute. The Open Issues section
   of that PEP includes a paragraph on garbage collection difficulties
   caused by this attribute, namely a "exception -> traceback ->
   stack frame -> exception" reference cycle, whereby all locals are 
   kept in scope until the next GC run. This PEP intends to resolve
   this issue by adding a cleanup semantic to ``except`` clauses in
   Python 3 whereby the target name is deleted at the end of the
   ``except`` suite.
   
4. In the spirit of "there should be one -- and preferably only one
   -- obvious way to do it" [#zen]_, it is desirable to consolidate
   duplicate functionality. To this end, the ``exc_value``,
   ``exc_type`` and ``exc_traceback`` attributes of the ``sys``
   module [#sys-module]_ will be removed in favor of
   ``sys.exc_info()``, which provides the same information. These
   attributes are already listed in PEP 3100 [#pep3100]_ as targeted
   for removal.


Grammar Changes
===============
    
In Python 3, the grammar for ``except`` statements will change
from [#grammar]_ ::

    except_clause: 'except' [test [',' test]]
    
to ::

    except_clause: 'except' [test ['as' NAME]]

The use of ``as`` in place of the comma token means that ::

    except AttributeError, os.error:
    
can be clearly understood as a tuple of exception classes. This new
syntax was first proposed by Greg Ewing [#firstproposal]_ and
endorsed ([#firstproposal]_, [#renaming]_) by the BDFL.

Further, the restriction of the token following ``as`` from ``test``
to ``NAME`` means that only valid identifiers can be used as
``except`` targets.


Semantic Changes
================

In order to resolve the garbage collection issue related to PEP 344,
``except`` statements will generate additional bytecode to delete the
target, thus eliminating the reference cycle. The source-to-source
translation, as suggested by Phillip J. Eby [#except-translation]_ is
::

    try:
        ...
    except E as N:
        ...
    ...
    
is translated to ::

    try:
        ...
    except E as N:
        try:
            ...
        finally:
            N = None
            del N
    ...

An implementation has already been checked into the p3yk branch
[#translation-checkin]_.


Compatibility Issues
====================

Nearly all ``except`` clauses will need to be changed. ``except``
clauses with identifier targets will be converted from ::

    except E, N:
    
to ::
    
    except E as N:

``except`` clauses with non-tuple, non-identifier targets
(e.g., ``a.b.c[d]``) will need to be converted from ::

    except E, T:
    
to ::

    except E as t:
        T = t
        
Both of these cases can be handled by Guido van Rossum's ``2to3``
utility [#2to3]_ using the ``except`` fixer [#exceptfixer]_.
        
``except`` clauses with tuple targets will need to be converted
manually, on a case-by-case basis. These changes will usually need
to be accompanied by changes to the exception classes themselves.
While these changes generally cannot be automated, the ``2to3``
utility is able to point out cases where the target of an ``except``
clause is a tuple, simplifying conversion.

Situations where it is necessary to keep an exception instance around
past the end of the ``except`` suite can be easily translated like so
::

    try:
        ...
    except E as N:
        ...
    ...
    
is translated to ::

    try:
        ...
    except E as N:
        n = N
        ...
    ...
    
This way, when ``N`` is deleted at the end of the block, ``n`` will
persist and can be used as normal.

Lastly, all uses of the ``sys`` module's ``exc_type``, ``exc_value``
and ``exc_traceback`` attributes will need to be removed. They can be
replaced with ``sys.exc_info()[0]``, ``sys.exc_info()[1]`` and
``sys.exc_info()[2]`` respectively, a transformation that can be
performed by ``2to3``'s ``sysexcinfo`` fixer.


Open Issues
===========

"except" Statements in Python 2.x
-----------------------------------

It has been proposed that the grammar for ``except`` statements be
changed to accommodate both Python 2's ``,`` and Python 3's ``as``
between the statement's type and target portions. The grammar
would thus change from ::

    except_clause: 'except' [test [',' test]]

to ::

    except_clause: 'except' [test [('as' | ',') test]]

It has not been decided whether the proposed end-of-suite cleanup
semantic for ``except`` statements should be included in the 2.x
series.


References
==========

.. [#pep352]
   http://www.python.org/dev/peps/pep-0352/
   
.. [#zen]
   http://www.python.org/dev/peps/pep-0020/
   
.. [#sys-module]
   http://docs.python.org/lib/module-sys.html
   
.. [#pep3100]
   http://www.python.org/dev/peps/pep-3100/
   
.. [#pep344]
   http://www.python.org/dev/peps/pep-0344/

.. [#firstproposal]
   http://mail.python.org/pipermail/python-dev/2006-March/062449.html

.. [#renaming]
   http://mail.python.org/pipermail/python-dev/2006-March/062640.html
   
.. [#grammar]
   http://www.python.org/doc/current/ref/try.html
   
.. [#except-translation]
   http://mail.python.org/pipermail/python-3000/2007-January/005395.html
   
.. [#translation-checkin]
   http://svn.python.org/view?rev=53342&view=rev
   
.. [#2to3]
   http://svn.python.org/view/sandbox/trunk/2to3/
   
.. [#exceptfixer]
   http://svn.python.org/view/sandbox/trunk/2to3/fixes/fix_except.py


Copyright
=========

This document has been placed in the public domain.



..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 70
   coding: utf-8
   End:
-------------- next part --------------
PEP: 3109
Title: Raising Exceptions in Python 3000
Version: $Revision$
Last-Modified: $Date$
Author: Collin Winter <collinw at gmail.com>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 19-Jan-2006
Python-Version: 3.0
Post-History:


Abstract
========

This PEP introduces changes to Python's mechanisms for raising
exceptions intended to reduce both line noise and the size of the
language.


Rationale
=========

One of Python's guiding maxims is "there should be one -- and
preferably only one -- obvious way to do it" [#zen]_. Python 2.x's
``raise`` statement violates this principle, permitting multiple
ways of expressing the same thought. For example, these statements
are equivalent: ::
    
    raise E, V
    
    raise E(V)
    
There is a third form of the ``raise`` statement, allowing arbitrary
tracebacks to be attached to an exception [#grammar]_: ::

    raise E, V, T
    
where T is a traceback. As specified in PEP 344 [#pep344]_,
exception objects in Python 3.x will possess a ``__traceback__``
attribute, admitting this translation of the three-expression
``raise`` statement: ::

    raise E, V, T
    
is translated to ::

    e = E(V)
    e.__traceback__ = T
    raise e
    
Using these translations, we can reduce the ``raise`` statement from
four forms to two:

1. ``raise`` (with no arguments) is used to re-raise the active
   exception in an ``except`` suite.
   
2. ``raise EXCEPTION`` is used to raise a new exception. This form has
   two sub-variants: ``EXCEPTION`` may be either an instance of
   ``BaseException`` or a subclass of ``BaseException`` [#pep352]_.
   If ``EXCEPTION`` is a subclass, it will be called with no arguments
   to obtain an exception instance.
   
   To raise anything else is an error.

There is a further, more tangible benefit to be obtained through this
consolidation, as noted by A.M. Kuchling [#amk-line-noise]_. ::

    PEP 8 doesn't express any preference between the 
    two forms of raise statements:
    raise ValueError, 'blah'
    raise ValueError("blah")

    I like the second form better, because if the exception arguments
    are long or include string formatting, you don't need to use line
    continuation characters because of the containing parens.
    
The BDFL has concurred [#guido-declaration]_ and endorsed the
consolidation of the several ``raise`` forms.
    

Grammar Changes
===============
    
In Python 3, the grammar for ``raise`` statements will change
from [#grammar]_ ::

    raise_stmt: 'raise' [test [',' test [',' test]]]
    
to ::

    raise_stmt: 'raise' [test]
    
    
Changes to Builtin Types
========================
    
Because of its relation to exception raising, the signature for the
``throw()`` method on generator objects will change, dropping the
optional second and third parameters. The signature thus changes 
from [#throw-sig]_ ::

    generator.throw(E, [V, [T]])
    
to ::

    generator.throw(EXCEPTION)
    
Where ``EXCEPTION`` is either a subclass of ``BaseException`` or an
instance of a subclass of ``BaseException``.

    
Semantic Changes
================

In Python 2, the following ``raise`` statement is legal ::

    raise ((E1, (E2, E3)), E4), V
    
The interpreter will take the tuple's first element as the exception
type (recursively), making the above fully equivalent to ::

    raise E1, V
    
As of Python 3.0, support for raising tuples like this will be
dropped. This change will bring ``raise`` statements into line with
the ``throw()`` method on generator objects, which already disallows
this.
    

Compatibility Issues
====================

All two- and three-expression ``raise`` statements will require
modification, as will all two- and three-expression ``throw()`` calls
on generators. Fortunately, the translation from Python 2.x to
Python 3.x in this case is simple and can be handled mechanically
by Guido van Rossum's 2to3 utility [#2to3]_ using the ``raise`` and
``throw`` fixers ([#raise-fixer]_, [#throw-fixer]_).

The following translations will be performed:

1. Zero- and one-expression ``raise`` statements will be left
   intact.
   
2. Two-expression ``raise`` statements will be converted from ::
        
        raise E, V
        
   to ::
   
        raise E(V)
        
   Two-expression ``throw()`` calls will be converted from ::
        
        generator.throw(E, V)
        
   to ::
   
        generator.throw(E(V))
        
   See point #5 for a caveat to this transformation.
        
3. Three-expression ``raise`` statements will be converted from ::

        raise E, V, T
        
   to ::
   
        e = E(V)
        e.__traceback__ = T
        raise e
        
   Three-expression ``throw()`` calls will be converted from ::
   
        generator.throw(E, V, T)
        
   to ::
   
        e = E(V)
        e.__traceback__ = T
        generator.throw(e)
        
   See point #5 for a caveat to this transformation.
        
4. Two- and three-expression ``raise`` statements where ``E`` is a
   tuple literal can be converted automatically using ``2to3``'s
   ``raise`` fixer. ``raise`` statements where ``E`` is a non-literal
   tuple, e.g., the result of a function call, will need to be
   converted manually.
   
5. Two- and three-expression ``raise`` statements where ``E`` is an
   exception class and ``V`` is an exception instance will need
   special attention. These cases break down into two camps:
   
   1. ``raise E, V`` as a long-hand version of the zero-argument
      ``raise`` statement. As an example, assuming F is a subclass
      of E ::
      
          try:
              something()
          except F as V:
              raise F(V)
          except E as V:
              handle(V)
              
      This would be better expressed as ::
      
          try:
              something()
          except F:
              raise
          except E as V:
              handle(V)
              
    2. ``raise E, V`` as a way of "casting" an exception to another
       class. Taking an example from
       distutils.compiler.unixcompiler ::
       
            try:
                self.spawn(pp_args)
            except DistutilsExecError as msg:
                raise CompileError(msg)
                
       This would be better expressed as ::
       
            try:
                self.spawn(pp_args)
            except DistutilsExecError as msg:
                raise CompileError from msg
   
       Using the ``raise ... from ...`` syntax introduced in
       PEP 344.


References
==========

.. [#zen]
   http://www.python.org/dev/peps/pep-0020/
   
.. [#grammar]
   http://www.python.org/doc/current/ref/raise.html
   
.. [#throw-sig]
   http://www.python.org/dev/peps/pep-0342/
   
.. [#pep344]
   http://www.python.org/dev/peps/pep-0344/
   
.. [#pep352]
   http://www.python.org/dev/peps/pep-0352/
   
.. [#amk-line-noise]
   http://mail.python.org/pipermail/python-dev/2005-August/055187.html

.. [#guido-declaration]
   http://mail.python.org/pipermail/python-dev/2005-August/055190.html
   
.. [#2to3]
   http://svn.python.org/view/sandbox/trunk/2to3/
   
.. [#raise-fixer]
   http://svn.python.org/view/sandbox/trunk/2to3/fixes/fix_raise.py
   
.. [#throw-fixer]
   http://svn.python.org/view/sandbox/trunk/2to3/fixes/fix_throw.py


Copyright
=========

This document has been placed in the public domain.



..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 70
   coding: utf-8
   End:

From brett at python.org  Wed Jan 24 00:24:45 2007
From: brett at python.org (Brett Cannon)
Date: Tue, 23 Jan 2007 15:24:45 -0800
Subject: [Python-3000] Pre-peps on raise and except changes
In-Reply-To: <43aa6ff70701231503v5bc08d69n9d53bbaa3373c5c@mail.gmail.com>
References: <5.1.1.6.0.20070122214504.04425a80@sparrow.telecommunity.com>
	<45B5F946.8070109@gmail.com>
	<43aa6ff70701231411y759f8b4jc829b7f1d425ec1f@mail.gmail.com>
	<bbaeab100701231434n1f49e704y32c0d19b3e992859@mail.gmail.com>
	<43aa6ff70701231503v5bc08d69n9d53bbaa3373c5c@mail.gmail.com>
Message-ID: <bbaeab100701231524o38aaf630n2ce43c6b767b9b8b@mail.gmail.com>

On 1/23/07, Collin Winter <collinw at gmail.com> wrote:
> On 1/23/07, Brett Cannon <brett at python.org> wrote:
> > On 1/23/07, Collin Winter <collinw at gmail.com> wrote:
> > >This form has two sub-variants: ``E`` may be either an
> > >    instance of ``BaseException`` [#pep352]_ or a subclass of
> > >    ``BaseException``. If ``E`` is a subclass, it will be called with
> > >    no arguments to obtain an exception instance.
> > >
> > >    To raise anything else is an error.
> > > """
> >
> > I don't think that calling them a variant is right.  You can only
> > raise subclasses of BaseException (which implicitly implies
> > BaseException itself).  If you want to mention BaseException itself
> > that's fine, but there is no variant here; there is a single rule.
>
> You can raise both subclasses of BaseException and instances of
> subclasses of BaseException. Are you intended to make the latter
> illegal?
>

No, of course not.  As long as BaseException is in the inheritance
tree you are fine.

I just realized I misread your paragraph and took E to represent an
exception class, not a nebulous object that could be an exception
class or instance.

-Brett

From rasky at develer.com  Wed Jan 24 02:13:35 2007
From: rasky at develer.com (Giovanni Bajo)
Date: Wed, 24 Jan 2007 02:13:35 +0100
Subject: [Python-3000] Pre-peps on raise and except changes (was:
 Warning for 2.6 and greater)
In-Reply-To: <43aa6ff70701221445s5edb4b2do8c8cffdebc759c7@mail.gmail.com>
References: <43aa6ff70701221445s5edb4b2do8c8cffdebc759c7@mail.gmail.com>
Message-ID: <ep6brv$p3c$1@sea.gmane.org>

On 22/01/2007 23.45, Collin Winter wrote:

> target, thus eliminating the reference cycle. The source-to-source
> translation, as suggested by Phillip J. Eby [#except-translation]_ is
> ::
> 
>     try:
>         ...
>     except E as N:
>         ...
>     ...
>     
> is translated to ::
> 
>     try:
>         ...
>     except E as N:
>         ...
>         N = None
>         del N
>     ...
> 

The mail you link (which matched my recollection of the thread) has an 
additional try/finally suite in the except block translation:

    except ExcType, e:
            try:
                # body
            finally:
                e = None
                del e

which is missing in the PEP. I think you should probably explain why the 
try/finally is not needed (if it really is not).

Also, the second code snippet should probably use the "except E, N" form (no 
"as") to make more clear that you're speaking of the Python 2.x equivalent of 
the 3.x form.
-- 
Giovanni Bajo


From jimjjewett at gmail.com  Wed Jan 24 19:42:27 2007
From: jimjjewett at gmail.com (Jim Jewett)
Date: Wed, 24 Jan 2007 13:42:27 -0500
Subject: [Python-3000] gc, timestamping
Message-ID: <fb6fbf560701241042i6eef3aafqaba66391d331dd1e@mail.gmail.com>

(on python-dev) Kristj?n V. J?nsson wrote:

> We have been using gc.get_objects() but it has several problems:
>  1) ... results in a list so long that it often kills the system.

In Py3K, should this also return an iterator, rather than a list?

In other words, is the dict.items change something specific to dicts,
or something that could happen with any large sequence?

> 2) There is no way to frame certain operations and get just those
> objects that were created during their execution.  ... storing the id()s of all objects
>... the ambiguity of id() in the face of objects being created and destroyed.

Changing id() to be something other than the address probably isn't
practical, but with the elimination of old-style classes, everything
should inherit from object.  Would it make sense to add optional
timestamping (or other alternative id-like attributes)?

Right now, the best solution I see is to use a debug build and a C
debugger to inspect the serial number that _PyObject_DebugMalloc
sticks after its FORBIDDENBYTE barrier.

-jJ

From guido at python.org  Wed Jan 24 21:19:13 2007
From: guido at python.org (Guido van Rossum)
Date: Wed, 24 Jan 2007 12:19:13 -0800
Subject: [Python-3000] gc, timestamping
In-Reply-To: <fb6fbf560701241042i6eef3aafqaba66391d331dd1e@mail.gmail.com>
References: <fb6fbf560701241042i6eef3aafqaba66391d331dd1e@mail.gmail.com>
Message-ID: <ca471dc20701241219p6aa03225m6a180b6d81f0537b@mail.gmail.com>

On 1/24/07, Jim Jewett <jimjjewett at gmail.com> wrote:
> (on python-dev) Kristj?n V. J?nsson wrote:
>
> > We have been using gc.get_objects() but it has several problems:
> >  1) ... results in a list so long that it often kills the system.
>
> In Py3K, should this also return an iterator, rather than a list?
>
> In other words, is the dict.items change something specific to dicts,
> or something that could happen with any large sequence?

It should be considered on a case-by-case basis, depending on how the
result is most commonly used, or perhaps what's the most useful
result. I don't know if it makes sense given the GC implementation to
do it in this case.

(For example, I looked briefly into making os.listdir() an iterator,
but decided against it because you'd have a file descriptor open while
iterating, and the list is never truly long.)

> > 2) There is no way to frame certain operations and get just those
> > objects that were created during their execution.  ... storing the id()s of all objects
> >... the ambiguity of id() in the face of objects being created and destroyed.
>
> Changing id() to be something other than the address probably isn't
> practical, but with the elimination of old-style classes, everything
> should inherit from object.  Would it make sense to add optional
> timestamping (or other alternative id-like attributes)?
>
> Right now, the best solution I see is to use a debug build and a C
> debugger to inspect the serial number that _PyObject_DebugMalloc
> sticks after its FORBIDDENBYTE barrier.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From tomerfiliba at gmail.com  Wed Jan 24 21:45:12 2007
From: tomerfiliba at gmail.com (tomer filiba)
Date: Wed, 24 Jan 2007 22:45:12 +0200
Subject: [Python-3000] new pickle semantics/API
Message-ID: <1d85506f0701241245s45f5ca3bh931d3d7d8a8320ec@mail.gmail.com>

i'm having great trouble in RPyC with pickling object proxies.
several users have asked for this feature, but no matter how hard
i try to "bend the truth", pickle always complains. it uses
type(obj) for the dispatching, which "uncovers" the object
is actually a proxy, rather than a real object.

recap: RPyC uses local proxies that refer to objects of a
remote interpreter (another process/machine).

if you'd noticed, every RPC framework has its own serializer.
for example banna/jelly in twisted and bunch of other
XML serializers, and what not.

for RPyC i wrote yet another serializer, but for different purposes,
so it's not relevant for the issue at hand.

what i want is a standard serialization *API*. the idea is that
any framework could make use of this API, and that it would
be generic enough to eliminate copy_reg and other misfortunes.
this also means the built in types should be familiarized with
this API.

- - - - - - - -

for example, currently the builtin types don't support __reduce__,
and require pickle to use it's own internal registry. moreover,
__reduce__ is very pickle-specific (i.e., it takes the protocol number).
what i'm after is an API for "simplifying" complex objects into
simpler parts.

here's the API i'm suggesting:

def __getstate__(self):
    # return a tuple of (type(self), obj), where obj is a simplified
    # version of self

@classmethod
def __setstate__(cls, state):
    # return an instance of cls, with the given state

well, you may already know these two, although their
semantics are different. but wait, there's more!

the idea is of having the following simple building blocks:
* integers (int/long)
* strings (str)
* arrays (tuples)

all picklable objects should be able to express themselves
as a collection of these building blocks. of course this will be
recursive, i.e., object X could simplify itself as object Y,
where object Y might go further simplification, until we are
left with building blocks only.

for example:
* int - return self
* float - string in the format "[+-]X.YYYe[+-]EEE"
* complex - two floats
* tuple - tuple of its simplified elements
* list - tuple of its simplified elements
* dict - a tuple of (key, value) tuples
* set - a tuple of its items
* file - raises TypeError("can't be simplified")

all in all, i choose to call that *simplification* rather than *serialization*,
as serialization is more about converting the simplified objects into a
sequence of bytes. my suggestion leaves that out for the
implementers of specific serializers.

so this is how a typical serializer (e.g., pickle) would be implemented:
* define its version of a "recursive simplifier"
* optionally use a "memo" to remember objects that were already
visited (so they would be serialized by reference rather than by value)
* define its flavor of converting ints, strings, and arrays to bytes
(binary, textual, etc. etc.)

- - - - - - - -

the default implementation of __getstate__, in object.__getstate__,
will simply return self.__dict__ and any self.__slots__

this removes the need for __reduce__, __reduce_ex__, and copy_reg,
and simplifies pickle greatly. it does require, however, adding
support for simplification for all builtin types... but this doesn't
call for much code:
    def PyList_GetState(self):
        state = tuple(PyObject_GetState(item) for item in self)
        return PyListType, state

also note that it makes the copy module much simpler:
    def copy(obj):
        state = obj.__getstate__()
        return type(obj).__setstate__(state)

- - - - - - - -

executive summary:
simplifying object serialization and copying by revising
__getstate__ and __setstate__, so that they return a
"simplified" version of the object.

this new mechanism should become an official API to
getting or setting the "contents" of objects (either builtin or
user-defined).

having this unified mechanism, pickling proxy objects would
work as expected.

if there's interest, i'll write a pep-like document to explain
all the semantics.



-tomer

From collinw at gmail.com  Wed Jan 24 22:22:17 2007
From: collinw at gmail.com (Collin Winter)
Date: Wed, 24 Jan 2007 15:22:17 -0600
Subject: [Python-3000] Pre-peps on raise and except changes (was:
	Warning for 2.6 and greater)
In-Reply-To: <ep6brv$p3c$1@sea.gmane.org>
References: <43aa6ff70701221445s5edb4b2do8c8cffdebc759c7@mail.gmail.com>
	<ep6brv$p3c$1@sea.gmane.org>
Message-ID: <43aa6ff70701241322o6967cb27s4bf27279caf354cd@mail.gmail.com>

On 1/23/07, Giovanni Bajo <rasky at develer.com> wrote:
> On 22/01/2007 23.45, Collin Winter wrote:
>
> > target, thus eliminating the reference cycle. The source-to-source
> > translation, as suggested by Phillip J. Eby [#except-translation]_ is
> > ::
> >
> >     try:
> >         ...
> >     except E as N:
> >         ...
> >     ...
> >
> > is translated to ::
> >
> >     try:
> >         ...
> >     except E as N:
> >         ...
> >         N = None
> >         del N
> >     ...
> >
>
> The mail you link (which matched my recollection of the thread) has an
> additional try/finally suite in the except block translation:
>
>     except ExcType, e:
>             try:
>                 # body
>             finally:
>                 e = None
>                 del e
>
> which is missing in the PEP. I think you should probably explain why the
> try/finally is not needed (if it really is not).

The try/finally is indeed needed; I just screwed up when copying it
out for the PEP : )

> Also, the second code snippet should probably use the "except E, N" form (no
> "as") to make more clear that you're speaking of the Python 2.x equivalent of
> the 3.x form.

Thanks! I've changed the "as" to "," and included a note that the
second code sample is Python 2.

Collin WInter

From collinw at gmail.com  Wed Jan 24 22:25:22 2007
From: collinw at gmail.com (Collin Winter)
Date: Wed, 24 Jan 2007 15:25:22 -0600
Subject: [Python-3000] Pre-peps on raise and except changes
In-Reply-To: <bbaeab100701231524o38aaf630n2ce43c6b767b9b8b@mail.gmail.com>
References: <5.1.1.6.0.20070122214504.04425a80@sparrow.telecommunity.com>
	<45B5F946.8070109@gmail.com>
	<43aa6ff70701231411y759f8b4jc829b7f1d425ec1f@mail.gmail.com>
	<bbaeab100701231434n1f49e704y32c0d19b3e992859@mail.gmail.com>
	<43aa6ff70701231503v5bc08d69n9d53bbaa3373c5c@mail.gmail.com>
	<bbaeab100701231524o38aaf630n2ce43c6b767b9b8b@mail.gmail.com>
Message-ID: <43aa6ff70701241325y43546eacwafcab2b5b1db41c3@mail.gmail.com>

On 1/23/07, Brett Cannon <brett at python.org> wrote:
> I just realized I misread your paragraph and took E to represent an
> exception class, not a nebulous object that could be an exception
> class or instance.

That's what I figured. Is it clearer if E is changed to EXCEPTION?

"""
2. ``raise EXCEPTION`` is used to raise a new exception. This form has
   two sub-variants: ``EXCEPTION`` may be either an instance of
   ``BaseException`` or a subclass of ``BaseException`` [#pep352]_.
   If ``EXCEPTION`` is a subclass, it will be called with no arguments
   to obtain an exception instance.
"""

Thanks,
Collin Winter

From aahz at pythoncraft.com  Thu Jan 25 00:54:42 2007
From: aahz at pythoncraft.com (Aahz)
Date: Wed, 24 Jan 2007 15:54:42 -0800
Subject: [Python-3000] new pickle semantics/API
In-Reply-To: <1d85506f0701241245s45f5ca3bh931d3d7d8a8320ec@mail.gmail.com>
References: <1d85506f0701241245s45f5ca3bh931d3d7d8a8320ec@mail.gmail.com>
Message-ID: <20070124235441.GA5475@panix.com>

On Wed, Jan 24, 2007, tomer filiba wrote:
>
> i'm having great trouble in RPyC with pickling object proxies. several
> users have asked for this feature, but no matter how hard i try to
> "bend the truth", pickle always complains. it uses type(obj) for the
> dispatching, which "uncovers" the object is actually a proxy, rather
> than a real object.

This should go to python-ideas, I think
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

Help a hearing-impaired person: http://rule6.info/hearing.html

From tomerfiliba at gmail.com  Thu Jan 25 11:03:17 2007
From: tomerfiliba at gmail.com (tomer filiba)
Date: Thu, 25 Jan 2007 12:03:17 +0200
Subject: [Python-3000] new pickle semantics/API
In-Reply-To: <20070124235441.GA5475@panix.com>
References: <1d85506f0701241245s45f5ca3bh931d3d7d8a8320ec@mail.gmail.com>
	<20070124235441.GA5475@panix.com>
Message-ID: <1d85506f0701250203w342080bcq6f3da853551a597f@mail.gmail.com>

what exactly is the criterion for posting to python-ideas?
i don't think it's off-topic.


-tomer

On 1/25/07, Aahz <aahz at pythoncraft.com> wrote:
> On Wed, Jan 24, 2007, tomer filiba wrote:
> >
> > i'm having great trouble in RPyC with pickling object proxies. several
> > users have asked for this feature, but no matter how hard i try to
> > "bend the truth", pickle always complains. it uses type(obj) for the
> > dispatching, which "uncovers" the object is actually a proxy, rather
> > than a real object.
>
> This should go to python-ideas, I think
> --
> Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/
>
> Help a hearing-impaired person: http://rule6.info/hearing.html
>

From jimjjewett at gmail.com  Thu Jan 25 15:55:31 2007
From: jimjjewett at gmail.com (Jim Jewett)
Date: Thu, 25 Jan 2007 09:55:31 -0500
Subject: [Python-3000] gc, timestamping
In-Reply-To: <45B7F447.5000108@develer.com>
References: <fb6fbf560701241042i6eef3aafqaba66391d331dd1e@mail.gmail.com>
	<ca471dc20701241219p6aa03225m6a180b6d81f0537b@mail.gmail.com>
	<45B7F447.5000108@develer.com>
Message-ID: <fb6fbf560701250655y7f9a3498t9cfa92da764598db@mail.gmail.com>

On 1/24/07, Giovanni Bajo <rasky at develer.com> wrote:
> On 24/01/2007 21.19, Guido van Rossum wrote:

> > (For example, I looked briefly into making os.listdir() an iterator,
> > but decided against it because you'd have a file descriptor open while
> > iterating, and the list is never truly long.)

> What is so wrong about having a file descriptor open while iterating? I can't
> help but thinking of "for L in open(...)" which works out pretty well.

Directories are a special case.  Iteration over a file probably won't
cause you to need additional file descriptions.  Iteration over a
directory often will, and the function is likely to be called
recursively, like the various *.walk functions.  Using up one file
descriptor to keep memory use low is reasonable.  Using up ten may not
be, unless directories are truly huge.

-jJ

From aahz at pythoncraft.com  Thu Jan 25 16:21:42 2007
From: aahz at pythoncraft.com (Aahz)
Date: Thu, 25 Jan 2007 07:21:42 -0800
Subject: [Python-3000] new pickle semantics/API
In-Reply-To: <1d85506f0701250203w342080bcq6f3da853551a597f@mail.gmail.com>
References: <1d85506f0701241245s45f5ca3bh931d3d7d8a8320ec@mail.gmail.com>
	<20070124235441.GA5475@panix.com>
	<1d85506f0701250203w342080bcq6f3da853551a597f@mail.gmail.com>
Message-ID: <20070125152142.GA4089@panix.com>

On Thu, Jan 25, 2007, tomer filiba wrote:
>
> what exactly is the criterion for posting to python-ideas?
> i don't think it's off-topic.

Per previous discussion, python-ideas is for thrashing out ideas until
they're ready for a PEP or a patch.  python-ideas is for *both* 2.x and
3.x to make it easier to figure out where features should go without
cross-posting.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

Help a hearing-impaired person: http://rule6.info/hearing.html

From brett at python.org  Thu Jan 25 20:09:24 2007
From: brett at python.org (Brett Cannon)
Date: Thu, 25 Jan 2007 11:09:24 -0800
Subject: [Python-3000] Pre-peps on raise and except changes
In-Reply-To: <43aa6ff70701241325y43546eacwafcab2b5b1db41c3@mail.gmail.com>
References: <5.1.1.6.0.20070122214504.04425a80@sparrow.telecommunity.com>
	<45B5F946.8070109@gmail.com>
	<43aa6ff70701231411y759f8b4jc829b7f1d425ec1f@mail.gmail.com>
	<bbaeab100701231434n1f49e704y32c0d19b3e992859@mail.gmail.com>
	<43aa6ff70701231503v5bc08d69n9d53bbaa3373c5c@mail.gmail.com>
	<bbaeab100701231524o38aaf630n2ce43c6b767b9b8b@mail.gmail.com>
	<43aa6ff70701241325y43546eacwafcab2b5b1db41c3@mail.gmail.com>
Message-ID: <bbaeab100701251109l2e3abc07p952df34ac4125fa6@mail.gmail.com>

On 1/24/07, Collin Winter <collinw at gmail.com> wrote:
> On 1/23/07, Brett Cannon <brett at python.org> wrote:
> > I just realized I misread your paragraph and took E to represent an
> > exception class, not a nebulous object that could be an exception
> > class or instance.
>
> That's what I figured. Is it clearer if E is changed to EXCEPTION?
>
> """
> 2. ``raise EXCEPTION`` is used to raise a new exception. This form has
>    two sub-variants: ``EXCEPTION`` may be either an instance of
>    ``BaseException`` or a subclass of ``BaseException`` [#pep352]_.
>    If ``EXCEPTION`` is a subclass, it will be called with no arguments
>    to obtain an exception instance.
> """
>

I guess.  I would just say "is used to raise a new exception, either a
class or instance" and then explain the restrictions.  But this could
easily just be a quirk of my head.  =)

-Brett

From kristjan at ccpgames.com  Thu Jan 25 10:17:02 2007
From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_V._J=F3nsson?=)
Date: Thu, 25 Jan 2007 09:17:02 +0000
Subject: [Python-3000] gc, timestamping
In-Reply-To: <fb6fbf560701241042i6eef3aafqaba66391d331dd1e@mail.gmail.com>
References: <fb6fbf560701241042i6eef3aafqaba66391d331dd1e@mail.gmail.com>
Message-ID: <4E9372E6B2234D4F859320D896059A9504F58E8C@exchis.ccp.ad.local>


Well, the gc linked list of objects is so volatile that doubt that it is simple to create an iterator over it.  And the semantics of such an iterator would be hard to define, with code running while a sequence of live objects is being traversed.

I am still pursuing my "creation hook" idea, primariliy because getting a list of live objects on a running EVE server win 1Gb of allocated python memory will stall it and then fail due to allocation error.

I'll let you know why I get crashes in case I find out.

Cheers,
Kristj?n

-----Original Message-----
From: Jim Jewett [mailto:jimjjewett at gmail.com]
Sent: 24. jan?ar 2007 18:42
To: Python 3000
Cc: Kristj?n V. J?nsson
Subject: gc, timestamping

(on python-dev) Kristj?n V. J?nsson wrote:

> We have been using gc.get_objects() but it has several problems:
>  1) ... results in a list so long that it often kills the system.

In Py3K, should this also return an iterator, rather than a list?

In other words, is the dict.items change something specific to dicts,
or something that could happen with any large sequence?

> 2) There is no way to frame certain operations and get just those
> objects that were created during their execution.  ... storing the id()s of all objects
>... the ambiguity of id() in the face of objects being created and destroyed.

Changing id() to be something other than the address probably isn't
practical, but with the elimination of old-style classes, everything
should inherit from object.  Would it make sense to add optional
timestamping (or other alternative id-like attributes)?

Right now, the best solution I see is to use a debug build and a C
debugger to inspect the serial number that _PyObject_DebugMalloc
sticks after its FORBIDDENBYTE barrier.

-jJ

From collinw at gmail.com  Sat Jan 27 02:59:36 2007
From: collinw at gmail.com (Collin Winter)
Date: Fri, 26 Jan 2007 19:59:36 -0600
Subject: [Python-3000] Pre-peps on raise and except changes
In-Reply-To: <45BAAC1E.8050506@canterbury.ac.nz>
References: <5.1.1.6.0.20070122214504.04425a80@sparrow.telecommunity.com>
	<45B5F946.8070109@gmail.com>
	<43aa6ff70701231411y759f8b4jc829b7f1d425ec1f@mail.gmail.com>
	<45BAAC1E.8050506@canterbury.ac.nz>
Message-ID: <43aa6ff70701261759v4bfa81e3s910a725ce007a021@mail.gmail.com>

On 1/26/07, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Collin Winter wrote:
> > Does this language work for you?
> >
> > 2. ``raise E`` (with a single argument) is used to raise a new
> >    exception. This form has two sub-variants: ``E`` may be either an
> >    instance of ``BaseException`` [#pep352]_ or a subclass of
> >    ``BaseException``.
>
> That wording doesn't seem to cover the case where
> E is an instance of a subclass of BaseException.

>>> class A: pass
>>> class B(A): pass
>>> isinstance(B(), A)
True

Collin Winter

From greg.ewing at canterbury.ac.nz  Sat Jan 27 02:34:27 2007
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 27 Jan 2007 14:34:27 +1300
Subject: [Python-3000] gc, timestamping
In-Reply-To: <ca471dc20701241219p6aa03225m6a180b6d81f0537b@mail.gmail.com>
References: <fb6fbf560701241042i6eef3aafqaba66391d331dd1e@mail.gmail.com>
	<ca471dc20701241219p6aa03225m6a180b6d81f0537b@mail.gmail.com>
Message-ID: <45BAAC23.8020505@canterbury.ac.nz>

Guido van Rossum wrote:
> On 1/24/07, Jim Jewett <jimjjewett at gmail.com> wrote:
> 
>> (on python-dev) Kristj?n V. J?nsson wrote:
>>
>>> We have been using gc.get_objects() but it has several problems:
>>> 1) ... results in a list so long that it often kills the system.
>>
>> In Py3K, should this also return an iterator, rather than a list?

> I don't know if it makes sense given the GC implementation to
> do it in this case.

I'd say it almost certainly doesn't, since the list of
objects would be changing in all sorts of ways while you
were iterating over it.

--
Greg


From greg.ewing at canterbury.ac.nz  Sat Jan 27 02:34:23 2007
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 27 Jan 2007 14:34:23 +1300
Subject: [Python-3000] Pre-peps on raise and except changes
In-Reply-To: <fb6fbf560701231522h3fea803fnca23ca8714ccf881@mail.gmail.com>
References: <5.1.1.6.0.20070122214504.04425a80@sparrow.telecommunity.com>
	<45B5F946.8070109@gmail.com>
	<43aa6ff70701231411y759f8b4jc829b7f1d425ec1f@mail.gmail.com>
	<fb6fbf560701231522h3fea803fnca23ca8714ccf881@mail.gmail.com>
Message-ID: <45BAAC1F.8030507@canterbury.ac.nz>

Jim Jewett wrote:
> it even clears up the pathological
> case of an object which is both a subclass and an instance of
> BaseException.

Is that possible???

--
Greg


From greg.ewing at canterbury.ac.nz  Sat Jan 27 02:34:22 2007
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 27 Jan 2007 14:34:22 +1300
Subject: [Python-3000] Pre-peps on raise and except changes
In-Reply-To: <43aa6ff70701231411y759f8b4jc829b7f1d425ec1f@mail.gmail.com>
References: <5.1.1.6.0.20070122214504.04425a80@sparrow.telecommunity.com>
	<45B5F946.8070109@gmail.com>
	<43aa6ff70701231411y759f8b4jc829b7f1d425ec1f@mail.gmail.com>
Message-ID: <45BAAC1E.8050506@canterbury.ac.nz>

Collin Winter wrote:

> Does this language work for you?
> 
> 2. ``raise E`` (with a single argument) is used to raise a new
>    exception. This form has two sub-variants: ``E`` may be either an
>    instance of ``BaseException`` [#pep352]_ or a subclass of
>    ``BaseException``.

That wording doesn't seem to cover the case where
E is an instance of a subclass of BaseException.

--
Greg


From greg.ewing at canterbury.ac.nz  Sat Jan 27 03:19:26 2007
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 27 Jan 2007 15:19:26 +1300
Subject: [Python-3000] Pre-peps on raise and except changes
In-Reply-To: <43aa6ff70701261759v4bfa81e3s910a725ce007a021@mail.gmail.com>
References: <5.1.1.6.0.20070122214504.04425a80@sparrow.telecommunity.com>
	<45B5F946.8070109@gmail.com>
	<43aa6ff70701231411y759f8b4jc829b7f1d425ec1f@mail.gmail.com>
	<45BAAC1E.8050506@canterbury.ac.nz>
	<43aa6ff70701261759v4bfa81e3s910a725ce007a021@mail.gmail.com>
Message-ID: <45BAB6AE.7010306@canterbury.ac.nz>

Collin Winter wrote:
> On 1/26/07, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:

>> That wording doesn't seem to cover the case where
>> E is an instance of a subclass of BaseException.
> 
>>>> class A: pass
>>>> class B(A): pass
>>>> isinstance(B(), A)

Technically I suppose it's right, but it doesn't sound
very clear to me.

Maybe something like

    E may be an exception class or an instance of an
    exception class. Valid exception classes are
    BaseException and its subclasses.

--
Greg


From collinw at gmail.com  Sat Jan 27 03:41:18 2007
From: collinw at gmail.com (Collin Winter)
Date: Fri, 26 Jan 2007 20:41:18 -0600
Subject: [Python-3000] Pre-peps on raise and except changes
In-Reply-To: <45BAB6AE.7010306@canterbury.ac.nz>
References: <5.1.1.6.0.20070122214504.04425a80@sparrow.telecommunity.com>
	<45B5F946.8070109@gmail.com>
	<43aa6ff70701231411y759f8b4jc829b7f1d425ec1f@mail.gmail.com>
	<45BAAC1E.8050506@canterbury.ac.nz>
	<43aa6ff70701261759v4bfa81e3s910a725ce007a021@mail.gmail.com>
	<45BAB6AE.7010306@canterbury.ac.nz>
Message-ID: <43aa6ff70701261841q307835bfs8bd21d629bf9b070@mail.gmail.com>

On 1/26/07, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Collin Winter wrote:
> > On 1/26/07, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
>
> >> That wording doesn't seem to cover the case where
> >> E is an instance of a subclass of BaseException.
> >
> >>>> class A: pass
> >>>> class B(A): pass
> >>>> isinstance(B(), A)
>
> Technically I suppose it's right, but it doesn't sound
> very clear to me.
>
> Maybe something like
>
>     E may be an exception class or an instance of an
>     exception class. Valid exception classes are
>     BaseException and its subclasses.

I'll use that.

Thanks,
Collin Winter

From ncoghlan at gmail.com  Sat Jan 27 16:09:34 2007
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 28 Jan 2007 01:09:34 +1000
Subject: [Python-3000] Pre-peps on raise and except changes
In-Reply-To: <45BAAC1F.8030507@canterbury.ac.nz>
References: <5.1.1.6.0.20070122214504.04425a80@sparrow.telecommunity.com>	<45B5F946.8070109@gmail.com>	<43aa6ff70701231411y759f8b4jc829b7f1d425ec1f@mail.gmail.com>	<fb6fbf560701231522h3fea803fnca23ca8714ccf881@mail.gmail.com>
	<45BAAC1F.8030507@canterbury.ac.nz>
Message-ID: <45BB6B2E.6070609@gmail.com>

Greg Ewing wrote:
> Jim Jewett wrote:
>> it even clears up the pathological
>> case of an object which is both a subclass and an instance of
>> BaseException.
> 
> Is that possible???

Strange but true (Python 2.4):

 >>> class DumbMeta(type, Exception): pass
...
 >>> class WTF(object, Exception):
...     __metaclass__ = DumbMeta
...
 >>> isinstance(WTF, Exception)
True
 >>> issubclass(WTF, Exception)
True

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From sluggoster at gmail.com  Sun Jan 28 11:53:35 2007
From: sluggoster at gmail.com (Mike Orr)
Date: Sun, 28 Jan 2007 02:53:35 -0800
Subject: [Python-3000] Unipath package
Message-ID: <6e9196d20701280253h6c0b887dy5fec85e55c76c0c@mail.gmail.com>

I finally finished my path package (Unipath) and put it in the Cheeseshop.
http://sluggo.scrapping.cc/python/unipath/

There's a Path class for pathname calculations, and a FSPath subclass
for filesystem calls.  I'm hoping Path -- or something resembling it
-- will find its way into os.path in Python 2.6 or 3.0.  FSPath is
full of convenience methods so it may not be everybody's cup of tea,
but perhaps something similar can go into Python in the farther future

Unipath is an early alpha release so the API may change as it gets
more real-world use.  There's an extensive unittest suite, which
passes on Python 2.5 and 2.4.4 on Linux.  Windows and Macintosh
testers are needed.

Following are highlights from the python-3000 discussion and deviations from it:

-  Path subclasses unicode, or str if the platform can't handle
Unicode pathnames.

- FSPath subclasses Path.  This allows you to do "from unipath import
FSPath as Path" and pretend there's only one class.  I find this much
more convenient in applications, so you don't have to cast objects
back and forth between the classes.  Also, it just became infeasable
not to let them inherit, because so many methods call other methods.

- Nevertheless, you can use Path alone and rest assured it will never
touch the filesystem.  You can even use Path objects with os.*
functions if you are so heretically inclined.  If Path is accepted
into the stdlib, FSPath will inherit it from there.

- I tried splitting FSPath into several mixins according to type of
operation, but the need for methods to call other methods in a
different category sabotaged that too.  So FSPath proudly has about
fifty methods.  (Path has 10 public methods and 4 properties.)

- The dirname property is called .parent.  The basename property is
.name.  The extension property is .ext.  The name without extension is
.stem.

- .components() returns a list of directory components.  The first
component is "/", a Windows drive root, a UNC share, or "" for a
relative path.  .split_root() returns the root and the rest.

- PosixPath, NTPath, and MacPath are Path subclasses using a specific
path library.  This allows you to express non-native paths and convert
paths.  Passing a relative foreign path to a *Path constructor
converts it to the destination type.  Passing an absolute foreign path
is an error, because there's no sane way to interpret "C:\\" on Posix
or "/" on Windows.  I'm skeptical whether this non-native support is
really worth it, because .norm() and .norm_case() already convert
slashes to backslashes (which is what Talin really wanted to do --
have Posix paths in a config file and automatically convert them to
the native format).  And if Python is going to drop Mac OS 9 soon then
MacPath is obsolete.  So maybe this non-native path code will prove
less than useful and will be deleted.  On the other hand, if someone
is burning to write a zippath or ftppath library, you can use it with
this.

- Setting Path.auto_norm to true will automatically normalize all
paths on construction.  I'm not sure if this should be the default,
because normalizing may produce the wrong path (e.g., if it contains a
symlink).  You can always pass norm=True or norm=False to the
constructor to enable/disable it.

- p.child("subdir", "grandkid") is Glyph's favorite "safe join" method
that prevents creating a path pointing above 'p'.  You can use it as a
.joinpath if you don't mind this restriction.  The constructor allows
multiple positional arguments, which are joined using os.path.join().

- Listing is p.listdir(pattern=None, filter=None, names_only=False).
This returns a non-recursive list of paths.  If 'names_only' is true
it returns the same as os.listdir().  p.walk(pattern=None,
filter=None, top_down=True) is the recursive counterpart, yielding
paths.  In both cases 'pattern' is a glob pattern, and 'filter' is one
of the constants (FILES, DIRS, LINKS, FILES_NO_LINKS, DIRS_NO_LINKS,
DEAD_LINKS) or a custom boolean function.  I spent a lot of time going
back and forth between Orendorff's six listing methods and Raphael's
one, and finally decided on this, partly because I'm not satisfied
with how either Orendorff's class or Raphael's or os.walk() handle
symlinks -- sometimes you want to ignore the links and then iterate
them separately.

- .read_file(mode) and .write_file(content, mode) are a compromise
between Orendorff's seven methods and purists' desire for zero
methods.

- .mkdir(), .rmdir(), .rmtree(), .copy(), .copy_stat(), .copy_tree(),
and .move() are more fancy than their stdlib/Orendorff counterparts.
They silently succeed if the operation is already done, and have
arguments to smartly create/delete intermediate directories, etc.

- Two extra functions are in 'unipath.tools'.  'dict2dir' creates a
directory hierarchy modeled after a dict.  'dump_path' displays an
ASCII tree of a directory hierarchy, with file sizes and symlink
targets showing.

Enjoy!  and please provide feedback.

--Mike Orr <sluggoster at gmail.com>

From thobes at gmail.com  Sun Jan 28 15:06:25 2007
From: thobes at gmail.com (Tobias Ivarsson)
Date: Sun, 28 Jan 2007 15:06:25 +0100
Subject: [Python-3000] [Python-Dev] Unipath package
In-Reply-To: <6e9196d20701280253h6c0b887dy5fec85e55c76c0c@mail.gmail.com>
References: <6e9196d20701280253h6c0b887dy5fec85e55c76c0c@mail.gmail.com>
Message-ID: <9997d5e60701280606m7c5c1656k9ca1ed1039e09a70@mail.gmail.com>

Hi!

I am a MSc of Computer Engineering student from Sweden. A frequent reader of
python-3000, and an occasional reader of python-dev for about a year now. I
have wanted to get involved in the development of python for a while but
haven't found the time before. Now, when I'm in the process of trying to
figure out what project to do for my thesis, I have actually found myself
having more time than before. Therefore I thought that now is a better time
than never to get involved. After that compulsory first-post-introduction of
myself, lets get to the reason I choose to send the post at all.

I noticed that you wanted Mac OS tests, here are the test results from my
Mac OS X (10.4), running on an Intel Core Duo MacBook, using python 2.5:

FF...F....................................................
======================================================================
FAIL: test_absolute (__main__.TestCalculatingPaths)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "./test.py", line 235, in test_absolute
    eq(p1, p2)
AssertionError: FSPath('/private/tmp/tmpH9fppC/images') !=
FSPath('/tmp/tmpH9fppC/images')

======================================================================
FAIL: test_chdir_absolute_relative (__main__.TestCalculatingPaths)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "./test.py", line 220, in test_chdir_absolute_relative
    eq(FSPath.cwd(), self.d)
AssertionError: FSPath('/private/tmp/tmpRJzqbt') != FSPath('/tmp/tmpRJzqbt')

======================================================================
FAIL: test_relative (__main__.TestCalculatingPaths)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "./test.py", line 240, in test_relative
    eq(p, "images")
AssertionError: FSPath('../../../tmp/tmpdw9juF/images') != 'images'

----------------------------------------------------------------------
Ran 58 tests in 1.587s


Please let me know if there are any further tests or improvements I could
help you with.

/Tobias

On 1/28/07, Mike Orr <sluggoster at gmail.com> wrote:
>
> I finally finished my path package (Unipath) and put it in the Cheeseshop.
> http://sluggo.scrapping.cc/python/unipath/
>
> There's a Path class for pathname calculations, and a FSPath subclass
> for filesystem calls.  I'm hoping Path -- or something resembling it
> -- will find its way into os.path in Python 2.6 or 3.0.  FSPath is
> full of convenience methods so it may not be everybody's cup of tea,
> but perhaps something similar can go into Python in the farther future
>
> Unipath is an early alpha release so the API may change as it gets
> more real-world use.  There's an extensive unittest suite, which
> passes on Python 2.5 and 2.4.4 on Linux.  Windows and Macintosh
> testers are needed.
>
> Following are highlights from the python-3000 discussion and deviations
> from it:
>
> -  Path subclasses unicode, or str if the platform can't handle
> Unicode pathnames.
>
> - FSPath subclasses Path.  This allows you to do "from unipath import
> FSPath as Path" and pretend there's only one class.  I find this much
> more convenient in applications, so you don't have to cast objects
> back and forth between the classes.  Also, it just became infeasable
> not to let them inherit, because so many methods call other methods.
>
> - Nevertheless, you can use Path alone and rest assured it will never
> touch the filesystem.  You can even use Path objects with os.*
> functions if you are so heretically inclined.  If Path is accepted
> into the stdlib, FSPath will inherit it from there.
>
> - I tried splitting FSPath into several mixins according to type of
> operation, but the need for methods to call other methods in a
> different category sabotaged that too.  So FSPath proudly has about
> fifty methods.  (Path has 10 public methods and 4 properties.)
>
> - The dirname property is called .parent.  The basename property is
> .name.  The extension property is .ext.  The name without extension is
> .stem.
>
> - .components() returns a list of directory components.  The first
> component is "/", a Windows drive root, a UNC share, or "" for a
> relative path.  .split_root() returns the root and the rest.
>
> - PosixPath, NTPath, and MacPath are Path subclasses using a specific
> path library.  This allows you to express non-native paths and convert
> paths.  Passing a relative foreign path to a *Path constructor
> converts it to the destination type.  Passing an absolute foreign path
> is an error, because there's no sane way to interpret "C:\\" on Posix
> or "/" on Windows.  I'm skeptical whether this non-native support is
> really worth it, because .norm() and .norm_case() already convert
> slashes to backslashes (which is what Talin really wanted to do --
> have Posix paths in a config file and automatically convert them to
> the native format).  And if Python is going to drop Mac OS 9 soon then
> MacPath is obsolete.  So maybe this non-native path code will prove
> less than useful and will be deleted.  On the other hand, if someone
> is burning to write a zippath or ftppath library, you can use it with
> this.
>
> - Setting Path.auto_norm to true will automatically normalize all
> paths on construction.  I'm not sure if this should be the default,
> because normalizing may produce the wrong path (e.g., if it contains a
> symlink).  You can always pass norm=True or norm=False to the
> constructor to enable/disable it.
>
> - p.child("subdir", "grandkid") is Glyph's favorite "safe join" method
> that prevents creating a path pointing above 'p'.  You can use it as a
> .joinpath if you don't mind this restriction.  The constructor allows
> multiple positional arguments, which are joined using os.path.join().
>
> - Listing is p.listdir(pattern=None, filter=None, names_only=False).
> This returns a non-recursive list of paths.  If 'names_only' is true
> it returns the same as os.listdir().  p.walk(pattern=None,
> filter=None, top_down=True) is the recursive counterpart, yielding
> paths.  In both cases 'pattern' is a glob pattern, and 'filter' is one
> of the constants (FILES, DIRS, LINKS, FILES_NO_LINKS, DIRS_NO_LINKS,
> DEAD_LINKS) or a custom boolean function.  I spent a lot of time going
> back and forth between Orendorff's six listing methods and Raphael's
> one, and finally decided on this, partly because I'm not satisfied
> with how either Orendorff's class or Raphael's or os.walk() handle
> symlinks -- sometimes you want to ignore the links and then iterate
> them separately.
>
> - .read_file(mode) and .write_file(content, mode) are a compromise
> between Orendorff's seven methods and purists' desire for zero
> methods.
>
> - .mkdir(), .rmdir(), .rmtree(), .copy(), .copy_stat(), .copy_tree(),
> and .move() are more fancy than their stdlib/Orendorff counterparts.
> They silently succeed if the operation is already done, and have
> arguments to smartly create/delete intermediate directories, etc.
>
> - Two extra functions are in 'unipath.tools'.  'dict2dir' creates a
> directory hierarchy modeled after a dict.  'dump_path' displays an
> ASCII tree of a directory hierarchy, with file sizes and symlink
> targets showing.
>
> Enjoy!  and please provide feedback.
>
> --Mike Orr <sluggoster at gmail.com>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/thobes%40gmail.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20070128/7299dcf1/attachment.htm 

From brett at python.org  Mon Jan 29 06:08:46 2007
From: brett at python.org (Brett Cannon)
Date: Sun, 28 Jan 2007 21:08:46 -0800
Subject: [Python-3000] how should we handle changes to the C API?
Message-ID: <bbaeab100701282108r6b01e0c5ge4dba673c5894fc7@mail.gmail.com>

I know it has been said that extension modules will, at minimum, need
to recompile.  But what about changes to the API that are semantic?

What I am thinking of in general is getting rid of deprecated
functions and replacing them with their *Ex equilvalents (e.g.,
replace PyErr_Warn() with PyErr_WarnEx() and thus only having one
warning function).  Do we want a PEP specific of what C functions are
going to change and for what reason?

My specific need is that PyErr_GivenExceptionMatches() does not have
an exception return value.  This sucks for me in 2.6 for deprecating
catching string exceptions, but it sucks more in 3.0 since only
subclasses of BaseException can be raised.  But not allowing -1 to
represent that an error occurred is a pain for anyone who wants to
properly use the function.

This means that in 3.0 I want to change the meaning of
PyErr_GivenExceptionMatches(), but obviously this single change does
not need its own PEP.  Then again, there are a bunch of *Ex() modules
that could stand to be renamed and ditch the old way.

Should I just toss all of this into PEP 3100 (and add a C API
section), or should I start a new PEP that lists all general C API
changes (assuming that people are willing to rename C functions)?

-Brett

From greg.ewing at canterbury.ac.nz  Mon Jan 29 07:31:39 2007
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 29 Jan 2007 19:31:39 +1300
Subject: [Python-3000] how should we handle changes to the C API?
In-Reply-To: <bbaeab100701282108r6b01e0c5ge4dba673c5894fc7@mail.gmail.com>
References: <bbaeab100701282108r6b01e0c5ge4dba673c5894fc7@mail.gmail.com>
Message-ID: <45BD94CB.6060107@canterbury.ac.nz>

Brett Cannon wrote:

> My specific need is that PyErr_GivenExceptionMatches() does not have
> an exception return value.  This sucks for me in 2.6 for deprecating
> catching string exceptions, but it sucks more in 3.0 since only
> subclasses of BaseException can be raised.

Given that this would only be really useful during
the transition to wean people off string and/or non
BaseExceptions, do we really want to make a permanent
change to the API with all the upheaval it would
cause?

Seems to me you can still generate a warning for
these things, you just wouldn't be able to escalate
them into errors.

--
Greg

From brett at python.org  Mon Jan 29 18:51:40 2007
From: brett at python.org (Brett Cannon)
Date: Mon, 29 Jan 2007 09:51:40 -0800
Subject: [Python-3000] how should we handle changes to the C API?
In-Reply-To: <45BD94CB.6060107@canterbury.ac.nz>
References: <bbaeab100701282108r6b01e0c5ge4dba673c5894fc7@mail.gmail.com>
	<45BD94CB.6060107@canterbury.ac.nz>
Message-ID: <bbaeab100701290951o5e97b97cm41ac3db30a15e9fc@mail.gmail.com>

On 1/28/07, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Brett Cannon wrote:
>
> > My specific need is that PyErr_GivenExceptionMatches() does not have
> > an exception return value.  This sucks for me in 2.6 for deprecating
> > catching string exceptions, but it sucks more in 3.0 since only
> > subclasses of BaseException can be raised.
>
> Given that this would only be really useful during
> the transition to wean people off string and/or non
> BaseExceptions, do we really want to make a permanent
> change to the API with all the upheaval it would
> cause?
>
> Seems to me you can still generate a warning for
> these things, you just wouldn't be able to escalate
> them into errors.
>

Well, I already solved this in Python/ceval.c, it just doesn't do
anything for C API function calls (I raise the exception when the
comparison opcode is handled).

I was more generally wondering what the plan was for transitioning any
C API changes (if we were even going to do that level of transition).

-Brett

From brett at python.org  Tue Jan 30 00:40:49 2007
From: brett at python.org (Brett Cannon)
Date: Mon, 29 Jan 2007 15:40:49 -0800
Subject: [Python-3000] reference leak when pressing Enter at interpreter
	prompt
Message-ID: <bbaeab100701291540k1e06fce3qa1553257f65dc54a@mail.gmail.com>

Seems two references are leaking every time you press Enter at the
interpreter prompt in a debug build.  Anyone have an inkling of who
introduced it?

-Brett

From g.brandl at gmx.net  Wed Jan 31 08:51:42 2007
From: g.brandl at gmx.net (Georg Brandl)
Date: Wed, 31 Jan 2007 08:51:42 +0100
Subject: [Python-3000] reference leak when pressing Enter at interpreter
	prompt
In-Reply-To: <bbaeab100701291540k1e06fce3qa1553257f65dc54a@mail.gmail.com>
References: <bbaeab100701291540k1e06fce3qa1553257f65dc54a@mail.gmail.com>
Message-ID: <epphqf$oro$1@sea.gmane.org>

Brett Cannon schrieb:
> Seems two references are leaking every time you press Enter at the
> interpreter prompt in a debug build.  Anyone have an inkling of who
> introduced it?

If anyone wants to look into it:
It was rev. 53421, the merging of the long-int-unification branch.

Georg


From larry at hastings.org  Wed Jan 31 11:40:52 2007
From: larry at hastings.org (Larry Hastings)
Date: Wed, 31 Jan 2007 02:40:52 -0800
Subject: [Python-3000] Poll: Lazy Unicode Strings For Py3k
Message-ID: <45C07234.8070808@hastings.org>



I'd like to start a (hopefully final) round of discussion on
the "lazy strings" series of patches.  What follows is a summary
on the current state of the patches, followed by five poll
questions.

I apologize in advance for the length of this posting.


A NOTE BEFORE WE BEGIN

The "lazy strings" patches are actually two separate patches:
"lazy concatentation" and "lazy slices".  I did them together
to save work, but I fear I have created some confusion by doing
so.  *Please* keep in mind that "lazy concatenation" and "lazy
slices" are separate things, and should be independently
considered for inclusion.


"LAZY CONCATENATION"

Lazy string concatenation changes the + operator when adding
two strings.  Instead of returning the result of the concatenation,
it returns a placeholder (a PyUnicodeConcatenationObject) that
represents the concatenation.  The placeholder holds references
to the two strings; the first time any caller asks for the
string's value, the string is "rendered": memory is allocated
for the string, its value is copied in from the referenced strings
(recursively only where necessary), the references to the strings
are freed, and the newly computed value is returned.

Lazy concatenation changes the behavior of the Python C API in two
subtle ways:
1) All C API users asking for the value of a string *must* use
   the macro PyUnicode_AS_UNICODE() or the function
   PyUnicode_AsUnicode().  It is no longer permissable to
   directly access the object's "str" member.
2) It is now possible for PyUnicode_AS_UNICODE() and
   PyUnicode_AsUnicode() to *fail*,  returning NULL under
   low-memory conditions.  All code calling either of these
   would have to be amended to check for NULL; few of these
   checks have been added as of yet.  Josiah Carlson suggests
   there are 400+ such places in the Py3k source tree alone.

The implementation also adds complexity underlying a
PyUnicodeObject, though this is effectively hidden from anyone
not peering into "Objects/unicodeobject.c".

With this patch in place, Unicode string concatenation is
approximately 300% faster.  The Pybench "ConcatUnicode" benchmark
finishes in 1/4 the time when compared to an unpatched Py3k.  This
makes using + for string concatenation fast enough that the Python
idiom of "".join([]) is no longer necessary, thus making + the
"one obvious way to do it".


"LAZY SLICES"

Lazy string slicing changes the slicing operator when acting
upon a string and all string methods that return a slice of
the string (split(), strip(), partition(), etc).  Instead of
returning the result of the slice, it returns a placeholder
(a PyUnicodeSliceObject) that represents the slice.  The
placeholder holds a reference to the original string, and the
desired slice's start and end offsets.  (Lazy slice objects only
support slices with a stride of 1; slices with a stride greater
than 1 are computed directly as before.)  The first time any
caller asks for the value of the string, the slice is
"rendered", and its value is returned.

This by itself would not speed anything up; it would actually be
a net slowdown.  The speedup is achieved by avoiding rendering
wherever possible.  Most Unicode string processing is done entirely
within "Objects/unicodeobject.c", and it is relatively simple to
change code within this file such that it operates on *unrendered*
slices.  With such changes in place, most slices are never rendered,
thus obviating the need for allocating memory and memcpy()ing to
store the slice's value.

Lazy concatenation changes the behavior of Python in three subtle
ways.  First, it adds the same two changes in the C API behavior
that "lazy concatenation" does: requiring use of the accessor
macro/function, and stipulating that these can now fail.

Lazy slices also add a third wrinkle, a change to Python's
memory allocation behavior: slicing a string means the
slice holds a reference to the original string.  If the slice
is long-lived, that means the original string becomes long-lived.
This can produce undesirable memory consumption when a program
contains long-lived small slices to short-lived big strings. In
this patch's defence, I point out there are other scenarios when
this behavior would actually result in *less* overall memory use,
mainly when taking multiple long-lived slices of a single string.
Also, it is possible to tune your memory use of lazy slices by
explicitly forcing the slice to render.  Still, the worst-case
behavior here is bad enough to give definite pause.  (As We Shall
Soon See.)

Also, as with lazy concatenation, lazy slices add complexity to
PyUnicodeObject, though again this is all hidden within
unicodeobject.c.

With this patch in place, Unicode string slicing is approximately
110% faster.  The Pybench "UnicodeSlicing" benchmark finishes in
less than 1/2 the time when compared to an unpatched Py3k.


GUIDO'S EXAMINATION

Guido examined a "rollup" patch that contained both lazy slices
and lazy concatenation.  His major criticism of the patch was
the "third wrinkle" added by lazy slices, where long-lived small
slices of short-lived big strings cause the big strings to become
long-lived.  Here's his test case:
  a = []
  while True:
      x = u"x"*1000000
      x = x[30:60]  # Short slice of long string
      a.append(x)

An unpatched Py3k can run all day doing this; add the "lazy slice"
patch and it quickly throws a MemoryError (after 1035 iterations
on my XP box).

Guido was unwilling to accept a patch with this behavior.
He agreed (via a short email exchange) that more discussion
on the Py3k list was a good idea.


"V2 LAZY SLICES"

After meditating under a waterfall on a distant mountaintop,
I concieved of an alternate implementation of lazy slices that
wouldn't have this memory behavior while still preserving much
of the benefit of the original lazy slices patch.  I call this
approach "V2 lazy slices".

The implementation is, in a word, complicated.  In V2 lazy
slices, a lazy slice is still a placeholder, with a pointer
back to the original string.  However, it does *not* retain
an official Python reference to the original string object--
ob_refcnt is unchanged.  Instead, there is a sinister second
internal reference, hidden from the outside world.  The original
string maintains a circularly-linked list of all slices from it.
When the original string is dealloc()'d, it walks this list and
forces each slices to render.  As if that weren't enough, the
code must handle some gnarly boundary cases in low-memory
conditions.  This approach necessitated a change to the object
structure; no longer are lazy slices (and lazy concatenations)
internal subtypes of PyUnicodeObject.  Instead, a PyUnicodeObject
has a pointer to a "PyUnusualUnicodeObject" structure which reflects
the string's "unusual" state:
  * I am a lazy concatenation.
  * I am a ("V2") lazy slice.
  * I am a conventional string, but lazy slices point to me.
Apart from this change in reference counting and storage, the V2
lazy slice patch is much the same as the original lazy slice patch.

As before, V2 lazy slices change the behavior of the Python
C API in the same two ways that "lazy concatenation" does:
requiring use of the accessor macro/function, and stipulating
that these can now fail.  It does *not* share the change
in memory consumption added by the original "lazy slices" patch;
Guido's test case above now runs for as many iterations as you're
willing to sit around for.  It *does* still hide all this complexity
inside unicodeobject.c.


There is one more wrinkle to consider.  Consider the following:
        def isNotFoo(fileHandle):
                a = unicode(fileHandle.readline())
                x = a.strip()
                return x != u"foo"
Here we allocate the string a, then the V2 lazy slice of a.
When the function exits, it pops the frame, and the locals are
freed.  But!  "a" is freed *before* "x", which means the Unicode
object gets dereferenced, which means x is forced to render--
even though it's about to be freed.  So this rendering is a
waste of time.

I asked on Python-Dev about this.  The consensus I got was:
 * In CPython locals are freed in the order they were added
   to the dictionary, and
 * this behavior is not a defined part of Python and should
   not be relied on.

So I did what any loser hacker would do: I reversed the order.
Now local variables are freed in the reverse order of their
being added to the locals dictionary.  Obviously this does not
guarantee that all local slices will be freed before their local
original string is.  But I *suspect* that it's better for V2 lazy
slices than the other order, and I've been assured that nobody
can rely on the order here, so it should be a safe (cheap) hack.
This change certainly doesn't seem to break anything.


LOCAL FREELIST RETOOLING

While working on the V2 lazy slice patch, I realized that the
local freelist code in unicodeobject.c could use some renovation.
To make a long story slightly less long, I revamped it to have four
freelists: one for PyUnicodeObjects themselves, one for very short
string buffers, and two for the specialized objects I use with lazy
evaluation.  Even if we don't keep any of the rest of my patches,
I suspect this would be of interest.  I speculate that it will make
Python *slightly* faster and use *slightly* less memory, but I admit
I have not tried it in isolation.


THE NET RESULT

I have a Py3k with "V2 lazy slices", "lazy concatenation" (retooled
to mate better with the V2 lazy slices), and the "local freelist
retooling", all applied at once.  Preliminary benchmarks from one
run of Pybench, 10 rounds at "warp 10", on my main XP box:
        "ConcatUnicode":           249.1% faster
        "CreateUnicodeWithConcat":  72.0% faster
        "UnicodeSlicing":           90.2% faster
        Overall:                     3.3% faster
I would be remiss if I didn't point out: these tests are really
just microbenchmarks.  "UnicodeSlicing" creates 49 slices of the
same string, then throws them away--neither examining the slices'
values nor storing them in a local variable.  The effect of these
patches on real-world code is as yet untested as far as I know.

I haven't published this rollup patch because it flunks the
regression tests right now.


THE POLLS

Finally, the polls.

#1: Should Py3k incorporate the "lazy concatenation" patch?

#2: Should Py3k incorporate the original "lazy slices" patch?

#3: Should Py3k incorporate the "V2 lazy slices" patch?

#4: Assuming you are +0 or better on #3, is it okay to change the
    frame object such that locals are released in the reverse order
    of their being added to the dictionary?

#5: Should Py3k incorporate the "local freelist retooling" patch?


My votes are as follows:

1: +1
2: -1
3: +0.5
4: +1
4: +1

The reason I'm not fully behind "V2 lazy slices" is because they're
just so gosh-darned complicated.  Yes, it *is* faster, but at the
cost of a lot of possible programmer puzzlement.  My feeling is that
we Python core developers are willing to shoulder the brunt of a certain
amount of complexity to make the lives of Python users happier, but this
only goes so far.  Really I only implemented them because of a misguided
dogged relentlessness.  But I don't claim to have my finger on the pulse
of the Python core development community; perhaps you'll surprise me with
a "damn the complexity, full speed ahead" attitude and express great
interest.

If you'd prefer to see the "V2 lazy slices" patch before passing sentence,
post a request to the list (or email me) and I'll make one happen.

I await your judgment,


/larry/

From jcarlson at uci.edu  Wed Jan 31 19:58:13 2007
From: jcarlson at uci.edu (Josiah Carlson)
Date: Wed, 31 Jan 2007 10:58:13 -0800
Subject: [Python-3000] Poll: Lazy Unicode Strings For Py3k
In-Reply-To: <45C07234.8070808@hastings.org>
References: <45C07234.8070808@hastings.org>
Message-ID: <20070131093831.5A53.JCARLSON@uci.edu>


Larry Hastings <larry at hastings.org> wrote:
> Lazy concatenation changes the behavior of Python in three subtle
> ways.  First, it adds the same two changes in the C API behavior
> that "lazy concatenation" does: requiring use of the accessor
> macro/function, and stipulating that these can now fail.

I presume the second word in the above paragraph shuold read "slicing".


> Lazy slices also add a third wrinkle, a change to Python's
> memory allocation behavior: slicing a string means the
> slice holds a reference to the original string.  If the slice

Perhaps I missed something about the concatenation implementation, but
in order to prevent the rendering of lazy concatenation objects,
shouldn't you need to keep a reference and pointer to the left and right
strings/concatenation objects?  This isn't the same as a (small) slice
holding onto a reference to a (big) string, but there is still an object
lifetime consideration.


> I asked on Python-Dev about this.  The consensus I got was:
>  * In CPython locals are freed in the order they were added
>    to the dictionary, and
>  * this behavior is not a defined part of Python and should
>    not be relied on.

There isn't a locals dictionary, there is a locals array.  locals()
creates a mapping of names -> values based on the content of
func_code.co_varnames and the locals array.   The order of deletion is
that which the locals array is defined, which is based on when the
*names* are seen by the parser and compiler, which is (I believe), the
order listed in func_code.co_varnames .


> So I did what any loser hacker would do: I reversed the order.
> Now local variables are freed in the reverse order of their
> being added to the locals dictionary.  Obviously this does not
> guarantee that all local slices will be freed before their local
> original string is.  But I *suspect* that it's better for V2 lazy
> slices than the other order, and I've been assured that nobody
> can rely on the order here, so it should be a safe (cheap) hack.
> This change certainly doesn't seem to break anything.

The performance of that change is negated when confronted with the
following:

        def isNotFoo(fileHandle):
                x = None
                a = unicode(fileHandle.readline())
                x = a.strip()
                return x != u"foo"

I don't suspect that variants of the above is common, but it is a gotcha
for your change.


> THE POLLS
> 
> Finally, the polls.
> 
> #1: Should Py3k incorporate the "lazy concatenation" patch?
> 
> #2: Should Py3k incorporate the original "lazy slices" patch?
> 
> #3: Should Py3k incorporate the "V2 lazy slices" patch?
> 
> #4: Assuming you are +0 or better on #3, is it okay to change the
>     frame object such that locals are released in the reverse order
>     of their being added to the dictionary?
> 
> #5: Should Py3k incorporate the "local freelist retooling" patch?


You are probably not going to be surprised about this.

#1, #2, #3; -1

For me, the killer is the added complexity that the patches introduce. 
In optimizing my own code (changing algorithms, data structures, etc.),
there is a certain level of algorithm and structural complexity that I
am willing to endure in order to improve performance.  Beyond that level
I usually say "it's not worth it".

Are the performance improvements you are proposing worth the added
complexity of the Unicode object?  I suspect not.  But I'm tossing my
hat in the "No" ring because I've become fairly conservative when it
comes to changes in Python.

If people go with a selection that includes 3, I don't see any reason to
not reverse the order of the freeing.  The gotcha I provided isn't
terribly convincing, even to me.

I believe improvements to the freelist(s), assuming that they are a
relatively minor, would also be fine.


 - Josiah

P.S. One of the reasons why I have been pushing for a wrapper, is
primarily because I believe that the added complexity to the *base type*
is too much, while a wrapper object would be free to do just about
anything (with a sufficiently restricted meaning of 'anything').  Yes,
users would need to explicitly wrap unicode objects.  It's not ideal,
and would limit their use, but for those who have particular needs or
desires (and/or would want to use them with Python 2.x), it could be
helpful.

P.P.S. I'm not certain, but I believe the V2 lazy slicing could
be cleaned up with the use of weak references.


From jimjjewett at gmail.com  Wed Jan 31 20:56:40 2007
From: jimjjewett at gmail.com (Jim Jewett)
Date: Wed, 31 Jan 2007 14:56:40 -0500
Subject: [Python-3000] Poll: Lazy Unicode Strings For Py3k
In-Reply-To: <20070131093831.5A53.JCARLSON@uci.edu>
References: <45C07234.8070808@hastings.org>
	<20070131093831.5A53.JCARLSON@uci.edu>
Message-ID: <fb6fbf560701311156u57dbcf84od4d5c7396b635a96@mail.gmail.com>

On 1/31/07, Josiah Carlson <jcarlson at uci.edu> wrote:
> P.S. One of the reasons why I have been pushing for a wrapper, is
> primarily because I believe that the added complexity to the *base type*
> is too much, while a wrapper object would be free to do just about
> anything (with a sufficiently restricted meaning of 'anything').

What exactly do you mean by a wrapper?

Are you thinking of something like a 3rd-party library providing a
string-like object which doesn't claim to inherit from str, and has a
subtly different API?  That seems to get the bad from both choices,
and the good from neither.

-jJ

From jcarlson at uci.edu  Wed Jan 31 21:37:31 2007
From: jcarlson at uci.edu (Josiah Carlson)
Date: Wed, 31 Jan 2007 12:37:31 -0800
Subject: [Python-3000] Poll: Lazy Unicode Strings For Py3k
In-Reply-To: <fb6fbf560701311156u57dbcf84od4d5c7396b635a96@mail.gmail.com>
References: <20070131093831.5A53.JCARLSON@uci.edu>
	<fb6fbf560701311156u57dbcf84od4d5c7396b635a96@mail.gmail.com>
Message-ID: <20070131122426.5A5C.JCARLSON@uci.edu>


"Jim Jewett" <jimjjewett at gmail.com> wrote:
> 
> On 1/31/07, Josiah Carlson <jcarlson at uci.edu> wrote:
> > P.S. One of the reasons why I have been pushing for a wrapper, is
> > primarily because I believe that the added complexity to the *base type*
> > is too much, while a wrapper object would be free to do just about
> > anything (with a sufficiently restricted meaning of 'anything').
> 
> What exactly do you mean by a wrapper?
> 
> Are you thinking of something like a 3rd-party library providing a
> string-like object which doesn't claim to inherit from str, and has a
> subtly different API?  That seems to get the bad from both choices,
> and the good from neither.

Do you remember my "string view" post from last September/October or so? 
It implemented almost all of the string API exactly as the string API
did, except that rather than returning strings, it returned views.  The
portions that didn't conform were the portions I didn't implement. 
Implementing the single segment buffer interface made it usable to be
passed to socket.send(), file.write(), etc., and implementing a proper
__str__ method made the objects usable as a source perhaps when passing
to some other API that *needed* a str.

The point of such a lazy concatenation/slicing wrapper, derived from
Larry's patches, would be to offer the exact same API as unicode.
Basically anywhere you could use unicode before, you could use the new
object.  If you _needed_ to have a unicode object, you would use
unicode(obj) (or whatever it is renamed to in Python 3).

Certainly it could be distributed as a 3rd party module (I would
consider downloading and installing it), but I believe the point of it
would be to include it with Python, maybe accessable via 'import string',
'import collections', or somewhere else.


 - Josiah


From jimjjewett at gmail.com  Wed Jan 31 21:49:50 2007
From: jimjjewett at gmail.com (Jim Jewett)
Date: Wed, 31 Jan 2007 15:49:50 -0500
Subject: [Python-3000] Poll: Lazy Unicode Strings For Py3k
In-Reply-To: <45C07234.8070808@hastings.org>
References: <45C07234.8070808@hastings.org>
Message-ID: <fb6fbf560701311249q6d243419i36d39c1e254874f9@mail.gmail.com>

On 1/31/07, Larry Hastings <larry at hastings.org> wrote:
> Lazy concatenation changes the behavior of the Python C API in two
> subtle ways:

> 1) All C API users asking for the value of a string *must* use
>    the macro PyUnicode_AS_UNICODE() or the function
>    PyUnicode_AsUnicode().  It is no longer permissable to
>    directly access the object's "str" member.
> 2) It is now possible for PyUnicode_AS_UNICODE() and
>    PyUnicode_AsUnicode() to *fail*,  returning NULL under
>    low-memory conditions.

(Note that it might be possible to wrap the null-check inside an
access macro in a manner similar to Brett's object capability API.)

I believe the str API should make these changes **even if these
patches are not applied.**

The code-point representation issues alone are contentious enough that
I don't think it is possible to satisfy everyone with a single
implementation.

Even for the default implementation, we might want to change some of
the tradeoffs once unicode-for-everything is in normal use.

Given that, any of your three options (as well as the no-change) are
legitimate implementations, and the question is which to use by
default in 3.0, knowing that it could change with a minor (but not
bugfix) version.  I would be inclined to go with lazy slicing #1, but
the particular choice is less important than the API change.

-jJ

From jimjjewett at gmail.com  Wed Jan 31 22:14:04 2007
From: jimjjewett at gmail.com (Jim Jewett)
Date: Wed, 31 Jan 2007 16:14:04 -0500
Subject: [Python-3000] Poll: Lazy Unicode Strings For Py3k
In-Reply-To: <20070131122426.5A5C.JCARLSON@uci.edu>
References: <20070131093831.5A53.JCARLSON@uci.edu>
	<fb6fbf560701311156u57dbcf84od4d5c7396b635a96@mail.gmail.com>
	<20070131122426.5A5C.JCARLSON@uci.edu>
Message-ID: <fb6fbf560701311314s6cb16ba4l263c156de5c331f2@mail.gmail.com>

On 1/31/07, Josiah Carlson <jcarlson at uci.edu> wrote:
> "Jim Jewett" <jimjjewett at gmail.com> wrote:
> > On 1/31/07, Josiah Carlson <jcarlson at uci.edu> wrote:

> > > ... I believe that the added complexity to the *base type*
> > > is too much, while a wrapper object would be free to do just about
> > > anything

> > ... string-like object which doesn't claim to inherit from str, and has a
> > subtly different API?  That seems to get the bad from both choices,

> Do you remember my "string view" post from last September/October or so?
> It implemented almost all of the string API exactly as the string API
> did, except that rather than returning strings, it returned views.

So there would be places where you couldn't safely use it, even though
it had all the required functionality.

How would you feel if it also

(1)  Claimed to be a subclass of str (though it might not actually
inherit anything)
(2)  Implemented the rest of the methods by delegation.  (Call str on
itself, switch its "real" object to the new string, and delegate to
that.)

If these slight extensions are OK, then we agree that it should be
possible to use multiple implementations of str, and I don't see the
point of insisting that your strview isn't a real string.  (And then
we move to the next step, where I ask that the API stop promising to
preallocate the data in a particular place.)

-jJ

From paul at prescod.net  Wed Jan 31 22:18:46 2007
From: paul at prescod.net (Paul Prescod)
Date: Wed, 31 Jan 2007 13:18:46 -0800
Subject: [Python-3000] Poll: Lazy Unicode Strings For Py3k
In-Reply-To: <45C07234.8070808@hastings.org>
References: <45C07234.8070808@hastings.org>
Message-ID: <1cb725390701311318n21f57a7et95aef7d41dd8f130@mail.gmail.com>

String concatenation is a known issue in Python programming and
workarounds for it are common obfuscations in a language otherwise
famous for being clean. So I vote +1 on it. I abstain on slicing.

 Paul Prescod

From ncoghlan at gmail.com  Wed Jan 31 22:52:49 2007
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 01 Feb 2007 07:52:49 +1000
Subject: [Python-3000] Poll: Lazy Unicode Strings For Py3k
In-Reply-To: <fb6fbf560701311156u57dbcf84od4d5c7396b635a96@mail.gmail.com>
References: <45C07234.8070808@hastings.org>	<20070131093831.5A53.JCARLSON@uci.edu>
	<fb6fbf560701311156u57dbcf84od4d5c7396b635a96@mail.gmail.com>
Message-ID: <45C10FB1.40209@gmail.com>

Jim Jewett wrote:
> On 1/31/07, Josiah Carlson <jcarlson at uci.edu> wrote:
>> P.S. One of the reasons why I have been pushing for a wrapper, is
>> primarily because I believe that the added complexity to the *base type*
>> is too much, while a wrapper object would be free to do just about
>> anything (with a sufficiently restricted meaning of 'anything').
> 
> What exactly do you mean by a wrapper?
> 
> Are you thinking of something like a 3rd-party library providing a
> string-like object which doesn't claim to inherit from str, and has a
> subtly different API?  That seems to get the bad from both choices,
> and the good from neither.

FWIW, there's prior art for something like Josiah's approach in the form 
of Java's StringBuilder & StringBuffer classes [1].

A mutable container is used to build the string, then converted to a 
normal immutable string when complete (this is similar to the intended 
use of collections.defaultdict, with the default in place during 
initialisation, but removed before the dictionary is given to any other 
part of the program).

Cheers,
Nick.

Cheers,
Nick.

http://java.sun.com/j2se/1.5.0/docs/api/java/lang/StringBuilder.html



-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From jcarlson at uci.edu  Wed Jan 31 23:00:50 2007
From: jcarlson at uci.edu (Josiah Carlson)
Date: Wed, 31 Jan 2007 14:00:50 -0800
Subject: [Python-3000] Poll: Lazy Unicode Strings For Py3k
In-Reply-To: <fb6fbf560701311314s6cb16ba4l263c156de5c331f2@mail.gmail.com>
References: <20070131122426.5A5C.JCARLSON@uci.edu>
	<fb6fbf560701311314s6cb16ba4l263c156de5c331f2@mail.gmail.com>
Message-ID: <20070131133737.5A61.JCARLSON@uci.edu>


"Jim Jewett" <jimjjewett at gmail.com> wrote:
> On 1/31/07, Josiah Carlson <jcarlson at uci.edu> wrote:
> > "Jim Jewett" <jimjjewett at gmail.com> wrote:
> > > On 1/31/07, Josiah Carlson <jcarlson at uci.edu> wrote:
> > > > ... I believe that the added complexity to the *base type*
> > > > is too much, while a wrapper object would be free to do just about
> > > > anything
> 
> > > ... string-like object which doesn't claim to inherit from str, and has a
> > > subtly different API?  That seems to get the bad from both choices,
> 
> > Do you remember my "string view" post from last September/October or so?
> > It implemented almost all of the string API exactly as the string API
> > did, except that rather than returning strings, it returned views.
> 
> So there would be places where you couldn't safely use it, even though
> it had all the required functionality.

Almost certainly, but the point is that you could get back to what you
wanted via str(obj), unicode(obj), etc., which would incur (in the worst
case) the overhead you saved before, or raise a MemoryError exception 
(unless its linux, in which case you will likely segfault).


> How would you feel if it also
> 
> (1)  Claimed to be a subclass of str (though it might not actually
> inherit anything)
> (2)  Implemented the rest of the methods by delegation.  (Call str on
> itself, switch its "real" object to the new string, and delegate to
> that.)

I'm not terribly concerned about the implementation details of an object
I don't need to use.  As long as it works, it is fine.  I am concerned
about the implementation details of objects I will use.


> If these slight extensions are OK, then we agree that it should be
> possible to use multiple implementations of str, and I don't see the
> point of insisting that your strview isn't a real string.  (And then
> we move to the next step, where I ask that the API stop promising to
> preallocate the data in a particular place.)

I made the strview not a real string out of convenience of
implementation and what I knew about Pyrex at the time.  Whether or not
some *other* implementation of concatenation or view wrapper claims to
be a str subclass is up to them.

I believe the base type included with Python should allocate the memory
on creation.  Why?  Because the implementation is simple, and I believe
that a base type implementation should be as simple as possible.

    Simple is better than complex.
    If the implementation is hard to explain, it's a bad idea.
    If the implementation is easy to explain, it may be a good idea.

Are they simple?  Hard or easy to explain?  I don't know.  But
view/concatenations are *more complex* than what we have, and are
*harder to explain*.


 - Josiah


From larry at hastings.org  Wed Jan 31 22:59:03 2007
From: larry at hastings.org (Larry Hastings)
Date: Wed, 31 Jan 2007 13:59:03 -0800
Subject: [Python-3000] Poll: Lazy Unicode Strings For Py3k
In-Reply-To: <20070131093831.5A53.JCARLSON@uci.edu>
References: <45C07234.8070808@hastings.org>
	<20070131093831.5A53.JCARLSON@uci.edu>
Message-ID: <45C11127.30000@hastings.org>


Josiah Carlson wrote:
> Perhaps I missed something about the concatenation implementation, but in order to prevent the rendering of lazy concatenation objects, shouldn't you need to keep a reference and pointer to the left and right strings/concatenation objects?  This isn't the same as a (small) slice holding onto a reference to a (big) string, but there is still an object lifetime consideration.
>   
You didn't miss anything--your summary is correct.  The lazy 
concatenation object holds references to both the left and right 
objects.  Since this isn't wasting memory, and it doesn't change 
Python's memory allocation behavior in any significant way (apart from 
the aforementioned able-to-fail APIs), I omitted it.

> P.P.S. I'm not certain, but I believe the V2 lazy slicing could be cleaned up with the use of weak references.
That's an interesting point.  They would simplify the implementation 
slightly, though perhaps not as much as you'd hope.  I think I'd still 
need much of the existing code in order to solve the really nasty 
boundary case (when the original string goes away during low-memory 
conditions).

Also consider: weak references would add their own overhead to the 
implementation.  In particular, weakref callbacks are on Python objects, 
so it always builds a tuple and calls via PyObject_Call().

In the final analysis I'm not sure a weakref-based implementation would 
make V2 lazy slicing any more palatable to the discerning Py3k core dev 
community.


/larry/